Trends in SQL & data tools

What’s transforming SQL & data tools

The landscape of SQL tools is far from static 💫 These are the forces shaping the SQL and data tools of the future:

• Next-gen exploratory tools threaten rigid legacy tools
• More roles are writing SQL than ever before
• Collaboration saves time and reduces cost
• The cloud is a given (but it's not cheap)
• Isolation puts your team at risk
• Remote work is here to stay

Next-gen exploratory tools threaten rigid legacy tools

The decline of rigid, centralized dashboards has accelerated in recent years, with industry experts such as Gartner highlighting it as a top trend. Data tools can no longer be buffet-style (i.e., have as much data as you like as long as it's already been instrumented and lives on this dashboard) ⛔

Instead, Gartner predicts a "shift to in-context data stories [where] the most relevant insights will stream to each user based on their context, role, or use." Put simply, different users have different data needs at different times. Tools that slow down data exploration and require constant updates to make information available will continue to fall by the wayside.

More roles are writing SQL than ever before

The stereotype has long been that only data analysts and data scientists write SQL. While they are indeed power users, this pigeonholing ignores the diversity of roles writing SQL. To prove it, we looked at the roles of our own active users:

This trend is much broader than just PopSQL. Non-traditional roles are rushing toward SQL because of time pressure, an urgency to explore new business lines, and a hunger for personal growth 🌱

Collaboration saves time & reduces cost

Gartner also reported that "through 2023, data scientists and analysts will lose 60% to 70% of their productive time to activities like finding, preparing, integrating and sharing datasets" 🤯

Your most valuable data experts waste most of their time slogging through data prep and hunting for reliable data sources.

Tools that minimize this drudgery quickly pay for themselves. On the data prep side, data transformation tools — such as dbt — have exploded in popularity to help combat these issues.

However, even once data is ready for analysis, when it comes time to write SQL, data teams are wasting time. People are searching across Slack, Notion, Google Drive, SharePoint, and more for a “blessed” query to no avail. They’re rewriting each others’ queries from scratch. They’re jumping on unnecessary Zoom calls to debug code or Slacking each other repetitive, low-level questions about data models. Tools such as PopSQL help eliminate this additional collaboration overhead. The problem stems from not having a standardized collaborative SQL workspace to stay on the same page, much like how engineers have GitHub, designers have Figma, and project managers have ClickUp.

The cloud is a given (but it's not cheap)

"By 2022, public cloud services will be essential for 90% of data and analytics innovation." That’s from Gartner again. However, "data and analytics leaders need to focus on cost optimization when moving to cloud."

Cloud data warehouses like Snowflake and BigQuery charge for data processed (i.e., queries run) 💸 The simplest way to reduce cloud costs is to reduce queries that are wastefully re-run. Collaborative SQL editors such as PopSQL allow users to run a query once and then share those results. Everyone gets access, but without the redundant runs.

And remember: Even with this massive shift to the cloud, teams still have real-time production databases (MySQL and PostgreSQL being the two most common). Collaborative SQL editors such as PopSQL support all major databases and data warehouses, keeping users in just one tool as opposed to siloed in fragmented tools.

Isolation puts your team at risk

Nearly 80% of data professionals are considering leaving their jobs. And studies show a “staggering turnover rate” among data scientists.

Why all the turmoil? In a Towards Data Science post titled "Here’s why so many data scientists are leaving their jobs," a data science leader at Deliveroo warned that "data scientists quit their jobs because they feel isolated." More than 50,000 members of the data community clapped their hearts out at this post 👏 Clearly the sentiment resonated.

It’s acknowledged that "organizing isolated teams to work on collaborative projects in large enterprises is not easy." There’s no simple solution to address this problem. But tools that bring teammates together around their data certainly reduce that feeling of isolation.

Remote work is here to stay

Remote collaboration has become a fact of life 💻 And research shows that it’s changed how most people collaborate and communicate.

The question is: How should you respond? One of the most comprehensive guides to remote work on the internet stresses the importance of choosing the right tools.

Why collaboration is key to data tools

The data tools of the future trend toward exploration, as more roles have niche data needs and can't wait. With more individuals pulling their own data, there's ample opportunity for collaboration (and time savings if equipped with collaborative tools).

But collaboration remains a challenge 🛑 Learn what’s stopping data collaboration.