Dear Data Analysts, let's talk about your backlog

If you’re a data analyst, you’re likely going to leave your job in the next year. You play a mission-critical role in your organization and are a versatile data expert who can analyze, code, and solve business problems. And yet, you and your peers have one of the highest turnover rates, and it’s not surprising.

Analysts confront never-ending backlogs on a daily basis. As an analyst, you’re expected to communicate well with business partners, manage internal processes, and deliver quality products quickly (“we need this ASAP!”), all the while juggling various requests. Moreover, the work is often repetitive and redundant since most of it has been done before by you or your teammates, but not saved in an accessible location. All of this makes the role highly demanding and results in a backlog that weighs heavily on your shoulders and your happiness.

These issues affect you and your company’s performance, and more importantly, agitate you in your day-to-day work. Frustrated by this reality, we identified the top challenges that cause this logistical nightmare:

Capturing and Managing Data Tasks

Every morning, you probably open Trello to track personal tasks and platforms such as Monday.com, Asana, or Jira to collaborate with your team. Some requests also come in through Slack, email, or via a friendly ping on Google Hangouts. You likely deal with requests as they arrive or try to keep track of them on the fly (did anyone see a pen and paper?). Adding more noise, stakeholders and business partners might also text, call, or drop by your desk with “quick questions” at any time. These haphazard interactions often make it difficult to ask proper questions that gather relevant context required to solve the problem.

Wide Variety of Tasks

Throughout the day, some requests that fall on your plate are ad-hoc, which result in you creating one-off reports, tables, and analyses. Some are meta-questions about your work like “how-to’s” (“how do I filter in this report?”), while others may be repetitive debugging requests such as adjusting an SQL query to a recently updated schema. And if you’re lucky, you’re also managing more complex work that involves modeling and building pipelines. Juggling these many types of tasks can naturally become confusing and requires a great amount of effort, especially when “simple” tasks become the most time-consuming.

Repetitive and Redundant

During your weekly team meeting, you find out that a few months ago, Greg created a Looker view that is similar to what you've been working on tirelessly for the past few days. Meanwhile, Mary has already completed one of your requests on Snowflake but didn’t save it in an accessible path. It’s not the first time you find out that you’re doing redundant work. Can you guess how many tickets in your queue are almost identical to tasks that have been resolved in the past?

Too Many Tools

After all that, you finally get down to work. You structure a SQL query for your current task in your favorite IDE and run it on your database or data warehouse. Alternatively, you write python code in Sublime, even though you’ve been told multiple times that you should switch to PyCharm. To build your reports you need to pull and export data from BigQuery to Tableau, something that could take hours. You heard that a quick solution is to connect BigQuery to Google Data Studio or create a one-off Excel analysis, but this is harder to document and is easily forgotten. Did we mention your boss is thinking of switching to Power BI?

Lack of Data Discovery

You finally finish the work itself. Ideally, it should become a valuable asset for your company. But in order for your time investment to have a strong return, it should be utilized in the future. For that, you must make sure it’s easily accessible to you and your colleagues. How do you save, share, and obtain your assets today? Some analysts have a Google Doc with their most common work and SQL queries. Some use Github. Others document them as status updates in a random project management tool, while the fortunate few use BI tools that have version control and cataloging. We can only hope that the company’s assets are documented somewhere and that the owner hasn’t left the company yet, or else it’s all lost. Unfortunately, this situation is more common than not, leaving you to deal with more redundant work yet again.

So, how was your day?

By the end of it, you barely had time to look over the new data project you just received or to dive into that one problem you’ve been working on for over a month. The complexities, frustrations, and inefficiencies keep you from doing your best work - detracting from your time to think critically, dive deeper into analyses, and discover the best insights that help your team and business partners succeed.

With all of these roadblocks, it takes longer to deliver insights and answers to your business partners. As a result, they become exasperated from waiting and often take out their frustration on you, instead of recognizing the hard work.

Many companies have acknowledged this problem. Their solution? Hiring more analysts to lighten the workload, but that’s not scalable. Other companies try hacking together even more existing tools into the mix, but that doesn't address the root problem.

A Better Tomorrow

At Rupert, we imagine a different world. A world where analysts have proper support every step of the way. Our solution clears out the backlog that frustrates and blocks analysts from focusing on what matters most. It delivers data assets to analysts and their stakeholders when needed. It learns and automates work processes to eliminate redundancies, allowing analysts to push their companies forward, and makes work more enjoyable.

If you would like to be a part of this world, join our beta.