The Ultimate Coda Handbook for Engineering Teams

Explore

The Ultimate Coda Handbook for Engineering Teams

The Ultimate Coda Handbook for Engineering Teams

2. Engineering Analytics

Making informed decisions and moving faster with the right analytics.

Data is critical to making the right decisions, particularly when it comes to deciding where to spend your team’s time. But too often you either don’t have the data or you have too much data that cannot be presented at a helpful level of granularity. When faced with the latter, the question isn’t “where can I find the data?” but instead “how do I make sense of these different pieces?”

In my experience, the missing link stopping most engineering teams from making more informed decisions is having a platform that is broadly flexible enough to analyze countless data sources while not being too fragile or opinionated. Teams have data from 10s or 100s of sources, and the types of questions they need to answer vary wildly. So the key is being able to explore freely, looking at data in the way that works for your team.

⁠

Four categories of engineering leadership blind spots.

After hundreds of conversations with engineering leaders, I’ve found that there are four categories of blind spots where engineering leaders struggle to find clarity in their decision making:

Execution

Answering these questions can be very time intensive for engineering leaders not equipped with the right tools.

Who is working on what right now?

How are we progressing on our OKRs?

Which OKRs are stuck and need my help?

How much off are our plans from actual execution?

Quality

Managing quality and code debt is an important aspect of an engineering leader’s job, but it requires looking at many metrics to balance against new project work.

What is our incident rate and how are progressing on closing our postmortem issues?

Are bugs out of control in one team or area?

Which components are in the worst shape?

Are teams upholding their triage and bug fix SLAs?

What percentage of time are engineers spending fixing bugs or maintaining old features?

Projects

Leaders often shoulder uncertainties around project timeframes, tradeoffs, and broader team productivity.

What’s our feature velocity across teams?

What are the engineering costs of a specific project?

What is the burndown chart across teams?

What’s the ETA for this project?

Teams

It’s difficult to obtain objective metrics on engineering productivity and pipeline—you need to look at candidates and engineers holistically, not just their output.

How are our new-hires ramping up? Who might be struggling? Any early signs of a superstar?

Who are my most productive engineers and what can we learn from them?

Which teams are lacking senior developers?

Is our hiring pipeline meeting diversity goals?

How much new headcount will I have next quarter?

⁠

Existing tools struggle to create actionable insights.

Most engineering tools are powerful, yet incredibly rigid, so most leaders employ one of two strategies to mitigate their blind spots:

Leverage traditional docs and sheets to gain direction. Leaders often return to familiar documents and spreadsheets because they offer a convenient way to import data from multiple sources and perform calculations to aid in decision-making. However, errors can easily occur and it can be frustrating to begin again when faced with the same question in the future.

Purchase specific tools to help fill the gaps. There are countless tools that target specific challenges, ranging from software capitalization to tracking diversity within the hiring process. However, it's important to note that most of these tools come with a hefty price tag for their distinctive—and limited—features. As a result, it's common to accumulate a long list of new tools and corresponding invoices.

I’ve observed this theme in engineering and non-engineering teams alike over the years, and we’ve evolved the Coda platform to be the connective tissue across many tools. Doing so allows leaders to customize dashboards that report on their most critical metrics, join multiple data sources in one surface, and avoid learning curves associated with special purpose analytics tools.

⁠

Why engineering teams choose Coda to support their decision making.

The term “engineering analytics” can cover a wide range of use cases and end products, but for the sake of discussion, let’s simply define it as the process of summarizing engineering-related data sources to uncover insights that help your team make better decisions.

Interestingly, many engineering leaders don’t realize how reliant they are on traditional documents and spreadsheets for this type of summary and analysis until you point it out. They see the dedicated analytics tools, but not the long tail of purpose-built docs and sheets. So it follows that many leaders, even those already using Coda, don’t immediately think of it as an analytics or dashboard tool. But when I’ve shown other engineering leaders the analytics dashboards I use to run the Coda engineering team, it almost always clicks right into place. Coda is as flexible as it is powerful, allowing you to tailor it to your team’s unique needs:

1. A shared platform that connects to all your tools. No more importing stale data.

What so uniquely cements Coda as a perfect tool for engineering analytics is its ability to connect to literally 100s of data sources. This means that a single platform can give you a lens into pretty much every data source you could hope to analyze.

The only real alternative that provides the same ability to analyze data from hundreds of sources is a spreadsheet. But copy-pasting data into sheets is not at all the same as syncing data into Coda. Your data goes stale as soon as you’ve pasted it in, meaning you need to start from scratch every time you need to answer the same question.

With Coda, you get the same benefit of having a single platform to explore your data as a sheet—a single formula language to learn, a single charting system to know, a single login to remember—but your dashboards become living, connected dashboards that grow with you.

2. The flexibility of a spreadsheet, the power of an app. Explore with ease and confidence.

Purpose-built tools are unanimously rigid in structure and capability. Coda, by contrast, is flexible and scalable.

Let’s take a very simple example. Say you want to know the execution velocity of your various teams across your engineering organization. If you use Jira, you probably will immediately turn to a burndown chart. While this native capability is extremely useful to understand how fast a given epic is being worked on, it is extremely rigid and limited in its capabilities. You simply cannot look at velocity across epics, projects, and teams.

Your only options are to paste data into a spreadsheet, or purchase a dedicated tool to analyze your Jira data. But it’s not hard to see that you might quickly end up with dozens of dedicated tools meant to serve only a single purpose. So even if each does their job well, you will have to learn dozens tools and pay dozens bills.

In Coda, you can pull all of your Jira data together and build a burndown that perfectly captures the group of projects or people you care about. Better yet, you only pay for a single license and you only have to learn a single tool.

Beyond the cost and learning curve, Coda offers the flexibility and simplicity that push engineers towards spreadsheets in the first place. You can build and customize powerful dashboards and analysis without a single line of code. You don’t need to brush up your SQL skills or ask the data team for help. You can use simple structured builders to customize to your heart’s content.

3. Supercharge your analysis with AI. Uncover and take action on key insights.

Collecting data is always only the first half of the journey when we are talking about engineering analytics. Second, and equally important, you need to summarize and mine your data to uncover insights. Without doing so, your data is simply a list of numbers.

This starts simple with charts and summarized numbers, giving you a broad overview into breakdowns, sums, and pacing over time. But as AI becomes a more prominent, and an unavoidable part of our working lives, there is much we can do to save time and energy in this process of “insight mining.”

Here are two simple examples of how using Coda AI can save you time and energy in your engineering analytics:

When you collect feedback from your team—say asking engineers qualitatively and quantitatively how much time they are spending on their on-call rotation—use Coda AI to summarize the feedback, extracting overall sentiment and key action items.

When analyzing tasks and bugs from the past quarter to understand velocity and bottlenecks, use Coda AI to auto categorize tasks by project or triage bugs by priority to better segment your data.

⁠

How to get started with Coda for engineering analytics.

By this point, I hope it’s clear that engineering analytics are a major source of value, however, they can also be both a time and energy suck for your team. I’d like to give some practical examples for how you can solve some of the problems in each of the four categories of blind spots I discussed above. For each, I will describe a specific template that will help you see past your blind spots and uncover insights from what lays beneath.

Beyond creating new dashboards, you should consider where the dashboards live. Here are a few options:

Create a private, personal engineering analytics dashboard. Organize all of your dashboards in a single doc and make it a personal dashboard, not widely shared. You can also pull in other information you want to track to this hub. For example, I like to track my own meeting load with a dashboard using the

Google Calendar Pack⁠

Add dashboards to your department hub, sharing with the entire team. The

Jira dashboard⁠

is probably the most useful dashboard for this scenario. In this case, I would use caution including the developer productivity dashboard, as this might incentivize gaming or destructive competition. You can get started with the

department hub⁠

discussed in

1. Team Hubs⁠

Include key dashboards in your 1:1 templates with your reports. You are probably using a 1:1 doc with each of your reports to keep notes. I’ve found that 1:1 meetings are more effective if we’re able to look at key dashboards together. I recommend picking a few of the dashboards from this section and adding them to your 1:1 doc. You can do this through

Cross-doc⁠

, but it’s probably easier to do so with a

full-page embed⁠

1. Team: Measure developer productivity with a code metrics dashboard.

Improving developer productivity often starts with identifying and asking your more productive engineers to share their learnings while also identifying your least productive engineers to help improve. Tracking productivity over time is even more valuable, as it can help uncover slowly-changing issues or underinvestments in training, tooling, or communication.

Productivity is a complex topic that spans beyond writing code and into responsibilities like writing design docs, conducting interviews, resolving customer tickets, communicating effectively with teammates, and more. Judging overall productivity is a difficult aspect of an engineering manager’s job, and there is no set of metrics that captures it perfectly. In fact, all metrics in this space only capture a part of productivity and can be gamed. Nonetheless, I have found it very helpful to measure and look at the basic code output metrics, not as a final answer to productivity, but as one of the inputs.

Particularly, I would recommend starting with number of pull requests (or change lists) created, lines of code committed, and number of code reviews conducted.

These metrics capture code production, but they need to be put in context for accurate interpretation. Each team and project might be in a different phase or have different levels of complexity. Also, each specific role will have different expectations. For example, security engineering will have a very different expected code output than front-end, product engineers.

Want to see what a more sophisticated productivity dashboard looks like in Coda?

Here’s⁠

a great example from our head of core product engineering, Himanshu Vasishth that allows you to compare code output across teams, engineers, and functions.

Once you have other engineering analytics dashboards in Coda, you can also enhance your productivity dashboard with metrics beyond code output like bugs assigned and fixed per person, story points assigned and finished, interviews conducted, and more. This will allow you to have a more holistic view of productivity.

2. Execution: Start simple with an OKR tracker.

I consider it a best practice to keep your planned goals or OKRs front and center during execution in order to remind the team what’s most important throughout the execution cycle. I’ll elaborate on this more in

4. Planning and OKRs⁠

, but here are a few different ways you can incorporate your OKRs into your day-to-day work, based on where you perform OKR planning:

For OKRs stored in a standalone OKR app, you can embed your chosen app into Coda with

full-page embeds⁠

. For many applications, you can make direct edits to the data through the embed—saving you time and keeping everything up-to-date. If you’re running your planning cycles in one of these apps, I’d highly recommend reading through

4. Planning and OKRs⁠

as I believe there are a variety of reasons why these apps do more harm than good.

For OKRs stored in a spreadsheet, you have a few options. With any online tool, you can use

full-page embeds⁠

. Or, if you’re using Google Sheets, you can use the

Google Sheets Pack⁠

to sync your OKRs into Coda. The Pack is complete with two-way sync, so you can send updates back to your OKR spreadsheet in near real-time. An additional benefit of using the Pack is that you can interact with the data as a Coda table. This means you can create views, build dashboards, and more.

For OKRs stored in a different Coda doc, you can

Cross-doc⁠

your OKRs table from your OKRs doc into your team hub, then filter out all OKRs that are not yours. From there, you can add charts, progress bars, callouts, and more to your OKRs page to make it clear how you are progressing and what’s next.

3. Projects: Track progress through projects and tasks with a Jira dashboard.

I’ve often heard from engineering teams using Jira that they feel like they “can’t see the forest for the trees.”

And that’s not really a surprise, considering how hard Jira makes tracking across many tasks, teams, and initiatives. For example, if you need to complete five epics for your next major launch, the best way to visualize this is through a burndown chart that captures all epics across all the sprints to the release. This is something Jira simply does not support.

Coda allows you to sync tasks from your task tracker—whether that is Jira, Coda, or anywhere else—and even combine tasks from multiple repositories or tools to one consolidated view. From there, you can customize what data you want to see and visualize them in a way that makes sense to you.

This Jira dashboard template can be adapted to other, non-Jira task trackers as well. For example, you could replicate this template with data from Linear or any task tracker in Coda’s Gallery. To recreate this dashboard with Linear, type “/linear” in your fresh doc > click the Linear Pack > Add to doc. After signing into Linear, you can bring your issues, projects, and more into your doc through a sync table, create views of that data by copy/pasting the table, then change the formatting into pie, bar, line charts, and more.

4. Projects: Uncover project cost in time and dollars with a software capitalization tracker.

Developing software, hardware, or really anything new can be tremendously costly. But it’s not always easy to understand which projects and which workstreams are the biggest contributors. Software capitalization is the process of treating your development costs like capital expenditures, and in particular, looking at the amortized cost of engineering time required to develop software.

More broadly, understanding the type of tasks your engineers work on, like knowing how much time and effort is allocated to maintenance versus new feature development across individuals, teams, and projects can be tremendously valuable to uncover bottlenecks and better scope future work. At the highest level, it is tremendously useful to know quantitatively how costly each active project is to your team.

The key information needed here is an understanding of what projects an engineer worked on during a certain time period. You can capture this information with various methods in Coda.

Forms: One approach is to have engineers fill out a Coda form (weekly or monthly) on how much time they spent on each project or task they are associated with. That Coda doc that collects form submissions can even send reminders to engineers, in Slack or another tool, who have not filled out the form. Then, you can summarize and present the form responses natively in Coda with charts.

Code production: To avoid manual effort for your engineers, an alternative approach is to estimate the time spent per project using code production. For example, you can use the Github Pack to see the PRs merged by each engineer, categorizing them with Github tags. You could similarly segment by repository or branch. A drawback of this approach is that it misses time spent that did not lead to code, such as design work.

Task trackers: Regardless of what tool you use (perhaps Jira or natively in Coda), if your team uses task tracking comprehensively, it is generally the best approach to use that information directly for resource allocation accounting. Pulling directly from Jira, for example, allows you to see exactly how long was spent on each task with an associated project, team, and engineer.

Coda’s wonderful Technical Project Manager, Jane Chiu, built a simple Jira based template to help you uncover project costs:

5. Quality: Report and take action on product quality with a bug tracker and dashboard.

Nothing is more important than the quality of your work, so having a handle on correctness, performance, reliability, and polish is critical to creating great products. Since bugs are typically the most important aspect of measuring quality, I’ll narrow my focus to bugs for the sake of this discussion.

There are two obvious ways that Coda can support your quality tracking.

Bug tracker: Coda can serve as your org-wide bug tracker, saving you money from purchasing a standalone application. Coda is particularly great for this use case for teams of 100 engineers or fewer.

Software quality dashboard: Coda can serve as your single pane of glass for calculating and tracking bug metrics, whether those bugs are stored in Coda, Jira, or elsewhere.

Metrics tracking: To understand bugs, I recommend you track a variety of metrics at two altitudes:

Unfortunately, it’s not uncommon for quality metrics to slowly move in a negative direction. This can be missed when you’re looking only at snapshots of these metrics. That’s why it’s important to track the key metrics over time and check for trends a few times per quarter.

⁠

1. Team Hubs A single source of truth for your team's status and progress.⁠

3. Staffing OS End-to-end recruiting, hiring, and onboarding in a single place.⁠

⁠

Gallery

Want to print your doc?
This is not the way.

Try clicking the ⋯ next to your doc name or using a keyboard shortcut (

CtrlP

) instead.