Engineering Delivery Metrics and Systems

What you measure shapes what your teams optimize for. The wrong metrics create perverse incentives -- measuring lines of code rewards verbosity, measuring story points completed rewards point inflation. The right metrics illuminate whether your engineering organization is delivering value sustainably. EM interviews frequently test whether you understand which metrics matter and how to use them without turning them into weapons.

DORA Metrics

The most rigorously validated engineering metrics come from the DORA (DevOps Research and Assessment) programme, now part of Google Cloud. The foundational research, led by Nicole Forsgren, Jez Humble, and Gene Kim, was first published in the book Accelerate (2018) and has been extended each year since in the annual State of DevOps reports, backed by survey data from tens of thousands of organisations.

Two things about DORA have changed since the version most interview guides still teach, and getting them wrong is a credibility risk in front of a panel that reads the reports.

First, there are now five metrics, not four, and one has been renamed. DORA currently defines these:

Second, DORA retired the elite / high / medium / low performance clusters. The 2025 report replaced them with seven organisational profiles -- names like Legacy Bottleneck, Constrained by Process, Pragmatic Performers, and Harmonious High-Achievers -- on the reasoning that a single performance tier hid why a team was where it was. Two teams could land in the same tier for completely different reasons and need opposite interventions.

This is why this lesson does not print benchmark bands for each metric. Any figure of the form "elite teams deploy X times per day" was tied to a clustering model DORA itself has moved away from, and quoting it in an interview dates you precisely. Read the current numbers and profiles from DORA's own metrics guide and research index before an interview where you expect to be asked.

The durable finding underneath all of it has not changed: throughput and stability move together in strong organisations. High performers deploy more frequently and fail less and recover faster. That result is what disproved the assumption that speed must be traded against stability, and it is the point worth making in a room.

When discussing these metrics in interviews, make three points:

1. They are team-level indicators, not individual performance metrics. They reflect system health and process effectiveness, not whether a specific engineer is performing well. A manager who puts DORA metrics in a performance review has misunderstood the research.
2. Track your own trend, not someone else's benchmark. A team improving from monthly to weekly deployment is making real progress whether or not it clears a published threshold -- and given that DORA has retired its tier model, the trend is now the more defensible framing anyway.
3. Move throughput and stability together. State this as the reason you would not accept a mandate to double deployment frequency without also being measured on failure and rework.

Velocity Tracking

Velocity -- the amount of work a team completes per sprint, measured in story points -- is useful for internal planning and trend analysis. It answers the question: "Based on our recent pace, how much can we realistically commit to next sprint?"

Common pitfalls with velocity:

• Comparing velocity across teams. Team A completing 40 points per sprint and Team B completing 25 points says nothing about relative productivity. Each team calibrates story points differently.
• Using velocity as a performance metric. When managers use velocity to evaluate teams, teams inflate point estimates to look productive. The metric becomes meaningless.
• Ignoring velocity variance. A team that averages 30 points per sprint but swings between 15 and 45 has a predictability problem worth investigating.

The correct use of velocity is to calculate a rolling average (typically over 3-5 sprints) and use it as a forecasting input. If the rolling average is 32 points and the backlog for the next milestone contains 128 points, you can forecast approximately four sprints of work.

That arithmetic is the easy part, and it is also where the forecast quietly becomes dishonest. A rolling average includes the sprints where a third of the team was on incident response; a forecast built from it and then presented as a date does not. The widget below starts from exactly the worked example above -- 32 points, 128 in the backlog, four sprints -- and then lets you apply the two adjustments that separate a forecast from a promise.

Run it once with the defaults and you get the four sprints from the text. Now set the debt reserve to 20% and a realistic swing of eight points, and the pessimistic case lands well past that. That spread is the answer to "when will it be done?" -- not the headline number. A manager who quotes the single figure has committed to the optimistic branch without noticing, which is the mechanism behind most missed commitments described in this course's failure-story lesson.

Roadmap Creation

Engineering roadmaps bridge strategy and execution. They answer three questions: What are we building? Why does it matter? When will it be ready?

A practical roadmap structure uses three time horizons:

• Now (current quarter): Committed work with high confidence in scope and timing. Items here should be broken into epics with estimates.
• Next (next quarter): Planned work with moderate confidence. Scope is defined at the theme or initiative level. Timing is approximate.
• Later (beyond two quarters): Directional bets with low confidence. These represent strategic intentions, not commitments. They will change as you learn more.

This structure communicates honestly about certainty. Stakeholders who see a detailed 12-month roadmap with exact dates should be skeptical -- no one has that level of foresight in software development. A Now-Next-Later roadmap sets appropriate expectations.

When building roadmaps, account for these common traps:

• All features, no infrastructure. If the roadmap contains zero investment in platform improvements, tech debt, or reliability, the team will slow down over time.
• No dependencies marked. Call out cross-team dependencies explicitly. A feature that requires work from three teams needs coordination, and the roadmap should reflect that.
• Missing capacity buffers. Plan for roughly 70% of capacity. The remaining 30% absorbs unplanned work, production incidents, and scope adjustments.

Dependency Management

Dependencies between teams are the primary source of delivery delays in mid-to-large engineering organizations. Managing them requires making them visible and reducing them systematically.

Tactics for dependency management:

• Dependency boards. Maintain a shared board (physical or digital) where every cross-team dependency is tracked with an owner, a status, and a target date.
• Regular dependency syncs. A weekly 15-minute standup where teams surface blockers and update dependency status. This catches problems early when they are cheap to fix.
• Architecture that reduces dependencies. Invest in APIs, event-driven systems, and platform abstractions that allow teams to work independently. The best dependency management is having fewer dependencies.
• Sequence work to minimize blocking. When planning a quarter, identify which items will unblock other teams and prioritize them early.

OKR Setting for Engineering

OKRs (Objectives and Key Results), originated by Andy Grove at Intel and popularized by John Doerr at Google, provide a framework for connecting engineering work to business outcomes.

The structure is straightforward:

• Objective: A qualitative, inspirational goal. Example: "Make our API the fastest and most reliable in the industry."
• Key Results: 2-4 measurable outcomes that indicate whether the objective is being achieved. Example: "Reduce p99 API latency from 800ms to 200ms" or "Achieve 99.95% uptime for all Tier 1 endpoints."

Common mistakes EMs make with OKRs:

• Output-based key results. "Ship the new caching layer" is an output. "Reduce database read latency by 60%" is an outcome. Outcome-based key results allow teams to find the best solution rather than being locked into a predetermined implementation.
• Too many OKRs. Three objectives with three key results each means nine measurable targets. That is already at the upper limit. More than that dilutes focus.
• Sandbagging targets. If every key result is achieved at 100%, the targets were not ambitious enough. A healthy OKR system expects 60-70% achievement on stretch goals.
• No mid-cycle check-ins. OKRs set at the start of a quarter and reviewed only at the end are not useful for course correction. Run check-ins at weeks 3 and 6 to assess progress and adjust.

When an interviewer asks you to set OKRs for a hypothetical team, demonstrate the connection between the team's work and the company's strategic priorities. Show that you think in terms of outcomes, not outputs.

You have now completed the four lessons in this module on Leadership System Design. Take the module quiz to test your understanding of organizational design, team topologies, engineering processes, and delivery metrics. :::