Seeking Effective Metrics for Cost-Focused Technical Proposals

Good question! Unfortunately, the answers are quite tricky and vary a lot based on company 😅

What are some quantifiable metrics that engineers can use to demonstrate the efficacy and cost-effectiveness of their technical proposals?

Well, there's only 1, which is money $$$. The tricky part is how do you calculate the $$$ saved?

The process here is to calculate a metric that's 1 layer removed with a known conversion to $$$. For example, if you know that storing 1TB of user data costs you $1000 and you optimize a flow via data compression to generate 50TB of data per month instead of 100TB per month, you can say that you're saving $50,000 per month or $600,000 per year.

In terms of the conversion, it will vary based on the SaaS infrastructure products you use or how your own software is built if you're doing things in-house.

At Meta, we had all of this tooling built in-house where you could run an A/B test and just see the savings. If the thing you were measuring against didn't have the collection, you would need to talk to someone on the infra side like an SRE, data engineer, etc.

What are some quantifiable metrics that engineers can use to demonstrate the efficacy and cost-effectiveness of their technical proposals?

I really doubt there's standard tools besides general A/B testing frameworks where you would still need to add the custom logging. Again, this varies a ton based on the type of bug and product it affects.

For a concrete example, check this out: [Case Study] Solving A Multi-Million $$ Instagram Bug

What approaches can I use to measure and present reductions in coding and discovery time? Any suggestions on metrics or tools that can help quantify these time savings in terms of cost?

For projects that make the codebase better, it is really hard to holistically measure. You need more anecdotal evidence. You could also run a survey. More thoughts here: "How do you measure the impact of refactoring your codebase to use a new pattern?"

How can I illustrate the cost benefits of implementing scalable solutions that accommodate future changes without additional coding? Are there ways to forecast and present these long-term savings?

You would need to calculate dev time and multiply it by an average cost of an engineering hour. So if the average engineer gets paid $100/hour in your org, and you take a 40 hour coding task and reduce it to 0, every instance is worth $4000.

Are there recommended tools, methodologies, or frameworks that assist in gathering and presenting these metrics effectively to a cost-focused audience?

I recommend making a slide deck with as many charts as possible. Executives can't read code anymore, and people in general don't like walls of text. Be visual and show deltas.

There are two broad approaches here:

1. First principles thinking: Look at the data and come up with reasonable assumptions about how much you can save costs or grow revenue based on a project.
   1. You can compute things like bug prevention, user growth, revenue growth, time savings, etc. This requires looking at code, dashboards, or historical data.
2. Second principles thinking: Identify past projects that are similar to what you're proposing, and 'borrow' their metrics.
   1. Make the argument that "They achieved X, we should achieve something similar." Or something like "They failed due to Y, but this project will avoid that and achieve what was promised."

Your question is focused on #1, so I'd encourage you to also spend some time on approach #2. This can be much easier, since you're leveraging the past work of your teammates, which means it already has a lot of credibility.

Another consideration is to understand what metrics your company cares about, both at the company-level and for your individual performance review: Is it normal for a company to track performance related metrics and use it as input for promos/bonuses?