Skip to the stories
Windows & PowerShell

Matching New Tech Adoption to Measurable Business Results

  • Joao-Pierre S. Ruth
  • 382 views

CIOs, CTOs and CISOs face tougher questions as AI pilots scale. A CIO, CTO and CISO discuss proving tangible, secure and scalable gains.

Matching New Tech Adoption to Measurable Business Results

Pilots are easy to start and hard to justify. That's the pressure on CIOs, CTOs and CISOs right now, and it's sharper in the AI era, when a promising demo can stall the moment finance asks what it actually changed on the books.

This episode of the InformationWeek Podcast takes that problem head-on. Shatabdi Sharma, CIO at Capacity; Dan Spraggins, CTO at Backblaze; and Curtis Rebelein, CISO at DCM Services, discuss how they keep new technology tied to measurable results, and how they make sure those gains hold up on security and at scale, not just inside a proof of concept.

From pilot to proof

Here's the thing: experiments are forgiving. Nobody expects a pilot to pay back, so its sponsors can lean on enthusiasm and anecdotes. Once a tool moves past that stage, it needs a baseline, a target and a way to tell whether the number moved. Teams in this position typically start by documenting how the process worked before the change. That step is where many adoption cases quietly fall apart, because nobody wrote down the starting point.

The panel's framing centres on demonstrating improved efficiency and business results, rather than simply showing that a tool is in use. That distinction sounds small, but it changes which metrics you collect from day one.

Measuring productivity gains from AI

Productivity claims around AI are easy to make and hard to pin down. The episode covers the measures the speakers use to demonstrate productivity gains from AI and to improve tech ROI. In general, the most useful measures sit close to the work itself: cycle time on a specific task, volume handled per person, or the rate of rework. Licence activation and login counts tell you people touched the tool. They don't tell you the business got anything back.

My read is that the strongest measurement plans pick a small number of outcomes before rollout and accept that some of them will move slowly. Attribution is messy when several changes land in the same quarter, so it helps to agree in advance on how you'll separate the tool's effect from everything else.

Getting the CFO on side

Conversations about the benefits of new tech reach the finance team quickly, and the questions are usually about revenue impact rather than capability. The panel describes how these discussions unfold across the C-suite, particularly when CFOs are focused on what a deployment does to revenue.

That said, revenue is often the hardest claim to prove for an internal tool. The more productive conversations tend to start with cost avoidance, risk reduction and cycle-time savings, which finance can model and audit, before moving toward revenue arguments that need more evidence. Presenting the numbers in the terms the CFO already uses usually goes further than a technical demo.

Negotiating outcome-based pricing

The speakers also outline strategies for negotiating outcome-based pricing with vendors. Shifting fees toward results moves risk from the buyer to the supplier, but only if the terms are precise. A buyer needs an agreed definition of the outcome, access to the data needed to verify it, and clear language on what happens when the target is missed.

Without those pieces, outcome pricing can become a label on a standard contract. Vendors that accept it will want measurement windows and baselines written down, and buyers should read those clauses as closely as they read the price.

Secure and scalable, not just tangible

The episode's framing, that new tech benefits should be tangible, secure and scalable, matters here. A tool that saves hours but widens the attack surface, or falls over once usage grows tenfold, hasn't really delivered. Rebelein's perspective as CISO at DCM Services is where that test usually gets applied, because security review tends to arrive after enthusiasm has already set expectations.

The practical lesson is to bring security and scale into the pilot itself. Checking those conditions early costs far less than discovering them during a production rollout, when the business has already committed to the result.

A tabletop with goblins

To close, the three take on the Questionable Ideas tabletop exercise. Working together, they try to curb the goblins, gremlins and kobolds at a fictional company attempting to match its tech adoption with its operational results. It's a lighter format, but the underlying problem mirrors the serious discussion that comes before it: spending that never shows up in operations.

What to take into your next review

Before approving the next pilot, record the baseline and agree the two or three metrics that will decide success. Bring finance in while the business case is still being shaped, not after. If vendor pricing is tied to outcomes, write the definitions and data access into the contract. And test security and scale during the pilot, not after the purchase order is signed. The teams that get this right won't necessarily adopt less technology. They'll just be able to show what it returned.

Source: Joao-Pierre S. Ruth · www.informationweek.com

Neighbouring cells