How did Health NZ spend $72m on a single IT project?

Analysis - "You know it's all very technical ... sounds boring ... but it's super-important, right, because everyone can think of projects that have blown out."

That's Infrastructure Minister Chris Bishop, who has a big mess to sort out on the infrastructure front.

Business cases are not up to scratch, investment management has holes and gaps, risk assessments can be missing entirely.

This is all from recent warnings to Bishop from Treasury, which oversaw the previous, flawed system.

A test case for what can go wrong within that system is the $240m Health New Zealand IT project which RNZ on Friday reported had swallowed $72m to pay contractors and consultants in the span of two years.

Despite all that expertise and all that spending - so much it has helped tip HNZ into a deficit so large it plans to axe 1100 IT employees' positions to save money - it has been forced this year to undergo a full reset of its budget and timing baselines.

Here is a non-technical, less boring account of how that $72m Health Sector Agreements and Payments Programme (HSAAP) came to spin out of control - alongside points Bishop made in an interview with Morning Report and later in a media release.

Fix the holes

"It will take years of sustained action to address New Zealand's large infrastructure deficit," Bishop said.

Health NZ's IT deficit left it no choice.

It was processing $400m of payments every week using three or so IT systems built in the 1990s, for everything from organ donor services to bandages to outsourced operations to residential care.

Staff were - and still are - manually keying in invoices, receipts and suchlike.

"Many of the processes depend on paper forms, and printing and posting these around the country," said reports from June and August that RNZ dug up under the Official Information Act.

The systems were vulnerable to "operational failure, fraud and [BLANKED OUT] due to the age and obsolescence of the technology" - and suffering payment "leaks" of tens of millions of dollars.

It had to be fixed.

The fix is in - or is it?

"I mean, everyone can think of examples where there's been huge cost blow-outs over time and that fundamentally goes back to not getting the decision-making right at the start, and not scoping up the projects properly, and not having the scrutiny oversight," said Bishop.

At the start, Health NZ did a business case.

In 2021, the case said the full HSAAP programme would take 10 years to roll out, though the IT solution itself "would be complete by 2024".

"By June 2023 it became apparent that there were real delivery challenges," said the OIA reports.

By October 2023, a review instituted changes to three basics - oversight (often called governance), what the actual tech (often called a "stack") would be, and to "ensure the focus was on accelerating delivery".

By March 2024, Treasury was engaged in a Gateway review - it does these when projects are high risk, or wobbling.

"HSAAP is reporting AMBER/RED currently against delivery, which is a fair reflection of the progress being made mapped against the difficulties uncovered in recent months," it concluded a few months ago.

Treasury told Health not to stop, but to accelerate, though only after a full reset for get clear time and budget baselines.

By October 2024, it had some good news, when a big improvement to a new agreement management system went live.

The expectations are different now than in 2021. The public sector as a whole has received new orders, to trim their $8 billion IT capital wishlist, by sharing upgrades and avoiding fancy customised purchases.

Not such fast fixes

"This will ultimately slow the ability to deliver our infrastructure and investment priorities," said Bishop, referring to poor and patchy business cases being presented to Cabinet.

Health NZ, having been forced to hit reset on the big payments project, has been left exposed to higher risks for longer.

"Legacy system risk - and associated payment failures - has only become more pressing and serious since the business case was approved" in 2021, the OIA reports said.

The reset approach would eventually save an extra $7m a year, but take much longer to get to that point.

"Delivery will occur over a longer horizon, and with an associated delay to the realisation of benefits and continued legacy support costs."

This raises the question of futureproofing: Will the IT being put in today need overhaul again soon after it is all delivered in 2030?

Paying the price

"When it comes to these big capital expenditure projects, you know, billions and billions of dollars, that has a ginormous impact on the Crown books," Bishop said.

Health NZ has a huge financial hole.

The HSAAP, and other IT projects, might help fix that long term, but short term they add to it (though HSAAP is forecasting only a $2m over-run this year).

The delays mean the contractors and consultants, who cost $72m from 2022 to September 2024, stay on the payroll longer - outlasting staff as employees exit under cost-cutting restructuring (they may, of course, arrive back soon, as contractors).

Treasury told Health to "produce a long-term commercial strategy to deliver an affordable, sustainable technology solution through the suppliers. This should review whether existing contracts can be improved".

The project's spend on external hires is dropping, forecast to reduce by about half this financial year to $15m - compared to the last two years, from $800,000 a month on consultants alone, to $160,000.

Employee costs are a small fraction of the personnel costs, and some of them are no doubt being kept on to keep the old, legacy payments systems going.

By 2030, the aim is that staff costs to operate the new payments system will be cut by over half, to about $5m a year.

The ultimate fix

"It comes back to the questions that you asked at the start, of the decision making, and the oversight provided by the experts," Bishop told Morning Report on Friday.

"So we are looking at internal resourcing inside the government to make sure that we really, you know, have our best minds and our best resources focused on this."

At Health NZ, the labour continues, long and costly, to leave behind the risky legacy of old IT.

HSAAP is running a four percent capacity now.

By next June, it might be handling transactions at a rate of 600,000 a year, when it has to manage 120 million.

By 2030, it might get there. At least unlike many other projects that have proved too ambitious, this one is still forecast by the "experts" to deliver all the benefits it should and has not had to reduce its scope.

Not yet anyway.

Sign up for Ngā Pitopito Kōrero, a daily newsletter curated by our editors and delivered straight to your inbox every weekday.