No Measure, No Change: Project System Change Management Observations from the Field

I sometimes feel like a war correspondent embedded on the front lines of a major infantry battle. One army is trying to advance against the other. Except, in my case, I am a front-line observer of companies—owner companies, not military units—attempting to implement change efforts to improve capital project performance. The deeply entrenched enemy is the status quo. From my position along the battlefront, the carnage of an owner’s failed campaign for improving capital project outcomes is strewn everywhere.

The “change efforts” I speak of entail initiatives to improve project performance. For instance, let’s say a company wants to improve the cost and schedule predictability of its capital projects. The board of directors is tired of cost and schedule overruns and has told senior management to fix the problem. The change effort represents the collective actions taken to bring about improvement, including modifications to business processes, organizations, and project functions.

I have witnessed change efforts cut down in a hail of resistance as soon as they are ordered to begin. Others in essence veer off the battlefield because of weak leadership. Still others are able to advance well toward enemy lines but fail to breach the enemy’s defenses when resources and reinforcements are drained.

Change efforts can be derailed for any number of reasons, but the most common root cause, I believe, is the failure of change effort leaders to figure how they are going to measure success. If the change effort goal is to improve cost predictability, the measure of success would be determining whether cost predictability actually got better after the changes were implemented.

It is hard to understand why people leading a change effort would fail to measure success. We know intuitively that measurement is essential to drive change. Try losing weight without setting a target or measuring progress. Yet, somehow we routinely fail to make this connection to the change efforts undertaken to improve project performance.

In my experience, the role of measurement in change management is critically undervalued. Change efforts often start off strong; many of the participants are motivated because they think they’ll finally be able to make some improvements. At first there is a lot of excitement and discussion. The change team may even use many of the typical change management practices such as stakeholder analysis and communication plans, but without measure there isn’t enough to sustain improvement efforts over the long-term.

Having a measure of success is essential. It tells us what a comprehensive solution design looks like. This in turn forces us to define a detailed solution and develop a thorough roll-out plan, and, critically, it provides feedback on whether the solution and implementation is working. Such efforts are challenging, especially because of long project cycle times and the uncertainty of project estimates. For example, the average $5 million industrial sector project takes 14 months from authorization to startup. We don’t know if the work to improve predictability on that project was successful for more than a year! Even then, because of estimate uncertainty, it might take several more years to develop a big enough sample to really know if our changes were successful.

Contrast this with improving a manufacturing process where the effect of changes might be observed in a matter of minutes, hours, or maybe days. The feedback loop is much shorter.

Feedback from the Battlefield Is Needed Earlier and Faster

Project system change efforts have to use some type of leading indicator or project driver as the measure of success. Some examples include the level of project definition, team integration, or the use of other practices. Project outcomes still have to be measured and evaluated, but measuring project drivers shortens the time between cause and effect into something that can be used to sustain a change effort.

One of the best examples I have that demonstrates the importance of measurement using project drivers is a study that IPA completed on change efforts to improve capital project performance.¹ IPA performed at least two benchmarking studies at each site for 92 different chemical plants and petroleum refineries. The initial benchmarking established the site’s performance baseline. The second benchmarking, conducted 18 months to 3 years later, measured whether there was any improvement in construction safety, cost, schedule, and asset quality. Twenty-seven sites significantly improved their project outcomes. The other 65 failed to gain any ground. All the successful sites had one thing in common: They set a target for the completeness of project definition at authorization for every project and they measured whether the target was achieved. This was the single element that led to significant improvement in project results in just 2 to 3 years, while the other sites stagnated. Target setting for project definition completeness is vital because measuring the leading indicator of success at authorization reduces the time to get feedback on progress of improvement efforts.

Measurement Forces the Development of a Comprehensive Battle Plan 

Consider how project definition target setting changes the behaviors of the project team. The change team had to identify all the activities necessary to achieve the target. They had to get enough people and a big enough budget to do the work. They had to update their work process, develop instructions, and create tools. They had to engage stakeholders and users to figure out how to overcome resistance. They had to train people and support project teams as they learned what to do. They had to design and install the measurement system.

Of course, not all 27 sites developed a robust battle plan the first time around. There were undoubtedly varying levels of completeness. But, as General and later U.S. President Dwight D. Eisenhower is quoted as saying, “In preparing for battle I have always found that plans are useless, but planning is indispensable.” The enemy will discover weaknesses in most plans. By measuring a leading indicator of better project performance, the change team established a fast feedback loop to know whether they were successful or not. Any problems they found could be identified and fixed quickly. The problem gets more complicated the further the change is removed from the actual project outcomes. Let’s say a company creates a project management office (PMO) to deliver projects more effectively. A leading indicator of improved project performance might be lower owner costs or shorter project definition cycles, but how are these measured improvements attributable to the creation of the PMO? Are other internal or external influences driving performance changes?

Look Out for Performance Measurement Pitfalls 

Many change teams are tempted to choose compliance as the measure of success. It is easy to measure whether someone completes a form or checklist. Compliance has a role in measuring adoption of the change, but measuring compliance only eventually leads to tick the box behavior where work ends up being done in form rather than substance. If the work is important enough, the change team should set up a measurement system to critically evaluate the work.

Another hazard is that performance measures can go too far. IPA has several examples of change efforts that created so many performance targets and so much bureaucracy around measurement that the projects were uncompetitive despite implementing some Best Practices.

Measurement Is Essential

Clearly there is no magic formula for determining the best means of measuring change, but the change team cannot give up until a reasonable solution is identified. Then there should be some testing to see if it will work.

Any change effort without a quantified measure of success and mechanism for measuring should be stopped before moving out of the initial development stage. The chances of winning the battle, so that a company executes its capital projects more effectively than it did before, is very low without measuring success. Without a measurement of success, the payoff from all the money and time spent developing solutions and rolling out changes will be less than expected, if not wasted.

Paul Barshop is the author of Capital Projects: What Every Executive Needs to Know to Avoid Costly Mistakes and Make Major Investments Pay Off (Wiley, 2016).

¹ Alex Ogilivie and Kate Rohrbaugh, Site Improvement: Identifying the Pathway to Success, IBC 2015, IPA, March 2015.