Warship Cost Estimating Theater1

Introduction

This is the first in a series of posts about the not-so-sexy topic of major project cost estimating, specifically constructing naval warships. Why does cost estimating matter? If the Navy exceeds what it estimates and Congress appropriates for ship construction, it leaves less money for lifecycle management (fixing broken stuff) on existing ships. I can’t make cost estimating sexy, but I might succeed in making it interesting. Readers will be the judges.

I was inspired to begin this project from a LinkedIn post written by a regular reader (and frequent commenter, thanks!) of my blog. They expressed frustration with the Congressional Budget Office’s (CBO) practice of using cost per displacement (“weight” for non-Navy types) to estimate the cost of new warships. He referred me to this link. Intriguing.

I recall an all Engineering Duty (ED) Captain’s seminar (do they still do those?) when the Assistant Secretary of the Navy for Research, Development, and Acquisition mentioned that the CBO’s ship construction estimates were more accurate than the Navy’s. That was it, just a brief mention. We quickly moved on to other topics. Even more intriguing.

In this post, I set the stage for why understanding the Planning Fallacy offers some insight for lead ship of a class warship predictions. I’ll go into more details of the theory and research data in future posts. I continue with my purposes for this series, why I refer to the process of lead ship cost estimating as “theater,” give some examples of cost growth on large, non-Navy, public infrastructure projects, and conclude with a sketch of where I plan to go from here.

The Planning Fallacy or “You Can’t Price the Unknown”

The Planning Fallacy is the term Kahneman and Tversky (1977) used for the tendency (more like an iron-clad rule) to neglect the history of similar projects when making predictions about a current project. I became familiar with the Planning Fallacy when trying to understand why project team leaders would consistently underestimate ship availability durations at a shipyard that shall not be named. Why would the intelligent, hard-working professionals assigned to these teams consistently underestimate availability duration, despite knowing that none of the previous repair periods with similar work packages had finished on time? Why did shipyard leaders accept these estimates? Why was I the only one asking why we consistently “got it wrong”? This post and those that follow will address these questions.

Warning

I want to be clear about a few things at the outset of this series beginning with what I’m NOT doing. First, I am not minimizing the difficulty of estimating the costs of things that are super complicated and have never been built, lead ships of a class of Navy warships. Second, I am not advocating changes to current Navy or Department of Defense (DoD) cost-estimating practices. Third, I am not questioning how the Navy generates warship requirements and capabilities that result in complex designs. Like the adage about complaining about the weather, no one does anything substantive to change it anyway. Fourth, I am not minimizing the problem of aligning responsibility, accountability, and decision-making in large organizations for big, complex projects that take years to design and build. It probably isn’t possible, which will frustrate any attempt to hold individuals “accountable” for their decisions no matter how much it is called for. Fifth, I am not recommending changes to the process DoD uses to vet new technologies (mostly lots of meetings). Sixth, I am not questioning the integrity of Navy acquisition project managers. Just because they have strong incentives to be optimistic about program risks in order to get their programs funded doesn’t mean that they don’t sincerely believe in their strategies.

Optimism is essential for ambitious undertakings. The harsh realities resulting from the collision between prediction and fact in program management and ship repair probably weed out pessimists early in their careers. Or the optimists transfer before their predictions fail to come true. Shipyard test engineers are an exception to the selection bias for optimists. They don’t advance if the tests they manage go badly wrong.

I probably have a little test engineer in me so I tended to emphasize downsides more than my peers and my bosses. I’m a nuke after all. Good nukes spend a lot of time thinking about how things can go badly or they never get out of critiques. Because I strove to be realistic about risks and schedule predictions throughout my career, I was frequently criticized by my superiors for being pessimistic.

What I AM doing in this series is embarking on an intellectual exercise to share the research about biases involved in predictions about project durations. Since time is money, projects that take longer cost more.

Even though I am not a psychologist, I have studied decision biases extensively. Organizational decisions can be biased both for psychological and social reasons. Most big problems with cost and schedule predictions are a combination of both.

The Cost Estimating World’s a Stage

The title for the series, “Warship Cost Estimating Theater,” is tongue-in-cheek, but does have a basis in reality. A visitor to our planet knowing nothing about acquisition processes used by the U.S. military might be puzzled from observing what happens. First, they would observe Navy acquisition program managers declaring “We’re not like those other guys. This time things will be different.” Regardless, an alien would next note that the actual cost of a new warship class is higher than predicted in almost every case. The alien would not have to look hard to collect this data. For the newest generation of aircraft carriers, the cost growth on CVN 78 was about 27% and is projected to be $2.7b higher through CVN 81 (GAO 20-439). The next generation of frigates, the CONSTELLATION-class (cool name) will either cost $1.2 billion per hull (according to the CBO) or $870 million (according to the Navy and an unnamed “independent cost estimating group” (Kington, 2020)). Under-construction submarines of both types, ballistic-missile-lobbing and attack, are also likely to experience schedule delays and cost increases (GAO-23-106292). Finally, the alien observer would note that despite infrequent haranguing by Congressional committees, the cost increases continue to be funded through a combination of Navy and Congressional action. That’s a good thing because we need those ships.

*Government Accountability Office (2023). Defense acquisitions annual assessment-Drive to deliver capabilities faster increases importance of program knowledge and consistent data for oversight (GAO 20-439).

*Kington, T. (2020). 5 Things You Should Know About the US Navy’s New Frigate. Defense News.

*Government Accountability Office (2023). Columbia class submarine-Program lacks essential schedule insight amid continuing construction challenges (GAO-23-106292).

I perceive several theatrical aspects of warship cost estimating. By “theatrical,” I mean scripted statements and actions that don’t impact the real world, but might sound reassuring to some people (like the Barbie movie dialogue).

• First, senior acquisition leaders talk and act as if cost-estimating processes produce meaningful outcomes even though they are never accurate. For every new warship, there are vows that “This time we’ll get it right. We had good conversations with the shipbuilder and they know that we want them to be really efficient.”

• Second, if things go badly on a very visible ship construction program, Congress sets a cost cap (GAO-15-22, 2014). Only it isn’t a “cap” because Congress can raise it and often does.

• Third, changing the ship design to reduce costs isn’t possible once construction has started. During construction, any design change increases costs, which is why “changing the design to reduce costs” is an oxymoron. This is also why concurrent development (building with an incomplete design) is always more expensive than whatever was estimated before construction. It has to be because no construction prediction is valid when the design is being constantly changed, whether you are building ships, houses, or shoeboxes. There are legitimate reasons to accept this cost (it’s always a cost, not a risk), but they can’t be stated in public. A significant redesign was done for the Littoral Combat Ship during construction and didn’t end well. In fairness to those involved including the people who got fired, the goal of the LCS redesign was to increase costs because someone let it slip that the design wasn’t “survivable.” Hint: if your lead ship or assault vehicle (aka “Bradley”) gets tagged as “unsurvivable,” there is no hope for meeting your cost estimate.

• Fourth, since it is expensive to change a ship’s design during construction, the only way to limit construction cost growth is to defer work. The normal practice is to move some work outside of the new construction period and seek new funding to accomplish it later. This doesn’t make it cost less and is a second reason why cost caps aren’t real.

• Fifth, even with a cost cap, the GAO might write a report stating “With the shipbuilder embarking on one of the most complex phases of construction with the greatest likelihood for cost growth [Ralph-usually system testing and integration], cost increases beyond the current $12.9 billion cost cap appear likely … the Navy is deferring some work until after ship delivery [which] will result in the need for additional funding later, [but the] approach obscures visibility into the true cost of the ship” (GAO-15-22, 2014). Ouch. This is the GAO publicly stating that a cost cap isn’t real. It is also the GAO publicly not understanding acknowledging that the whole point was to obscure the true cost of the ship.

• Sixth, there are public denunciations. Members of Congress claim to have provided “unprecedented support” for ship construction so delays and cost overruns must be somebody else’s fault. The Navy can’t blame Congress (called “biting the hand that feeds you”) so they blame evil defense contractors for not contributing enough (enough what?). Members of Congress join them by decrying stock buybacks and asserting that contractors are not prioritizing correctly (prioritizing what, free work?). Contractors, at the low end of the hill, can’t blame the Navy or Congress (that would be biting two feeding hands, never a good idea if you want to get fed) so they commit to “continue partnering with the Navy” (what else could they do?) (Eckstein, 2024)

*Eckstein, M. (2024). Navy, senators argue over who is to blame for a too-small fleet. Defense News. Hat tip to Robert du Mont

• Seventh, if Navy leaders are brought before a Congressional committee to account for cost and schedule problems, they nobly accept responsibility for what they didn’t do (good strategy, nobody fires a martyr). They had nothing to do with the original plans like introducing 23 new technologies into the design of a new warship. They are thus free to say things like “that was a bad idea” to affirmative head nods (often accompanied by harrumphing). The not-responsible Navy leaders receive stern Congressional warnings at the hearings and make sincere promises not to repeat the errors made by the previous Navy leaders (Procurement, Acquisition, Testing, and Oversight of the Navy's Gerald R. Ford-Class Aircraft Carrier Program, 2015).

*Government Accountability Office (2014). Ford Class Aircraft Carrier-Congress Should Consider Revising Cost Cap Legislation to Include All Construction Costs (GAO-15-22).

*S. Hrg. 114-302: Procurement, Acquisition, Testing, and Oversight of the Navy's Gerald R. Ford-Class Aircraft Carrier Program: Hearings before the Senate Armed Services Committee, 114th Cong. 1 (2015).

It’s Not Just the DoD

The challenge of producing accurate cost estimates for large, complex undertakings is not specific to the Navy or the U.S. Department of Defense. The Channel Tunnel (Chunnel) rail connection between London and Paris was estimated to cost £5.5b. Its final cost was £9b. The California high-speed rail to connect San Francisco to Los Angeles had a 2008 estimate of $34b. The 2024 estimate is between $88.5b to $127.9b depending on which phase of construction is being considered (it gets messy fast).

Conclusion

The purpose of this post was not to criticize the Navy or recommend changes to DoD acquisition processes. The purpose was to identify a problem in new construction warship cost estimating: existing DoD processes don’t produce accurate estimates, cost or schedule. Subsequent posts in the series will examine the academic research on the Planning Fallacy and why CBO warship cost estimates are more reliable, though not necessarily better, than the Navy’s. “Better” is highly subjective in this context.

References for the Series

Arena, M.V., Blickstein, I., Younossi, O., and Grammich, C.A. (2006). Why Has the Cost of Navy Ships Risen? RAND Corporation. Retrieved from https://www.rand.org/content/dam/rand/pubs/monographs/2006/RAND_MG484.pdf

Arena, M.V., Leonard, R.S., Murray, S.E., and Younossi, O. (2006). Historical cost growth of completed weapon system programs. RAND Corporation. Retrieved from www.rand.org/pubs/technical_reports/TR343.html

Arena, M.V, Younossi, O., Galway, L.A., Fox, B., Graser, J.C., Sollinger, J.M., Wu, F., Wong, C. (2006). Impossible Certainty: Cost Risk Analysis for Air Force Systems. RAND Corporation. Retrieved from https://www.rand.org/pubs/monographs/MG415.html

Cregge, K. (2022). What I Learned from Coach Niumatalolo. U.S.N.I. Proceedings. Retrieved from https://www.usni.org/magazines/proceedings/2022/december/what-i-learned-coach-niumatalolo

Eckstein, M. (2024). Navy, senators argue over who is to blame for a too-small fleet. Defense News.

Government Accountability Office (2014). Ford Class Aircraft Carrier-Congress Should Consider Revising Cost Cap Legislation to Include All Construction Costs (GAO-15-22).

Government Accountability Office (2023). Columbia class submarine-Program lacks essential schedule insight amid continuing construction challenges (GAO-23-106292).

Government Accountability Office (2023). Defense acquisitions annual assessment-Drive to deliver capabilities faster increases importance of program knowledge and consistent data for oversight (GAO 20-439).

Kahneman, D. (2011). Thinking, fast and slow (First Edition). Farrar, Strauss and Giroux.

Kahneman, D. & Tversky, A. (1977). Intuitive prediction: Biases and corrective procedures, Technical Report PTR-1042-7746, Defense Advanced Research Projects Agency - Advanced Decision Technology, Decision Research. Retrieved from https://apps.dtic.mil/sti/pdfs/ADA047747.pdf

Kington, T. (2020). 5 Things You Should Know About the US Navy’s New Frigate. Defense News.

Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: how difficulties in recognizing one's own incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology, 77(6), 1121-1134.

Labs, E.J. (2018). How CBO Estimates the Cost of New Ships. Congressional Budget Office. www.cbo.gov/publication/53785

Labs, E.J. (2020). The Cost of the Navy’s New Frigate. Congressional Budget Office. www.cbo.gov/publication/53785

Labs, E.J. (2023). An Analysis of the Navy’s Fiscal Year 2024 Shipbuilding Plan. Congressional Budget Office. https://www.cbo.gov/system/files/2023-10/59508-shipbuilding.pdf

Lagrone, S. (2024). Carrier USS John C. Stennis overhaul delayed, work will take more than 5 years to complete. USNI News. https://news.usni.org/2024/06/18/carrier-uss-john-c-stennis-overhaul-delayed-work-will-take-more-than-5-years-to-complete

McNicol, D.L. & Wu, L. (2014) Evidence on the effect of dod acquisition policy and process on cost growth of major defense acquisition programs, IDA Paper P5126. Institute for Defense Analyses. Retrieved from https://apps.dtic.mil/sti/pdfs/ADA609472.pdf

Meyer, J. W., & Rowan, B. (1977). Institutionalized organizations: Formal structure as myth and ceremony. American journal of sociology, 83(2), 340-363.

S. Hrg. 114-302: Procurement, Acquisition, Testing, and Oversight of the Navy's Gerald R. Ford-Class Aircraft Carrier Program: Hearings before the Senate Armed Services Committee, 114th Cong. 1 (2015).

Verschuere, B., & Meijer, E. (2023). A simple heuristic for distinguishing lie from truth. Nature Human Behaviour, 7(5), 674-675.

Younossi, O, Arena, M.V., Leonard, C., Roll, Jr., R., Jain, A., and Sollinger, J.M. (2007). Is Weapon System Cost Growth Increasing? RAND Corporation. Retrieved from www.rand.org/pubs/monographs/MG588.html