Update 11/20: Chang-Tai Hsieh counters that Greaney’s critique ignores general equilibrium effects which make labor scale invariant. That doesn’t address the alleged coding errors. We’ll see – and perhaps I wrote an autopsy too early. Thanks to Bryan Caplan for getting Hsieh’s response out to the world.
Popular urban econ should be shaken with the revelation that its most famous academic paper had two coding errors, a serious theoretical flaw, and a hypothesized mechanism that – when executed correctly – did not work at all.
Chang-Tai Hsieh and Enrico Moretti’s paper Housing Constraints and Spatial Misallocation noted that “high productivity cities like New York and the San Francisco Bay Area have adopted stringent restrictions to new housing supply, effectively limiting the number of workers who have access to such high productivity” – and used a simple model to estimate how much US growth could have been unlocked by decreasing those restrictions.
Brian Greaney, an assistant professor at the University of Washington, released notes on his replication of The paper gained widespread notice as a 2015 NBER working paper and was published in 2019 in American Economic Journal: Macroeconomics. The paper already has an impressive 813 scholarly citations.
But the paper has problems – fatal problems – and is embarrassingly sloppy. The embarrassment extends beyond the authors to the many referees and editors who missed surface, implementation, architectural, and foundational problems over a four-year period of peer review and discussion.
Surface
Greaney is not the first to find a mistake in Hsieh & Moretti. Bryan Caplan caught a major inconsistency in 2021, one that readers (myself included) and referees should have caught earlier. The authors reported huge annual effects adding up to a merely-large effects over 45 years. He noted a few other arithmetical mistakes and generously concluded, “authors and referees alike focused so intently on the advanced mathematics that they glossed over some elementary yet crucial errors.” This turns out not to have been true.
Implementation
Greaney found two coding errors in Hsieh & Moretti’s publicly-posted Stata code. (Thank goodness for public code requirements!)
If the coding errors (only) were corrected, the headline result would be reversed: “their model predicts that the land-use deregulation experiment they propose would decrease output.” Nobody thinks that this is the right answer – but it’s an answer that would have alerted the authors to a deeper error.
Architecture
Why did correct code yield an unintuitive answer? Because Hsieh and Moretti’s model was scale-dependent. Anyone interested in the math should read Greaney’s brief explanation. This was not hard to fix – Greaney does so, and a diligent referee might have caught the problem.
With the problems fixed, Hsieh and Moretti’s conceptual argument delivers a big nothing: Greaney reports that liberalizing three superstar cities would boost national GDP by just 0.02%.
The nearly-null result is surprising to urbanists – but it isn’t so surprising when you realize the modeling approach that Hsieh and Moretti took to build their idea. They treat each metro area as a separate unit that takes labor as a factor of production with decreasing marginal returns. Thus, lower housing prices in superstar cities attract more migrants, decreasing local wages and increasing wages in the sending areas. Since each migrant decreases wages in one place and boosts them in another, it’s not so surprising that the net effect is near zero.
Foundation
Urban economists Gilles Duranton and Diego Puga offer a much richer treatment of local agglomeration and congestion effects. Their work ought to replace Hsieh and Moretti as the academic anthem for YIMBYism, but there’s still a fundamental problem:
Silicon Valley and Manhattan attract especially productive, high-income people. Even when the absolute number of workers is low, as in Silicon Valley, the concentration of complementary talents can be enough to achieve large agglomeration benefits through selective migration to innovation clusters
Contrary to the models, the next 35 million people to move to the Bay Area would probably not be as tech savvy or as risk loving as the first 5 million. The next 20 million people to move to New York would include a lower proportion of investment bankers willing to work 70-hour weeks.
There’s a vigorous debate in urban economics about the strength of local agglomeration effects. One paper says, “two thirds of the variation in observed wage premiums for working in different CZs is attributable to skill?based sorting.” (In English, that means that most of the difference between local wages arise because different kinds of people chose to live there. You can’t get a big raise by moving.) Another paper finds that the difference is “about half”, not two thirds. Oh wait, it’s the same paper – just a new revision.
These debates are hard enough when all the code and arithmetic are sound – we readers and referees should work harder to ensure that the papers we’re debating are executed correctly.
