Forecasting v. All-Else-Being-Equal policy analysis

Philip Tetlock's 2005 book Expert Political Judgment raised amusing doubts about the accuracy of professional public affairs forecasters. 

Tetlock was particularly derogatory about dogmatic "hedgehogs." Economist John H. Cochrane suggests in a Cato Unbound piece, however:

It was once hoped that really understanding the structure of the economy would also help in the sort of unconditional forecasting that Gardner and Tetlock are more interested in. Alas, that turned out not to be true. Big “structural” macroeconomic models predict no better than simple correlations. Even if you understand many structural linkages from policy to events, there are so many other unpredictable shocks that imposing “structure” just doesn’t help with unconditional forecasting. 

But economics can be pretty good at such structural forecasting. We really do know what happens if you put in minimum wages, taxes, tariffs, and so on. We have a lot of experience with regulatory capture. At least we know the signs and general effects. Assigning numbers is a lot harder. But those are useful predictions, even if they typically dash youthful liberal hopes and dreams.
Doing good forecasting of this sort, however, rewards some very hedgehoggy traits. 

Focusing on “one analytical tool”—basic supply and demand, a nose for free markets, unintended consequences, and regulatory capture—is essential. People who use a wide range of analytical tools, mixing economics, political, sociological, psychological, Marxist-radical and other perspectives end up hopelessly muddled. 

Keeping analysis “simple and elegant” and “minimizing distractions” is vital too, rather than being “comfortable with complexity and uncertainty,” or even being “much less sure of oneself.” Especially around policy debates, one is quickly drowned in mind-blowing detail. Keeping the simple picture and a few basic principles in mind is the only hope.

In other words, there is a big difference between macro and micro forecasting. Macro forecasting needs all the factors, and nobody understands all the factors. Micro forecasting, in contrasting, can benefit from theory. Micro forecasting is policy analysis, which benefits from using the economic concept of "all else being equal."

For example, in the late 1970s, the state of California passed a law allowing local municipalities to impose rent control on landlords. In microeconomic theory, this is not a good idea. And indeed, living in Santa Monica in 1981-82, as a newly minted economics major, I could observe rent control in action: my landlady treated me with all the warmth of a Serb hosting German soldiers in WWII. She invested nothing in the upkeep of the apartment in the 22 months I was there. 

On the other hand, the rent was a helluva deal for a place where I could bike to the beach a few times per week. I had no intention of moving out if I could find a job anywhere in the L.A. area. I figured that in five years the apartment building would look like a wreck from neglect, but I'd probably still be legally squatting in it, writing my $250 per month rent-controlled check. (I slowly figured out that Santa Monica had been undervalued in the past because, before antibiotics and smog, newcomers to Southern California had wanted to live inland in drier places like Pasadena to avoid TB; but by 1981 the whole climatic logic had long-since reversed, so demand for Santa Monica apartments was finally soaring.)

In the long run, it's clear that rent-control is not really a good idea. Not many municipalities in California still have it. On the other hand, ones that do, such as Santa Monica, West Hollywood, and Berkeley, are typically not post-apocalyptic wastelands, either.

Indeed, the rent-controlled municipalities tend to be ones favored by geography, and thus attract smart, well-heeled people, who figure out ways to make it work.

To make an all-else-being-equal forecast about the impact of imposing a policy of rent control on Santa Monica, you only need to know that it would cause less investment in apartment buildings than would happen otherwise.

On the other hand, to make an unconditional forecast about what Santa Monica would wind up like, you'd need to understand a lot more factors, especially about human capital, which economists are not very bright about.

And that's the bigger lesson to be learned.