Capitalisn’t: Economic Research—Nothing Precisely or Precisely Nothing?

Kate: In 2017, every child’s nightmare came true.

Speaker 2: Just before the holiday shopping season, Toys “R” Us files for Chapter 11 bankruptcy protection.

Luigi: The following year, the company was forced to close all of its stores across the country.

Speaker 4: This company was taken private back in 2005, and it went private with borrowed money, so the company has $5 billion of long-term debt. No matter how well you’re doing, that’s an awfully big load to carry.

Kate: At the time, I just couldn’t get the jingle out of my head.

Speaker 5: I don’t want to grow up, I’m a Toys “R” Us kid, they got a million toys at Toys “R” Us that I can play with …

Kate: So, I started thinking of a different jingle, which I was very proud of, and it’s now pinned to my Twitter. OK, I’m going to sing it, but it’s going to be really bad. You ready?

Luigi: Yes.

Kate: I don’t wanna be bought in a Toys “R” Us bid. They got a million bucks at KKR that they’ll buy me with. From bonds to loans to private-equity fees, it’s the biggest drain on cash there is—gee whiz—I don’t wanna be bought, ‘cause maybe if they did, 30,000 Americans would lose their jobs.

Speaker 5: Always a Toys “R” Us kid.

Kate: This question of what caused the retail apocalypse has been in my head for a long time, and I think that Amazon is probably the biggest culprit. And, to some extent, other big-box retailers like Walmart probably pushed Toys “R” Us to the brink, so I think this is an interesting empirical question to try to tease out.

Is it the case that private equity’s totally blameless, or do they actually push retail firms to file for bankruptcy earlier or contribute even further to the cost of bankruptcy? I think a lot of people have an emotional reaction to this issue about whether private equity is destroying our favorite retail companies. But we should have a better-informed sense of whether or not there is some sort of real relationship between private equity and the retail apocalypse.

On this episode, we’re trying something a little different. Luigi and I thought that we would try our hand at actually working on a project together and discussing the various pitfalls that we encountered, but also, at the end of the day, figuring out whether or not we could come away from this experiment with anything meaningful.

Luigi: This episode is more wonky than our average episode, but we want to give it a try. After all, the promise of this podcast was to bring you into the conversation among economists, and I’m sorry to say, we are wonky.

Kate: This time, you’re going to get to see how the sausage is made.

Luigi: From the University of Chicago, this is Luigi Zingales.

Kate: And from Georgetown University, this is Kate Waldock. You’re listening to Capitalisn’t, a podcast about what’s working in capitalism today.

Luigi: And, most importantly, what isn’t.

Kate: Before we jump into the technical analysis, you probably need a little bit of context. First of all, what’s this retail apocalypse? People say it started roughly after the financial crisis that brick-and-mortar retail stores in the United States are not doing so well, they’re closing their doors. Amazon is certainly part of the blame for this and most reasonable people think that, OK, retail expanded too much in the United States anyway before the financial crisis, and so it’s just a natural contraction.

Some data from the Census and the St. Louis Fed have shown that over 12,000 physical stores have closed. And you’ve probably seen it yourself, right? You’ve probably gone to the mall and noticed it’s felt a little empty. You might have gone to some retail streets where you shopped earlier and noticed that there are a lot of empty storefronts, so it’s something that’s on a lot of people’s minds.

Luigi: But also, you notice that when you walk down a street, you see all these Amazon packages sitting on the doors of the various houses. That’s the alternative to shopping at a mall.

Kate: Absolutely. But we’re not going to talk about Amazon, or at least we’re not going to focus on them on this episode. We’re interested in private equity. Well, a private-equity fund is basically just a pool of money coming from wealthy individuals, institutional accounts like hedge funds, in some cases even pension funds. And that fund is managed by a general partner who makes investment decisions in private companies.

Private-equity firms can pursue various strategies to try to maximize the return for their investors. And we’re going to focus on one particular strategy on this episode, which is the idea of a buyout.

Luigi: The positive view of leveraged buyouts is that this is a great way to achieve two objectives for somebody to buy a company with a limited amount of equity. Imagine that I want to grow a company, or I see a possibility of changing the structure of a company, and by doing so, I will dramatically increase its value. If I were to do it without acquiring it, that benefit will go to other people, so I want to get control, and the only way I get control with a limited amount of equity is by taking on a lot of leverage.

The other possibility is that you want to force the company to restructure, you know that the writing is on the wall, and so you take a lot of debt as a form of commitment device that if you don’t do the right thing, you’re going to go into bankruptcy.

Kate: Regardless of the impetus, and I’m sure that both of those factors were at play here, we know that most buyouts by private-equity firms involve a lot of leverage, they involve a lot of debt. And at the end of the day, it’s not the private-equity firm per se that’s responsible for that debt. It’s the company itself, it’s the company that they purchase. Those assets of the company, of the target, are the assets that are pledged against the debt.

A company like Toys “R” Us, for example, before they were bought out in 2005, they had some debt. But after they were purchased by a private-equity firm, all of a sudden, their debt exploded.

So, Luigi, the idea is that we want to figure out whether or not the involvement of private equity through these leveraged buyouts is causing the retail apocalypse. What’s the ideal setting that would allow us to answer this question?

Luigi: The ideal setting is that you are a private-equity guy, you look at potential targets. And then you flip a coin on which one you’re going to buy and which one you’re not going to buy. And then you compare the one you bought with the one that you didn’t buy to see whether this makes a difference. In other terms, this is exactly like how the medical professions test their new drugs. You treat a group and you don’t treat a group that is identical, and then you see whether the treatment makes a difference.

So, unfortunately, this kind of exercise is difficult to do in practice. I don’t think that you will convince any private-equity investor to behave in that way.

Kate: Yeah. Luigi, you raise some great points about why this is really hard to measure causally, because we don’t have randomized controlled trials for whether private-equity firms bought companies or they didn’t. But there’s this other, simpler issue, which is that in an ideal world, we would have thousands, hundreds of thousands of these sorts of observations or these little experiments taking place that would allow us to make inferences about the way that private equity works.

But in reality, we just don’t have that much data. There aren’t that many public companies that are bought out by private-equity firms. And even if there are, it’s kind of hard to see what happens, because private firms by nature don’t really need to report a whole lot.

Luigi: You’re absolutely right. I think that what makes it even more difficult is the nontransparency of privately held companies, because if we had access to all the buyouts, big and small, and data on ex-post performance of all of them, that would be a much easier analysis to do.

Kate: Being fully aware of the hurdles, let’s try to do our best. I’m going to start by talking about this data set called Compustat. Most universities, most research universities, any sort of group doing research in finance or economics, they usually have access to a certain set of data sets. And Compustat, this data set, which collects financial reporting information from publicly traded companies, is probably one of the most standard data sets there is.

What’s nice about it is that for each company, you can see every year how it’s doing. You can see what are its assets, what was its return on assets, basically any financial variable that’s important in accounting is reported on a yearly basis in this data set called Compustat.

Now, remember that the whole point of this exercise is to figure out the effect of leveraged buyouts by private-equity firms and the effect that it would have on bankruptcy. So, we also need some data on what firms were bought out. That’s not necessarily going to be in Compustat, so I decided to start with a different data set called Capital IQ that has some better data on what sort of transactions they were exposed to. So first, I went to Capital IQ, I downloaded a list of firms that had been bought out through a leveraged buyout. And then the idea was to match it to Compustat.

And, fortunately, they both have a unique identifier that allows me to do that match relatively easily. So now I have a set of formerly public firms that were bought out by private-equity firms, and I have as much financial-reporting information as I can collect about them.

Luigi: One of the first questions is to figure out how many observations we have. If we have only very few observations, it’s very hard to get a difference that is statistically significant. Even if the difference in magnitude is large, with very few observations, the variability is so large that you cannot conclude from a statistical point of view that one group is different than the other.

Kate, how many companies did you end up finding in this situation? And, by the way, what is your time period?

Kate: I started with a set of firms that experienced LBOs from 2000 to today, this is retail by the way. For all of those retail firms, I got Compustat information about their financial performance. And then I collected all other publicly available firms in Compustat that were also retail firms, so they’re the control group. At the end of the day, there were only 56 firms that were public and retail that experienced an LBO between 2000 and today.

Luigi: And how many publicly traded firms that did not experience LBOs are there?

Kate: Roughly 400.

Luigi: Of these 56 firms, how many do you have data for after the LBO? Because sometimes companies still report to the SEC if they have public debt outstanding. And so even if they have been bought out, you can get financial information afterwards. So, how many of these 56 do have this financial information afterward?

Kate: It’s hard to say what’s good and what’s bad financial information, so let’s say, for example, Toys “R” Us was bought out, they were bought out in 2005. Often the way that it shows up in Compustat is that it’ll disappear after 2005 and then it’ll reappear 10 years later. So, does that even count if you have data on 2003, 2004, 2005, and then 2012, 2013, 2014? Is that even good enough data to compare? And if you look at the firms that are consistently reporting before and after the LBO, I would say there weren’t more than a handful.

Luigi: I guess that’s a big problem, because with a big gap, you don’t really know what is being stripped out and what isn’t. I’m not an expert on the Chrysler buyout by Cerberus, but my understanding is, in spite of the fact that Chrysler went bankrupt, Cerberus actually did make some money in the buyout. And the reason is that they stripped some assets that they sold separately, and so they recovered more than their investments ahead of time.

When you see these holes in the data, you don’t know whether this has taken place. And that’s actually one of the things that might be more worrisome, that there is some asset stripping going on.

Kate: Exactly. If we were going to do a really in-depth analysis of what’s going on with these private-equity firms after the buyouts, we would want a consistent time series, so yearly information, about what’s going on at these companies. How much are they investing? How much are they paying down their debt? Are they selling any of their assets? But if there are huge holes, then we can’t answer that sort of question.

What we can see at the end of the day is which firms went bankrupt and which firms didn’t. But even then, it’s not like we have a year-by-year measure of some sort of distress intensity. We just have a number that equals one if the company eventually went bankrupt and zero if it didn’t. Now, most of the time, summary statistics, just averages on what happens, don’t really mean a whole lot. But in this case, given the limitations of our data, there isn’t a ton that we can do.

Just reporting the summary statistics, I can tell you that of the firms that experienced leveraged buyouts between 2000 to roughly 2017, about 20 percent of those firms eventually went bankrupt. If you compare that to the firms in the same industry, in the same retail industry, that didn’t have leveraged buyouts, only about 6 percent of them went bankrupt. So, just looking at the raw numbers, there does seem to be a difference.

Luigi: Yes. This is the tricky part, because the magnitude of this difference seems quite large, and so it’s very tempting to conclude that one causes the other. But people die at the hospital much more often, that doesn’t mean that the hospital causes the death. So, we need to somehow try to figure out why these firms did an LBO, and then to what extent the firms that did an LBO are different than the ones that didn’t.

Remember, the gold standard is the randomized trial and control. Here, we cannot afford to have the randomized trial, and so we are trying to see how we can produce something that looks like a randomized trial by controlling for some characteristics.

Kate: Here’s where a little bit of statistics comes in. This might seem like a good sort of setup for a regression, which is just a statistical analysis that economists use to figure out the relationship between various predicting variables and various outcomes. In this case, economists would do what’s typically called a difference-in-differences regression, where you look at the group that was treated, the group that had the LBOs. You compare that to the control group, the ones without LBOs. And then you compare their differences before and after the LBO treatment, so difference in differences.

Unfortunately, because of the limitations of our data, as we just mentioned, most of these firms don’t report after the LBO. We don’t know what happened to them after they went private. And so, it’s really hard to do a difference-in-differences sort of experiment here. Essentially, the only real measure of ultimate firm performance that we have is just whether the company eventually went bankrupt.

In some sense, even though it would be nice to take advantage of how the company was changing over time, we don’t have that time series data for the private-equity firms, the ones that were bought by private-equity firms. And so, instead, we’re kind of limited to what’s called a cross-sectional sort of analysis, where you’re not really taking advantage of how the company changes over time, you’re just looking at a fixed point in time.

Luigi: But even if we had all the full data and we could do a difference in differences, I don’t think that, in this particular case, this will alleviate our concerns about what we call in jargon the endogeneity of the choice in the LBOs. The fact that if you take two firms, let’s say Toys “R” Us and Sports Authority, and one chose to do an LBO and the other didn’t, there is a reason why one did an LBO and the other didn’t. And maybe that reason is that this company saw the writing on the wall faster or needed to try to avoid bankruptcy faster.

Or their financial situation was worse than what appeared in the books. All this stuff is plausible, but it’s stuff that we do not observe, and it’s very hard for us to know whether the difference that Kate just mentioned in the percentage of bankruptcies is driven by these initial differences we don’t observe or really caused by the buyout itself.

Kate: Totally agree. Not only can we not really address the question of causality very well, because there’s a bunch of stuff that we can’t observe that might have been affecting the buyout decision that also affected the eventual bankruptcy, but we don’t even have enough data to really do a meaningful difference-in-differences analysis.

Instead, we’re kind of left to a different type of statistical technique, this one a little bit more intuitive in some sense, but still involving math, which is called propensity score matching. The whole idea behind propensity score matching is that, OK, you have one firm that experienced an LBO, and you know whether or not it went bankrupt later on. You want to find a control group that looks exactly like that firm. Ideally, you would know everything that caused the firm to go bankrupt or not go bankrupt later on, and so you would match that treated company to another company that was identical along all of those dimensions, except it didn’t experience an LBO. So, you have to find a way to match one company to another company that looks virtually identical to it in some way.

Luigi: As the more attentive listeners might have noticed, there’s an intrinsic contradiction in this propensity score method. Because if two companies are really, really identical in every respect, why did one do an LBO and the other didn’t? One possibility is that God was playing dice and decided that one would do an LBO, the other didn’t. That is what we hope for, because that’s basically the randomization that we’re looking for.

But short of this pretty crazy hypothesis, I think that the risk is always that there is an underlying difference, and this is exactly what we’re not capturing. So, you’re kind of pushing under the rug the intrinsic cause that led one company to do an LBO and the other not to do it.

Kate: The attentive listener might also have noticed that it’s kind of confusing how you should even do matching here, right? Because for the firms that didn’t have an LBO, you’ve got a bunch of data. You’ve got 2001, 2002, 2003, et cetera through 2019, say. But then for the LBO firms, they drop out once the LBO happens. So, what are you matching to what? Are you counting a firm as LBO before it experienced the LBO, and if so, how long? And are you matching every before year to every control year, or are you limiting within-year? I mean, there’s all of these sorts of practical issues about what to do here.

So, Luigi and I did something pretty simple, we restricted the data set for the firms that had LBOs to the year in which it was LBO’d. So, Toys “R” Us, for example, only shows up in 2005, whereas firms that didn’t have an LBO, we’re going to include all of their data.

Then we did the matching, so we came up with propensity scores for each firm that experienced an LBO to firms that didn’t, but we only matched within-year. For Toys “R” Us, let’s say, we looked at the firm that looked most similar to Toys “R” Us, except only in the year 2005.

Luigi: Yeah, but Kate, you need to explain how you define similarity, because similarity can be multidimensional, and there is an important choice in the variables you match along.

Kate: Exactly. How do you know what’s similar and what’s not? At the end of the day, we used variables that are somewhat common in the literature, measures of firm size, firm performance, and leverage. So, what we did was that we used return on assets, leverage ... for firm size, we used market cap, the total value of the company’s shares times the number of shares outstanding. We looked at Tobin’s Q, which is just a comparison of market to book value.

We used cash that the firm had divided by assets, because the more cash you have, the less likely you are to default, and then a measure of the company’s sales. At the end of the day, we do this analysis, what we get is called an average treatment effect, where the treatment was the LBO, and remember we’re matching here. Matching what to what? You can kind of figure out how you want the matching to be. So, I can find one non-LBO’d firm for every LBO’d firm, I can find two, I can find any number of matches.

I just decided to keep it simple, I tried one and I tried two. In the case of one match, the experiment tells us that the average treatment effect of the LBO is about 14 percent. I did something fancy called bootstrapping my standard errors, and it shows that actually that’s pretty significant. It’s significant around the 2 percent level, which tells us that we can be relatively confident that this association is there between the LBO and the eventual bankruptcy.

And if I expand my analysis to match every LBO’d firm to two companies, I get something very similar. I get 15 percent, with roughly the same sort of level of significance. So, what do we get from all of this? The propensity score matching just tells us that the average treatment effect of the LBO was roughly 15 percentage points.

Luigi: At the end of the day, it’s not very different from what we obtained simply looking at the summary statistics of the two samples. That’s kind of, in my view, bad news, because this is saying that either there was no difference between the two samples to begin with, or that our propensity score method did not capture these differences very much. And so, we’re left with the same number we had to begin with, the naïve number. So, a lot of effort for not a lot of additional output.

Kate: We talked about some of the issues here, right? There’s issues with the data, there’s not that much, the PE data drops out. At the end of the day, our fancy matching results aren’t that different from just looking at the averages of one group over the other. Even if we had done this perfectly, Luigi, what are some of the other issues that we would have faced?

Luigi: One important issue is that even if you have controlled perfectly for the probability of bankruptcy before the buyout, there is an issue of differential cost of bankruptcy, if you do go to bankruptcy, what we economists call a cost of financial distress. So, if I think about taking on more debt for a company, I am more likely to take on more debt if I think that the costs I incur in case of bankruptcy are smaller.

On the other hand, if I know that the costs of financial distress are very large, then I stay away from taking on a lot of debt. For example, we don’t see any LBOs of pharmaceutical companies. Why? Because R&D is so important in pharmaceutical companies, and you don’t want to jeopardize the development of new products by putting on a lot of debt, so the cost of financial distress in R&D is very large.

On the other hand, retail stores, especially the ones that have a lot of physical assets, tend to be a primary candidate for an operation like this. But now think about two retail stores, there is one that owns a lot of real estate, the cost of financial distress might be very limited. On the other hand, there is a new online retail store. If things don’t go well, much of the value, not to say all the value, disappears, then the costs of financial distress are, are very high.

In that sense, you might be concerned that the old-fashioned brick-and-mortars, the ones that, by the way, are more likely to see the writing on the wall, are the ones that are more likely to do the LBO. And on the other hand, the more dynamic ones are the most likely not to do an LBO. And so now, again, we’re back to the hospital analogy where the LBOs are simply the manifestation of an underlying process rather than the cause of it.

Kate: Another point that I want to raise is that maybe an LBO is not a great measure of private-equity involvement, because there’s two different things that we can’t really disentangle here. One is the direct effect of having a bunch of debt loaded onto a company. The other is the effect of having private-equity firms actually make strategic decisions about the company’s future once they own that company.

And in theory, we’d want to be able to disentangle those two. We’d want to say, “All right, maybe the debt caused the company to eventually go bankrupt, but the private-equity firm actually did a better job of managing the company than the old owners would have.” But in the way that we’ve measured it, with leveraged buyout as our measure of private-equity involvement, we can’t disentangle those two things.

Luigi: Most people, especially in our profession, think that if a company is run by somebody who owns most of the equity, it is run in the most efficient way. And this is true to the extent that you cannot play games, you cannot strip assets, but one possibility is that you buy a company that is on the verge of a collapse. And what you do is you try to strip as many assets as possible from the company before they collapse, and then this company goes into bankruptcy.

You can extract some value, but the person that is able to run the company this way is not really adding value to society, it’s just appropriating value at the expense of other parties. In this particular case, most likely the bondholders but also the workers and potentially the customers. So historically, this particular risk was minimized by the fact that there is a long tradition both in common law and in civic law to go after people who strip assets, a process that in law is called fraudulent conveyance.

Now, is that still true that this fraudulent conveyance is aggressively pursued in the United States?

Kate: You raise a great point, and you’re not the only person talking about this. In fact, Elizabeth Warren has proposed a bill to prevent private-equity firms from being able to do this, from being able to just strip companies’ assets and push them into bankruptcy at the expense of potential other claimants such as creditors or even employees. We’re going to have get back to you with these answers in one of our future episodes.

We’re going to give you the fairer treatment of whether or not private equity is, let’s just say, good or evil by looking at the rest of the literature, the published literature, and what they’ve found about the ways that private equity in some cases helps and in some cases may hurt the companies that they invest in.

Luigi: I think that many listeners are probably left with kind of bad feelings. And the feeling is that we as economists are not very good at disentangling phenomena that are very important and that for which we don’t have definite answers, in the sense that the fact that we can’t really separate those two hypotheses very well is such that at the end of the day, we observe a huge increase in bankruptcy as a result of LBOs, but we cannot conclude one way or another.

Kate: I think this is an interesting point and maybe sort of a meta-point about data and economic research. I mean, the whole process we just described of collecting this messy data and running these best-shot regressions or analyses that are somewhat incomplete and still have a bunch of issues, that sort of analysis would never pass muster. We could never publish a paper like this, at least not in what are considered the good finance journals.

But that leaves us sort of unable to answer this question. I mean, unless someone’s truly brilliant and they find this great data resource that we don’t know about, it’s really hard to answer the question of whether private-equity firms through leveraged buyouts caused or hastened the retail apocalypse. It may be entirely impossible to answer that question. So, does that mean that we just shouldn’t try to ask the question at all?

Luigi: But more importantly, what should policymakers do? In the absence of better, more refined data, is the simple differences in average good enough, or is the difference in average adjusted for your fancy propensity score method good enough for a policymaker? As academic economists, we tend to actually ridicule policymakers who use these simple averages with the examples like the one I gave of the hospitals and people dying in hospitals. But if we don’t provide better data, what can they do?

Kate: I think that we should try to provide better analyses. I think that the way that things are now are such that we only strive for perfection but in a very limited set of circumstances. And oftentimes, as you mentioned, policymakers are interested in the big-picture questions that we can’t provide perfect answers to. And so, what they end up doing is relying on very simple analyses.

I think this is an important question, what to do about dying retail firms—whether, if anything, there should be some sort of policy intervention. This is important stuff that policymakers are talking about, and yet we don’t have enough to really allow them to inform their decisions. I think that analyses like the one we just described, flawed as it may be, are better than nothing.

And, you know, we might just be at the end of the day comparing summary statistics, but at least those summary statistics were the result of thinking carefully about what the sample was. I would like to see more types of papers that have limited data, limited causal potential, but still at least are upfront about all of those issues and just present whatever they can in the best way that they can but for important questions that people are talking about.

Luigi: The way I like to summarize this tension is that there is a trade-off between saying “nothing precisely” and “precisely nothing.”

Kate: And precisely nothing.

Luigi: And I think that, unfortunately, the academic profession is going very much in the direction of saying very precisely something not that relevant. I think we should fight this and try to actually think about what is relevant and do our best. This is not an excuse in my view to be sloppy, it’s not a justification for being sloppy, but it is an encouragement to tackle the most important problems.