RAND study shows 9 million signed up off exchanges - here's why that's relevant

Oh goody!!!!

So a RAND study is apparently coming out, and I’m still quite unclear on what it actually shows. The result Avik Roy is emphasizing is that ~1/3^rd of those who bought on the exchanges were previously uninsured. I’m going to go ahead and assume that’s actually NOT true, and that either the study’s methodology was flawed or, more likely, the interpretation of the data was flawed. Obviously, this wouldn’t be the first time. Exhibit A: The infamous McKinsey study, which has been and continues to be erroneously cited as showing that only 27% of those buying on the exchanges were previously uninsured.

What I AM willing to accept, for now, is the additional finding the study supposedly reports, which is that NINE MILLION PEOPLE have bought off the exchanges. If you’ve read Charles Gaba’s multiple entries on the McKinsey study, and perhaps also my piece elaborating on off-exchange enrollments, you’ll know that understanding the McKinsey poll depends heavily on knowing how many people enrolled off the exchanges. I made an estimate that ~36% of people enrolled off the exchanges, but that was based purely on estimates from numbers reported by companies that have BOTH on- and off-exchange plans – this COMPLETELY IGNORED the reality that there are several HUGE companies (e.g. Aetna) selling plans ONLY off the exchanges.

Just as importantly, the McKinsey study oversampled the previously insured by ~2.4 (they chose 1,000 insured and 1,000 uninsured from a total population of 10 million insured vs. 24 million uninsured). This skews their results to be even more biased towards previously insured people. Based on that alone, we’d expect a lot more people to have been previously uninsured, but we can’t be certain since they apparently also tried to correct for other factors. The point is that it ISN’T a random sample. I’m guessing the RAND study wasn’t a perfect random sample either – I’ll keep you posted on that.

So, in any case, the number was likely to be much higher, but I preferred to stick with a “conservative” (nyuck nyuck) estimate.

Well that is no longer necessary. Now that we know that there were 9 million off the exchanges (assuming of course that the RAND study is correct in this), we can make some much better estimates of what the proportion of on vs. off exchange enrollments were, and thus estimate the number of previously uninsured people who enrolled on the exchanges. Let’s assume that the survey was taken very recently, say when on-exchange enrollment was 6 million. That means that there were about 67% of people buying off the exchanges. At the time of the McKinsey study, 27% of the people buying insurance were previously uninsured. That proportion had gone up from 10% the month before, and I think we can assume that the rate only continued to increase, especially in the last couple of days, but to be conservative (again), let’s assume that it stayed flat at 27%. That would mean that 73% were previously insured. 73% of 15 million (6 million on the exchanges, 9 million off the exchanges) is ~11 million. That means that ~4 million were previously uninsured. Now, the RAND study also said that the “vast majority” of those who bought off the exchanges were previously insured. I’m going to go ahead and assume that means 90%. This of course means that 10% (900,000) of those who bought off the exchanges were previously uninsured. Let’s round that up to 1 million. Subtracting 1 million from 4 million gives us 3 million previously uninsured people who bought on the exchanges. That puts us at 50% of those who bought on the exchanges were previously uninsured (exactly what Gaba predicted, by the way)!

Here’s the thing – I’m CERTAIN this is an underestimate. The fact is that a HUGE surge has happened in the last couple of days, and the majority of those signing up in that last surge are more likely than not to be previously uninsured.

A reasonable estimate for the number of previously uninsured enrollees on the exchanges?

With regards to the ACA, many in the conservative media have inverted the old practice of "chocolate-covering shit", and have instead regularly partaken in "shit-covering chocolate". For example: There have been lots of claims out there about how many (or, rather, how few) people signing up on the exchanges were previously uninsured. This is a major talking point for opponents of the ACA, since one of the main goals of the exchanges was to put a significant dent in the number of uninsured Americans. Most commentators have been using a McKinsey study as the basis for their claims that only ~14% of people enrolling on the exchanges were previously uninsured. The problem is that the McKinsey report didn't distinguish between on- and off-exchange enrollments. 

Charles Gaba (the greatest data wonk paying attention to Obamacare - even better than Nate Silver) has carefully explained why this matters; why therefore the McKinsey report cannot be used to determine how many enrolling on the exchanges were previously uninsured:

The Exchange-Based QHP enrollments were at around 3.6 million at the time of the McKinsey study. Let's say that Off-Exchange QHP enrollments were also at 3.6 million, for 7.2 million total.

McKinsey is saying that 27% (in this example, around 1.94 million of the combined total) were previously uninsured, while 73% (5.26 million in this example) were already insured.

However, without knowing how many of that 27% came from the ACA exchange pool and how many came from the off-exchange pool, this tells us absolutely nothing about what percent of ACA exchange enrollees were previously uninsured. 

Perhaps all 3.6 million off-exchange enrollees already had insurance, and only 1.66 million of the exchange-based enrollees did. That would mean 54% of exchange-based enrollees were uninsured, not 27%.

Or, perhaps it's the reverse, and every single exchange enrollee was already insured (0% uninsured, a disaster!).

I'm not saying that either of these is the case; the point is that by mixing the two together, the only conclusion we can draw from this is that the correct number is somewhere betwee 0% - 54%, which isn't terribly helpful.

But wait, it gets worse!

The above example assumes that an equal number of exchange & off-exchange enrollees are out there, when in fact we only know one of the correct numbers: 3.6 million exchange-based as of mid-February.

The off-exchange number could easily be 10 million. Or 5 million. Or 1 million. I know it's at least 465,000, but that's only out of a handful of companies and one state.
So, if the off-exchange number is 10 million (or, for that matter, anything higher than 5.3 million), that could, in theory, mean that every single previously insured enrollee was done off-exchange, and 100% of exchange enrollees are newly insured. This obviously isn't the case (my own family switched from a non-compliant policy to a compliant one via HC.gov), but you get my point."

Charles is correct that we don't know how many people have enrolled off the exchanges. But we CAN make a reasonable estimate - if properly motivated to do so. We could also make some really crappy estimates: After a bit of back-and-forth tw**ts between Charles and Avik Roy (a Forbes opinion editor and Mitt Romney health policy advisor, and generally a brilliant expert on health policy who actually cares about expanding health care coverage in the USA - a rarity in conservatives, unfortunately), Avik updated his article, and cited an article by Scott Gottleib (an AEI health policy expert):

"...we have actual data from insurers on this topic; most analyses estimate that approximately 20 percent of sign-ups to date have been off the Obamacare exchanges.

If we (generously) assume that every single off-exchange signer-upper was previously insured, in order to make the math as flattering to Obamacare as possible, and that 20 percent of sign-ups were off-exchange, we arrive at the following paid enrollee breakdown on the exchange: 76 percent previously insured, 24 percent previously uninsured. 

24 percent is better than 14 percent, but not so much better as to justify a furious defense of the exchanges’ performance in this regard. And the percentage of off-exchange uninsured signups is unlikely to be zero."

While I appreciate Avik's effort to be "generous", he left much to be desired. What Gottleib's article actually says is "Off exchange enrollment could be as high as 20%, according to one Wall Street analyst."

I'll let you decide whether that one Wall Street analyst counts as "most analyses". Interestingly, later on in his article, Gottleib says:

The one data point we have on “off exchange” enrollment trends comes from Highmark, which recently announced that 31% of the 133,000 people it has enrolled across the three states where it operates (Pennsylvania, West Virginia, and Delaware) signed up off exchanges.

So, already, we know that Avik's 20% estimate wasn't quite as "generous" as he would have you believe. Charles Gaba does a brilliant job debunking Avik's logic - I suggest you read the entire thing. In summary, Roy cherry-picks minimal data, while ignoring Gaba's data showing that, actually, a considerably higher percentage of people have been enrolling off the exchanges. Let's take a look at all of the data we have on off-exchange enrollments, thanks to Charles' incredible work (off-exchange/total; percent off-exchange):

Washington: 184,317/268,604; 69%

Wisconsin: 7,900/57,900; 14%    

Wellpoint: 100,000/500,000; 20%

Coop of IA & NE: 32,252/50,178; 64%

Highmark: 41,230/133,000; 31%

Add them all up, and we get a total of 365,699/1,009,682; 36%. No, this isn't a total, comprehensive number, but it gives us a very reasonable estimate, based on one million people. It is certainly much more comprehensive than using just Wellpoint's survey, which actually included about the same number of people (~500,000) as the Washington, Wisconsin, Coop, and Highpoint numbers added together (509,682). More importantly, Wellpoint is one specific company with one specific portfolio/approach/model (more on that below).

Interestingly, the Forbes article reporting Wellpoint's numbers claims that Wellpoint "tends to have more business offered to individuals and those covered by Medicaid who would be newly insured under the health law." That suggests that Wellpoint works with more lower-income folks, who should gravitate towards the exchanges for subsidies, and thus might not actually represent the typical mix of on vs. off-exchange enrollees on a national scale. But here's the really interesting part (emphasis mine):

"Though it’s early in analyzing the data from new applications of for coverage, Wellpoint executives said “80 percent were not previously insured by Wellpoint,” executives said on the call this morning.

It’s unclear whether they are newly insured or came from another plan, but executives said they were pleased the company was benefiting from new customers."

If 80% weren't previously insured by Wellpoint, then 20% were. My feeling is that the 20% of enrollees that Wellpoint reported as off-exchange were people who were insured through Wellpoint before the exchanges opened. They are now entirely focused on enrolling via the exchanges. I could be wrong, but this seems like a logical explanation for why their off-exchange numbers are so much lower than what we see practically everywhere else. 

It should also be mentioned that there are many companies that aren't offering any plans on the exchanges in certain states - none of these companies (such as the giant Aetna) have reported their enrollment numbers for those states, as far as I know, but they are mathematically certain to increase the proportion of off-exchange enrollments.

So, hopefully, we can agree that 36% is a reasonably conservative estimate for off-exchange enrollments.

Using that number, and given that we're now at 6 million enrollees on the exchanges, we now have 9,375,000 enrollees total, and we have 3,375,000 off the exchanges.

Using McKinsey's report that 27% of enrollees were previously uninsured, we get that 73% of 9,375,000 (6,843,750) were previously insured. Note that this number has almost definitely increased by now, just as it had increased by ~250% (from 11 to 27%) between January and February, but probably even more so now that The Surge is well underway. But let's stick with 27%, just to be "conservative" about it. If we're assuming that all who bought off the exchanges were previously insured (as generously suggested by Avik Roy), that means that 6,843,750 - 3,375,000 = 3,468,750 of the previously insured bought on the exchanges, which means the remaining on-exchange enrollees (6,000,000 - 3,468,750 = 2,531,250) were previously uninsured. Which means that 2,531,250/6,000,000 = 42% of enrollees on the exchanges were previously uninsured.

Are those numbers great? Not in my opinion. Also - I'm sure that not all of the off-exchange enrollments were previously insured. However, if you want to correct for that, then you also have to correct for the fact that many companies which only sell off-exchange aren't included in these surveys - my guess is that they at least balance each other out. In any case, I think these numbers are much closer to reality than the 27% or even 14% that we've been reading everywhere else. Most notably, it's quite close to Gaba's original 50/50 prediction.

Here's a summary of some key points:

1. Kaiser reported that 18% of uninsured attempted to enroll on the exchanges, while 13% attempted to enroll directly from the insurer (off-exchange). Assuming that these "attempts" are directly proportional to where people actually end up enrolling, then 58% would be enrolling on the exchanges vs. 42% off the exchanges. 42% isn't too far off from my 36% - perhaps the "missing" 6% is from companies that are only off-exchanges? In any case, if this simplified assumption is true, then 58% of the enrollees on the exchanges are previously uninsured, which is significantly higher than my conservative 42% estimate.

2. Off-exchange enrollees are, in my mind, much more likely to have already paid (just send in a check with the enrollment application), whereas on-exchange enrollees may not necessarily have paid yet. I emphasize "yet", because as Charles points out, most enrollees who haven't paid yet are still very likely to pay in the near future. Just like people don't enroll in plans before the last minute, people don't pay until the last minute, either. Add to that the fact that some people are considered enrolled before they even receive their first invoice, and the whole "who paid?" argument gets blown to smithereens. In any case, the McKinsey survey was taken at the beginning of February, which means most new enrollees didn't have to pay until March 1st. My guess would be that at least 90% of reported enrollees end up counting in the end. Data has already rolled in on cancellations due to unpaid premiums, and it looks like only ~7% of people lost coverage due to delinquency. I wouldn't be surprised of some of them recover coverage after showing the delinquency was due to technical errors.

3. Many providers are only offering plans off the exchanges. All of the data we have for on vs. off-exchange enrollment seems to be from companies that offer both. This means there could be quite a lot more people enrolling off the exchanges than we can currently estimate. As you saw from the math we did above, the more previously insured people we find off the exchanges, the higher the proportion of previously uninsured people we know enrolled on the exchanges. Unless I'm missing something here, this would suggest that my 42% number is likely to be an underestimate.

4. We can assume that The Surge we're seeing now consists of almost entirely previously uninsured young people, based on what we've seen at eHealth. One more reason to think that 42% is an underestimate.

5. I personally believe we can assume that lots of those whose plans were canceled in October automatically enrolled in or chose new plans directly from their existing insurer. This means that potentially a huge number of off-exchange enrollees would be "previously insured". This also reduces the horrid number of people who "lost coverage" due to ACA-induced cancellations. 

6. One last VERY IMPORTANT point made by David Smith:

27% was incorrect even at the time of the study given that they over sampled the previously insured. It is 27% of the respondents but 49% of the population. Details below from previous comment.
The issue with the McKinsey Study is that the respondents are not representative of the QHP eligible population. They clearly state that their population from which they drew their "adult survey respondents is approximately 24 million uninsured and 10 million individually insured." So 71% uninsured. However, among the respondents only 48% were uninsured (1,013 uninsured and 1,083 insured). They over sampled the previously insured. If we apply the percentage of respondents who selected a plan for each group (10.5% for uninsured and 26.7% for the insured) the percentage of uninsured selecting a plan would be 49% with total QHP enrollment of 5.2 million. As mentioned earlier this includes off-exchange enrollments. The survey was Feb 4-13. ACAsignups had estimated about 3.3 million exchange based signups at this point suggesting about 1.9 million off-exchange enrollments.

All in all, I think there's a very reasonable case to be made that the ACA has significantly expanded coverage. Not just for the previously uninsured, but also for the previously underinsured.