Medicare's new method for buying medical supplies and
equipment—everything from wheelchairs and hospital beds to
insulin shots and oxygen tanks—is doomed to face severe
difficulties, according to a new study by Caltech researchers.
The Center for Medicare and Medicaid Services (CMS) implemented
the purchasing process—a novel type of auction—in nine
metropolitan areas across the country last year and plans to expand
it to 91 in 2013.
The competitive bidding process was designed to improve the
efficiency of Medicare's procurement system, potentially saving
taxpayers hundreds of millions of dollars. But many experts have
criticized the auction, pointing out fundamental flaws in its
design. Now, a series of experiments testing the auction structure
show that it does, in fact, fall short of expectations. The results
of the study, which was conducted by Caltech seniors Brian Merlob
and Yuanjun Zhang, and Charles Plott, the Harkness Professor of
Political Science and Economics, were published in the May issue of
the Quarterly Journal of Economics.
In principle, auctions are a cheap and efficient way to procure
goods and reward companies that can inexpensively produce their
goods. In the case of the Medicare auction, various companies make
bids to the government that represent their best prices for medical
supplies and equipment; the government, in turn, wants the best
deal and so chooses the lowest bids. For example, if 1,000 hospital
beds are needed and it would take five companies to supply them
all, the five lowest bids are chosen.
Unlike standard auctions, however, the CMS auction was designed
with two unorthodox rules. First, the eventual selling price is set
at the median of all of the winning bids. Second, bids are
nonbinding, so companies can change their mind once the prices are
Critics say that these rules cultivate a harmful bidding
strategy. To ensure a winning bid, each company will make a very
low offer; this carries no risk, because the companies can cancel
their bid if the median price turns out to be too low. The result
is that participants in the auction will tend to make low-ball
bids, assuring that the median price will also be very low—so
low, in fact, that few of the companies can actually afford it,
leading them to cancel their offers. At the extreme, nothing is
bought or sold and, Plott says, "the auction crashes. It's just not
an effective auction."
And what will happen then, critics warn, is that the government
will end up negotiating prices with individual
companies—negating the whole point of a competitive-bidding
scheme in the first place. "You can see immediately from
theoretical arguments that the potential for disaster is built
right in the strategic structures," Plott says.
To determine if such theoretical predictions translate into
real-world behavior, it is essential to examine experimentally how
people behave in an actual auction, says Plott—who, for the
last 30 years at Caltech, has been developing experimental
techniques to test economic theories. He posed the problem to his
experimental economics (EC 160) class. Two of his students, Merlob
and Zhang, took up the challenge and spent a year researching,
designing, and conducting experiments to test that behavior.
The team used computers at Caltech and the University of
Maryland to run a simplified version of the CMS auction and several
other auction types; one, for example, followed more standard
rules, with binding bids and prices set at the lowest bid that did
not win, instead of the median of all winning offers. Each
auction involved 12 or 16 bidders (student volunteers from Caltech
and the University of Maryland), who first had to pass a quiz
showing that they understood how the auctions worked. The
volunteers were given just one item to sell—a generic "thing"
(since the bidders' behavior should be the same in a given auction
type, regardless of the item being sold)—each at a different
cost to them.
The Caltech team also examined the effect of other auction
features, such as whether the costs of each item for each bidder
are public knowledge and the effect of charging bidders to
The results, the researchers say, convincingly support critics
of the CMS auction design. "It's pretty disastrous what the bidders
ended up doing," Zhang says. In the simulated CMS-type auction,
some people bid $0, and the "government" was not able to buy all
the items it needed. The experiments also showed that a standard
auction is much more efficient and successful: the government was
able to buy all the items it needed, and the bidders who had the
lowest costs were the winners.
Using this experimental approach, the researchers were able to
pinpoint the fundamental problem of the CMS auction design: the two
rules. "If you just get rid of one of those two rules, it doesn't
help—you still have problems," Plott explains. "So you have
to get rid of both of them."
Last summer, 244 economists and auction experts, including Plott
and Caltech professors John Ledyard, Thomas Palfrey, and Matthew
Shum, signed a letter to urge President Obama to change the CMS
auction system; the letter cited the Caltech experiments.
In April, however, a study released by the Department of Health
and Human Services reported that the auction saved taxpayers $202.1
million in 2011, with no negative effects on health care. The
report also estimates that the auction will save taxpayers and
beneficiaries $42.8 billion over 10 years.
A preliminary analysis of the pilot program by Peter Cramton of
the University of Maryland, an outspoken critic of the CMS auction
and an economist who was not part of the Caltech study, found that
the auction did in fact suffer from the problems predicted by
theory and experiment. Because of the auction design, prices
plunged to unsustainable levels, and suppliers dropped out, forcing
Medicare to find new suppliers. Cramton also found that the number
of submitted claims for equipment declined, which, he says, led to
increased rates of visits to the emergency room and
hospitalization. As a result, not only were overall costs higher,
but so were health risks.
While Plott says he cannot comment on the report without knowing
exactly how the study measured and collected its data, he remains
confident in the experimental results and the theoretical arguments
against the auction. "The theory gives a rather clear picture about
the implications of the auction architecture," he says. "But only
the data can tell us how these ideas actually play out in such a
complex application with variables too numerous to be considered in
Regardless of whether the CMS will continue to expand the
auction or will heed the critics' warnings, this type of research,
Plott says, is a good example for how basic, scientific experiments
can have direct impact on society. He considers that an especially
important lesson for his students. Over the years, he says, several
of his undergraduates have stuck with a project long enough to
publish, occasionally leading to several awards and seminal papers.
"Taking Professor Plott's class was fantastic," Merlob says. "He's
a master experimentalist and an amazing mentor."
Zhang, who is doing an independent study in decision-making and
neuroscience, will start graduate school in economics at UCLA in
the fall; Merlob, a political science major, is still exploring his
options. "This entire experience was pretty awesome," he says.
"It's probably one of the best of my Caltech career."