All Commentary
Wednesday, April 11, 2018

Why Big Data Won’t Save Central Planners from the Knowledge Problem

Consumer data have to come from somewhere, and that somewhere is markets.

Among statisticians, economists, and business executives, “Big Data” is all the rage. Large and detailed data sets that, until recently, couldn’t even be stored on a computer are now managed and analyzed using innovative statistical techniques. Hopes are high that these advances will improve scientists’ ability to predict human behavior. Some enthusiasts even speculate that Big Data will render markets obsolete, enabling central planning of the economy. Big Data is more than a buzzword, but its potential is often wildly overstated.

How Are We Using Big Data?

To make sense of enormous databases, statisticians have developed innovative analytical tools, including machine learning, A/B testing, and natural-language processing. Storage and computation speeds have also improved in recent years.

Google constantly updates its search algorithm to make search results more relevant.

Firms are seeking to capitalize on these advances. According to a survey conducted by Dresner Advisory Services, 53 percent of companies utilize Big Data in some capacity as of 2017. quotes average salaries for data scientists at $120,000, indicating a high demand for their skills.

More and more, firms use algorithms to spot market trends and predict consumer behavior. 21st-century consumers constantly interact with such algorithms.

When Spotify customers open the “discover” section of their mobile apps, a proprietary algorithm uses information about past music listening habits to create playlists tailored to their tastes. Google constantly updates its search algorithm to make search results more relevant. Investment giant Blackrock, which boasts over $6 trillion in assets under management, recently replaced several portfolio managers with a sophisticated trading algorithm to inform stock picks.

The risk of a data-fueled dystopia is not trivial.

Of course, Big Data raises important ethical concerns. Data are often drawn from oblivious populations. Although Internet users typically sign a Terms of Service contract granting social media companies permission to record behavior, the legalistic jargon in these agreements usually leave users unaware of the scope and scale of surveillance. The Cambridge Analytica debacle clearly demonstrates these problems (though it does not imply government would be a better steward of information).

Governments, on the other hand, make no such pretensions about obtaining permission. Officials in the Xinjiang region of China use Big Data analytics to track the movements of the Uighur ethnic minority. The Associated Press reported thousands of Uighurs have been sent to political indoctrination camps and prevented from communicating with relatives abroad. The risk of a data-fueled dystopia is not trivial.

Big Data Does Not Equal Big Knowledge

Some Big Data enthusiasts are somewhat reminiscent of 20th century “market socialists.” For example, a 2017 article by economists Binbin Wang and Xiaoyan Li of Sichuan University for the World Review of Political Economy contends that Big Data solves many of the problems faced by 20th-century socialism. Tech journalists and even entrepreneurs like Alibaba’s Jack Ma have expressed similar sentiments. Since today’s governments have far finer-grained knowledge of citizens, the argument goes, central planning could be more viable.

It is only from market transactions that prices emerge.

Big Data represents an important scientific advance, but it is fundamentally inadequate to achieve these more ambitious goals. The problem is not merely a time-lag of data collection or the inability to predict future innovations and sudden changes in preferences (though these limitations are also important).

The reason that Big Data can’t enable central planning is that any data on economic activity is inextricably predicated on the existence of markets. The algorithms which private firms use to better predict demand and supply rely on an incoming flow of market data. Without a market, that data ceases to exist.

It is only from market transactions that prices emerge. Consumer behavior is only conceivable when consumers have the freedom to choose between products. Profits and losses reflect the performance of actual firms and market participants engaged in rivalrous competition.

Governments who use Big Data to control their economies will face predictable consequences.

Information is not knowledge. Take away the market that produces economic data, and governments would be flying blind. What to produce? How much should be produced? What production processes should be used? Who should be employed in production? Eliminate the freedom of individuals to choose, and central planners would have no way to answer these questions despite possessing mountains of past information on their hard drives. Such knowledge simply can’t be generated otherwise than by the market process. All the data in the world can’t change that.

Big Data will likely continue to help firms improve consumer satisfaction, but these improvements in marketing and supply-chain management are inherently predicated on free markets. Grandiose predictions of a new scientific revolution to replace markets with socialism amount to updated Lysenkoist rhetoric. Governments who use Big Data to control their economies will face predictable consequences.

  • Matthew Kelly is a Research Fellow at the Colloquium for the Advancement of Free-Enterprise Education (CAFÉ) at UT Dallas.

  • Peter Lewin is a University of Chicago Ph.D. Economics Clinical Full Professor and an Economics Director at the Colloquium for the Advancement of Free-enterprise Education at the University of Texas' Naveen Jindal School of Management. Learn more about him at his faculty web page.