Ensuring That Real Information And Analysis Impacts Decisions

We are in an era of instantaneous and inexpensive information and communication, and we have learned that the old methods of filtering fiction from fact no longer work. Recently, the New York Times fact-checked President Trump’s first hundred days in office and found “at least one false or misleading claim per day on 91 of his first 99 days (Saturday is Day 100). On five days, Mr. Trump went golfing, and on two he made limited public statements.” If the President of the United States is an unreliable source of information, it is difficult to see where we might go for an authoritative voice to present and verify information. For public policy and economic analysts, U.S. government documents represent a traditional, objective source of information. If the head of that government lies, how can we trust the data of the agencies that work for him?

Before the Web, when information dissemination required expensive print publication or broadcast resources, there was clearly a bias in the information disseminated that favored the economic interests controlling those media outlets. In other words, information has often been biased and rarely truly “objective.” But the volume of it was low enough to be digested, considered, and either utilized or ignored by decision makers. The best leaders sought multiple sources of information. Franklin Roosevelt could not travel easily as president, but Eleanor Roosevelt traveled extensively and brought FDR first-hand accounts of the condition and mood of the country. Direct observation, reports from trusted media observers, and the opinions of experts are the traditional sources of information for decision makers. In a more complex world, the analytic work of academics has also become a trusted source of guidance.

As the world became more complicated, we added data, often collected by the government, and sometimes an analysis of that data to the decision-making process. Here in New York City, the COMPSTAT crime tracking system provides real-time data on where and when crimes are being committed. That information cannot possibly be 100% accurate when NYPD management receives it, but they use it immediately as an input to police deployment decisions.

Increasingly, our personal and organizational decisions depend on electronic reports of trending behavior. Waze helps us plot our trip to the airport; Yelp helps us decide where to go to dinner; course evaluations help us decide whether to promote a professor. But how do we know that this data is reliable and valid? How was it collected? How is it verified? And how do I judge if the information is real or fake?

Direct experience with a terrible restaurant may cause you to discard Yelp and go back to Zagat’s. Photos comparing Obama and Trump’s inaugural crowds can be superimposed on each other and that can shed light on President Trump’s claims. The process of distinguishing fact from fiction, myth from reality is not a new one, but in a world overwhelmed by complexity and data, it has become more important.

Information is more valuable because we are more dependent on each other for survival than ever before. In 1900, 40% of Americans worked in agriculture. Today that number hovers around 1%. Most of us could not figure out how to grow or gather food, make clothing or build shelter if our life depended on it. And of course, our lives depend on food, clothing and shelter. But in our complex global economy, we rely on others to provide those necessities and our work becomes increasingly automated and specialized.

Our complex society and networked economy is built on a foundation of information. For example:

• Publicly traded companies are required by government to report financial data and hire a qualified outside auditor to verify the accuracy of that data.
• Many organizations are trying to develop or use indicators to measure their sustainability, although auditing is not yet required.
• Inspectors from New York City’s Department of Health grade the cleanliness of restaurants and ensure the grade is prominently displayed in the restaurant’s window.

We see a growing number of attempts to collect, analyze and utilize data in everyday life and in management decision making. We make efforts to compare that data to direct experience. If an air pollution report is positive, but the sky is orange, we question the data. If the restaurant receives an A and we see a mouse running by, we question that grade. But questioning can only carry you so far, and we rely on authoritative verification of facts and cues from experts to navigate our daily world.

Moving away from the individual level to the organizational level, the volume and complexity of information increases exponentially. One of the key decisions a senior executive must make is what information should be the focal point of decision making? If it’s a university I can count admit rates and application data, student demographics, tuition revenue, job placement data, grants and gifts, faculty retention and diversity rates. But how do I know if a disruptive competitor is ready to launch? If it’s a software company I can analyze revenues, costs, profits and capital investments. But how do I know if social trends are moving away from the functions our software facilitates? In short, along with the quantitative data that has the comfort of specificity and even precision, we also need information on the social, political and economic context of the work we are engaged in. Some of these contextual variables can be measured, but some are so difficult to measure that they are not worth the effort. They will need to be described and understood, but not measured.

It is obvious how important information is, but how do we bring it to bear on decision making? How do we bring new knowledge to decision makers and convince them to pay attention to the information and possibly act on it? President Trump is a good, but far from unique, example of a decision maker who has survived in management by an idiosyncratic method that he views as self-justifying. He made money, he became president, therefore in his view, his methods lead to success. Sales techniques used to pretend a 20-story building is 30 stories tall are simply adapted to advocating a health care bill that somehow “costs less and provides more.” Information is combined with hype in a sales pitch. Decisions require facts, but salesmanship is another story. A decision maker with Trump’s approach will only accept facts that feed into his narrative. If analysis can be presented to a decision maker that is consistent with the “story line,” the new information may well become the basis for decision making. But inconvenient facts (or truths) will be very difficult to communicate to this type of decision maker.

In contrast to Trump’s approach to data, one can look at former New York City Mayor Michael Bloomberg. Bloomberg learned that his city would add 1,000,000 people in a couple of decades and after checking the data, developed New York City’s sustainability PlaNYC 2030. He took the data seriously and wanted to ensure that growth added to the city’s quality of life and did not detract from it. He was a decision maker who used the data he had and collected data he needed. To develop a measure of “customer responsiveness”, he took the city’s new 311 non-emergency response complaint phone number and used it to develop a database to assess agency response to public requests for service. He then judged agencies on their responsiveness.

The attitude of the CEO to information is critical, but we still have the issue of hacked, false and misleading information all over the web. We have the problem of people who make money from lying on the internet. And sometimes those lies compete with facts for public attention. The Nazis did it with the Big Lie, and we read about it in Orwell’s 1984, a novel that tried to define: “war as peace, freedom as slavery and ignorance as strength”. My fear is that this is starting to be seen in the American political media. The new technology of information, computation and communication requires that we develop new methods for verifying facts and data. We don’t yet know how to do this, but we better figure it out fast.