Michael Cohen's Effort To Rig Reader 'Polls' Shows Exactly Why They Mean Nothing

Michael Cohen , President Donald Trump s former lawyer, promised an IT company tens of thousands of dollars to influence online reader polls like the ones conducted on the Drudge Report in Trumps favor, according to The Wall Street Journal . The results of those polls were often regurgitated by media outlets , conservative figures like Sean Hannity and Trump himself .

That Cohen was able to cook up such a plan highlights the glaring problems with reader polls, and why as weve written previously they shouldnt be confused with real surveys.

Scientific polling, whether conducted by phone, using an online panel, or in some other fashion, is fundamentally designed to be representative . It relies on some mix of sampling (choosing whos selected to take the survey) and weighting (adjusting the data to account for the fact that some types of people are more likely than others to respond).

Recent changes in technology have complicated that process , and even rigorous polling is far from infallible . But theres an enduring, basically sound underlying principle: making the pool of respondents look as much as possible like the larger population whose opinions the pollsters are trying to measure.

While theyre fun to take, reader polls fail on multiple levels as tools to gauge public sentiment.

Thats why, even if a scientific poll reaches only a thousand or so people, it can reflect the opinions of a broader group, whether thats something like registered Republican voters or the American public as a whole.

These techniques, of course, require pollsters to have a basic ability to control who takes their survey and to monitor the demographics of those who respond.

Reader polls, by contrast, offer none of that. While theyre fun to take, they fail on multiple levels as tools to gauge public sentiment. The people reading any particular website arent representative of the public at large. Those who take the time to read a particular story and to weigh in on it are even less so.

Would you be surprised, for example, to learn that HuffPost readers who clicked on a story about partisan acrimony are more politically argumentative than the average American?

For instance: 38% of a nationally representative sample of Americans said that they'd had an argument about the 2016 election. When I included a reader poll with the same question in a story, 72% of HuffPost readers who answered said that they had. pic.twitter.com/7UiwWnbLXy

Ariel Edwards-Levy (@aedwardslevy) January 17, 2019","type":"rich","meta":{"author":"Ariel Edwards-Levy","author_url":"https://twitter.com/aedwardslevy","cache_age":86400,"description":"For instance: 38% of a nationally representative sample of Americans said that they'd had an argument about the 2016 election. When I included a reader poll with the same question in a story, 72% of HuffPost readers who answered said that they had. pic.twitter.com/7UiwWnbLXy Ariel Edwards-Levy (@aedwardslevy) January 17, 2019\n\n","provider_name":"Twitter","thumbnail_height":321,"thumbnail_url":"https://pbs.twimg.com/media/DxHyQWpXcAAUTIB.jpg:large","thumbnail_width":579,"title":"Ariel Edwards-Levy on Twitter","type":"rich","url":"https://twitter.com/aedwardslevy/status/1085915353727799296","version":"1.0"},"flags":[],"fullBleed":false,"options":{"theme":"news","device":"mobile","editionInfo":{"id":"us","name":"U.S.","link":"https://action.news","locale":"en_US"},"originalEdition":"us","isMapi":false,"isAmp":false,"isVideoEntry":false,"isEntry":true,"isMt":false,"entryId":"5c40b5efe4b027c3bbbf19de","entryPermalink":"https://action.news/entry/michael-cohen-reader-polls-meaningless_n_5c40b5efe4b027c3bbbf19de","entryTagsList":"donald-trump,michael-cohen-lawyer,polling,reader-polls,@pollster,pollster analysis","sectionSlug":"politics","deptSlug":"politics-news","sectionRedirectUrl":null,"subcategories":"science","isWide":false,"headerOverride":null,"noVideoAds":false,"disableFloat":false,"isNative":false,"commercialVideo":{"provider":"custom","site_and_category":"us.politics","package":null},"isHighline":false,"vidibleConfigValues":{"cid":"60afc111dcf87c2cd2f5d8bf","overrides":{"front_page_top_videos":{"desktop":"60b64354b171b7444beaff4d","mobileweb":"60b64354b171b7444beaff4d"},"top_media":{"desktop":"60b8e6bdc5449357a7ada147","mobile":"60b8e701c5449357a7ada2ee","iphone":"60b8e643cdd90620331bb1f6","ipad":"60b8e643cdd90620331bb1f6","androidphone":"60b8e699c5449357a7ada04c","androidtablet":"60b8e699c5449357a7ada04c"},"anthology":{"desktop":"60b8e616cdd90620331bb0ba","mobile":"60b8e671c5449357a7ad9f66","iphone":"60b8e643cdd90620331bb1f6","ipad":"60b8e643cdd90620331bb1f6","androidphone":"60b8e699c5449357a7ada04c","androidtablet":"60b8e699c5449357a7ada04c"},"content":{"desktop":"60b8e616cdd90620331bb0ba","mobile":"60b8e671c5449357a7ad9f66","iphone":"60b8e643cdd90620331bb1f6","ipad":"60b8e643cdd90620331bb1f6","androidphone":"60b8e699c5449357a7ada04c","androidtablet":"60b8e699c5449357a7ada04c"}},"playerUpdates":{"5668ae6ee4b0b5e26955d6a6":"60d2472d9340d7032ad7e443","56aa41bae4b091744c0440d8":"60e869dc7c5f3b17b6741b81","5841b2b5cc52c716ec6e5a7f":"60b8e355cdd90620331ba185","58b5e2b8d85a10302feee895":"60b64316b171b7444beafdb2","58b74698f78ced31417819ae":"60b8e5bec5449357a7ad9b52","58b74ccecebcea57e2c3a3d1":"60b8e5eac5449357a7ad9ca5","58cff690d85a100b9992bc39":"60b8e616cdd90620331bb0ba","58cffb3fb6d9b972a49a3c9d":"60b8e643cdd90620331bb1f6","58cffdd74d96935d7d6ec180":"60b8e671c5449357a7ad9f66","58d03a84f78ced6518eb2fa7":"60b643c82e76be41f112735c","592edf20e0fa177b0c26f7fd":"60b8e699c5449357a7ada04c","5b35266b158f855373e28256":"60b64354b171b7444beaff4d","5c116f29f79c4171d82b7c2a":"60b64440b171b7444beb040b","5c1170fc600c9a697bf0c6b9":"60b646102e76be41f1127ffc","5c47791afa1b317df8ae0c4f":"60b8e6bdc5449357a7ada147","5c477987a6b48b35f164773d":"60b8e701c5449357a7ada2ee","5c4779ee943c3c2a64f28371":"60b8e747cdd90620331bb861","5c477a26fcd67b26879bc7c2":"60b8e788c5449357a7ada67b","5d8921a78c3ae845f366c9b6":"60ae7be5f3a7c13a30417ff9","58b98b00ba82aa39a6534321":"60d0de7c9340d7032ad1146c","58b9d14cb6d9b96c9ec32af3":"60d0dec19340d7032ad115a0","58cff8eccebcea42931e0436":"60d0e005b627221e9d819d44","592edf5de0fa177b0c26f95b":"60d0e38fb627221e9d81adcf","58cff72fd85a100b9992c112":"60d0e447b627221e9d81b0da","56b4d34fe4b022697697c400":"60d2472d9340d7032ad7e443","60b8e4c0c5449357a7ad957d":"60e869dc7c5f3b17b6741b81"}},"connatixConfigValues":{"defaultPlayer":"ff7fdddc-5441-4253-abc4-f12a33fad58b","clickToPlayPlayer":"d014396e-b366-4c17-aeac-3ce906fa3fd0","videoPagePlayer":"f010447b-d244-4111-a314-7b4542ae4145","verticalPlayer":"e58cb05a-0bc8-4210-9108-fea82726c065","stickyPlayerControl":"52ea1755-d601-4ad1-bccc-d8cce3f0e5da","stickyPlayerON":"8055e9e5-3bda-4933-8d45-9ad814fb6e22"},"topConnatixThumnbailSrc":"https://img.connatix.com/b02c2386-395c-414e-99e5-ce05986f56dd/1_th.jpg?crop=480:270,smart&width=480&height=270&format=jpeg&quality=60&fit=crop","customAmpComponents":[],"ampAssetsUrl":"https://amp.assets.huffpost.com","videoTraits":null,"positionInUnitCounts":{"buzz_head":{"count":0},"buzz_body":{"count":0},"buzz_bottom":{"count":0}},"positionInSubUnitCounts":{"article_body":{"count":18},"blog_summary":{"count":0},"before_you_go_content":{"count":0}},"connatixCountsHelper":{"count":1},"buzzfeedTracking":{"context_page_id":"5c40b5efe4b027c3bbbf19de","context_page_type":"buzz","destination":"huffpost","mode":"mobile","page_edition":"en-us"},"tags":[{"name":"Donald Trump","slug":"donald-trump","links":{"relativeLink":"topic/donald-trump","permalink":"https://action.news/topic/donald-trump","mobileWebLink":"https://action.news/topic/donald-trump"},"relegenceId":2789214,"department":{"name":"Latest News","slug":"politics-news"},"section":{"title":"Politics","slug":"politics"},"topic":{"title":"Donald Trump","slug":"donald-trump","overridesSectionLabel":false},"url":"https://action.news/news/topic/donald-trump"},{"name":"Michael Cohen lawyer","slug":"michael-cohen-lawyer","links":{"relativeLink":"topic/michael-cohen-lawyer","permalink":"https://action.news/topic/michael-cohen-lawyer","mobileWebLink":"https://action.news/topic/michael-cohen-lawyer"},"department":{"name":"Latest News","slug":"politics-news"},"section":{"title":"Politics","slug":"politics"},"topic":{"title":"Michael Cohen","slug":"michael-cohen","overridesSectionLabel":false},"url":"https://action.news/news/topic/michael-cohen"},{"name":"Polling","slug":"polling","links":{"relativeLink":"topic/polling","permalink":"https://action.news/topic/polling","mobileWebLink":"https://action.news/topic/polling"},"department":{"name":"Latest News","slug":"politics-news"},"section":{"title":"Politics","slug":"politics"},"topic":{"title":"Polls","slug":"polls","overridesSectionLabel":false},"url":"https://action.news/news/topic/polls"},{"name":"reader polls","slug":"reader-polls","links":{"relativeLink":"topic/reader-polls","permalink":"https://action.news/topic/reader-polls","mobileWebLink":"https://action.news/topic/reader-polls"},"url":"https://action.news/topic/reader-polls"}],"isLiveblogLive":null,"isLiveblog":false,"cetUnit":"buzz_body","bodyAds":["

\r\n\r\n","

\r\n\r\n"],"adCount":0},"isCollectionEmbed":false,"enhancements":{}}">

For instance: 38% of a nationally representative sample of Americans said that they'd had an argument about the 2016 election. When I included a reader poll with the same question in a story, 72% of HuffPost readers who answered said that they had. pic.twitter.com/7UiwWnbLXy
Ariel Edwards-Levy (@aedwardslevy) January 17, 2019

Perhaps most troublingly, because reader polls have no means of gatekeeping or measuring who responds, theyre intensely vulnerable to intentional manipulation by people with a vested interest in the outcome, whether thats someone like Cohen or online trolls . (This is also how you end up at risk of being told to name your research ship Boaty McBoatface or your soccer team Footy McFooty Face .)

The problem with reader polls isnt that theyre conducted online. In this day and age, there are plenty of scientific web-based pollsters with procedures in place to conduct representative surveys. The problem with reader polls is the lack of any provisions for sampling or weighting and the lack of even basic safeguards against being gamed by online mobs.

Your Support Has Never Been More Critical

Other news outlets have retreated behind paywalls. At HuffPost, we believe journalism should be free for everyone.

Would you help us provide essential information to our readers during this critical time? We can't do it without you.

Can't afford to contribute? Support HuffPost by creating a free account and log in while you read.

You've supported HuffPost before, and we'll be honest we could use your help again . We view our mission to provide free, fair news as critically important in this crucial moment, and we can't do it without you.

Whether you give once or many more times, we appreciate your contribution to keeping our journalism free for all.

Whether you give just one more time or sign up again to contribute regularly, we appreciate you playing a part in keeping our journalism free for all.

Support HuffPost

Lets be careful not to lump clickbait, vote-as-many-times-as-you-want polls w/ demographically representative online polls looking to solve a problem of declining phone response rates.

We, collectively, must come up w/ terminology to better explain & educate on the difference https://t.co/7Dyjxvfzt4

Nick Gourevitch (@nickgourevitch) January 17, 2019","type":"rich","meta":{"author":"Nick Gourevitch","author_url":"https://twitter.com/nickgourevitch","cache_age":86400,"description":"Lets be careful not to lump clickbait, vote-as-many-times-as-you-want polls w/ demographically representative online polls looking to solve a problem of declining phone response rates.We, collectively, must come up w/ terminology to better explain & educate on the difference https://t.co/7Dyjxvfzt4 Nick Gourevitch (@nickgourevitch) January 17, 2019\n\n","provider_name":"Twitter","title":"Nick Gourevitch on Twitter","type":"rich","url":"https://twitter.com/nickgourevitch/status/1085944407910727683","version":"1.0"},"flags":[],"fullBleed":false,"options":{"theme":"news","device":"mobile","editionInfo":{"id":"us","name":"U.S.","link":"https://action.news","locale":"en_US"},"originalEdition":"us","isMapi":false,"isAmp":false,"isVideoEntry":false,"isEntry":true,"isMt":false,"entryId":"5c40b5efe4b027c3bbbf19de","entryPermalink":"https://action.news/entry/michael-cohen-reader-polls-meaningless_n_5c40b5efe4b027c3bbbf19de","entryTagsList":"donald-trump,michael-cohen-lawyer,polling,reader-polls,@pollster,pollster analysis","sectionSlug":"politics","deptSlug":"politics-news","sectionRedirectUrl":null,"subcategories":"science","isWide":false,"headerOverride":null,"noVideoAds":false,"disableFloat":false,"isNative":false,"commercialVideo":{"provider":"custom","site_and_category":"us.politics","package":null},"isHighline":false,"vidibleConfigValues":{"cid":"60afc111dcf87c2cd2f5d8bf","overrides":{"front_page_top_videos":{"desktop":"60b64354b171b7444beaff4d","mobileweb":"60b64354b171b7444beaff4d"},"top_media":{"desktop":"60b8e6bdc5449357a7ada147","mobile":"60b8e701c5449357a7ada2ee","iphone":"60b8e643cdd90620331bb1f6","ipad":"60b8e643cdd90620331bb1f6","androidphone":"60b8e699c5449357a7ada04c","androidtablet":"60b8e699c5449357a7ada04c"},"anthology":{"desktop":"60b8e616cdd90620331bb0ba","mobile":"60b8e671c5449357a7ad9f66","iphone":"60b8e643cdd90620331bb1f6","ipad":"60b8e643cdd90620331bb1f6","androidphone":"60b8e699c5449357a7ada04c","androidtablet":"60b8e699c5449357a7ada04c"},"content":{"desktop":"60b8e616cdd90620331bb0ba","mobile":"60b8e671c5449357a7ad9f66","iphone":"60b8e643cdd90620331bb1f6","ipad":"60b8e643cdd90620331bb1f6","androidphone":"60b8e699c5449357a7ada04c","androidtablet":"60b8e699c5449357a7ada04c"}},"playerUpdates":{"5668ae6ee4b0b5e26955d6a6":"60d2472d9340d7032ad7e443","56aa41bae4b091744c0440d8":"60e869dc7c5f3b17b6741b81","5841b2b5cc52c716ec6e5a7f":"60b8e355cdd90620331ba185","58b5e2b8d85a10302feee895":"60b64316b171b7444beafdb2","58b74698f78ced31417819ae":"60b8e5bec5449357a7ad9b52","58b74ccecebcea57e2c3a3d1":"60b8e5eac5449357a7ad9ca5","58cff690d85a100b9992bc39":"60b8e616cdd90620331bb0ba","58cffb3fb6d9b972a49a3c9d":"60b8e643cdd90620331bb1f6","58cffdd74d96935d7d6ec180":"60b8e671c5449357a7ad9f66","58d03a84f78ced6518eb2fa7":"60b643c82e76be41f112735c","592edf20e0fa177b0c26f7fd":"60b8e699c5449357a7ada04c","5b35266b158f855373e28256":"60b64354b171b7444beaff4d","5c116f29f79c4171d82b7c2a":"60b64440b171b7444beb040b","5c1170fc600c9a697bf0c6b9":"60b646102e76be41f1127ffc","5c47791afa1b317df8ae0c4f":"60b8e6bdc5449357a7ada147","5c477987a6b48b35f164773d":"60b8e701c5449357a7ada2ee","5c4779ee943c3c2a64f28371":"60b8e747cdd90620331bb861","5c477a26fcd67b26879bc7c2":"60b8e788c5449357a7ada67b","5d8921a78c3ae845f366c9b6":"60ae7be5f3a7c13a30417ff9","58b98b00ba82aa39a6534321":"60d0de7c9340d7032ad1146c","58b9d14cb6d9b96c9ec32af3":"60d0dec19340d7032ad115a0","58cff8eccebcea42931e0436":"60d0e005b627221e9d819d44","592edf5de0fa177b0c26f95b":"60d0e38fb627221e9d81adcf","58cff72fd85a100b9992c112":"60d0e447b627221e9d81b0da","56b4d34fe4b022697697c400":"60d2472d9340d7032ad7e443","60b8e4c0c5449357a7ad957d":"60e869dc7c5f3b17b6741b81"}},"connatixConfigValues":{"defaultPlayer":"ff7fdddc-5441-4253-abc4-f12a33fad58b","clickToPlayPlayer":"d014396e-b366-4c17-aeac-3ce906fa3fd0","videoPagePlayer":"f010447b-d244-4111-a314-7b4542ae4145","verticalPlayer":"e58cb05a-0bc8-4210-9108-fea82726c065","stickyPlayerControl":"52ea1755-d601-4ad1-bccc-d8cce3f0e5da","stickyPlayerON":"8055e9e5-3bda-4933-8d45-9ad814fb6e22"},"topConnatixThumnbailSrc":"https://img.connatix.com/b02c2386-395c-414e-99e5-ce05986f56dd/1_th.jpg?crop=480:270,smart&width=480&height=270&format=jpeg&quality=60&fit=crop","customAmpComponents":[],"ampAssetsUrl":"https://amp.assets.huffpost.com","videoTraits":null,"positionInUnitCounts":{"buzz_head":{"count":0},"buzz_body":{"count":0},"buzz_bottom":{"count":0}},"positionInSubUnitCounts":{"article_body":{"count":21},"blog_summary":{"count":0},"before_you_go_content":{"count":0}},"connatixCountsHelper":{"count":1},"buzzfeedTracking":{"context_page_id":"5c40b5efe4b027c3bbbf19de","context_page_type":"buzz","destination":"huffpost","mode":"mobile","page_edition":"en-us"},"tags":[{"name":"Donald Trump","slug":"donald-trump","links":{"relativeLink":"topic/donald-trump","permalink":"https://action.news/topic/donald-trump","mobileWebLink":"https://action.news/topic/donald-trump"},"relegenceId":2789214,"department":{"name":"Latest News","slug":"politics-news"},"section":{"title":"Politics","slug":"politics"},"topic":{"title":"Donald Trump","slug":"donald-trump","overridesSectionLabel":false},"url":"https://action.news/news/topic/donald-trump"},{"name":"Michael Cohen lawyer","slug":"michael-cohen-lawyer","links":{"relativeLink":"topic/michael-cohen-lawyer","permalink":"https://action.news/topic/michael-cohen-lawyer","mobileWebLink":"https://action.news/topic/michael-cohen-lawyer"},"department":{"name":"Latest News","slug":"politics-news"},"section":{"title":"Politics","slug":"politics"},"topic":{"title":"Michael Cohen","slug":"michael-cohen","overridesSectionLabel":false},"url":"https://action.news/news/topic/michael-cohen"},{"name":"Polling","slug":"polling","links":{"relativeLink":"topic/polling","permalink":"https://action.news/topic/polling","mobileWebLink":"https://action.news/topic/polling"},"department":{"name":"Latest News","slug":"politics-news"},"section":{"title":"Politics","slug":"politics"},"topic":{"title":"Polls","slug":"polls","overridesSectionLabel":false},"url":"https://action.news/news/topic/polls"},{"name":"reader polls","slug":"reader-polls","links":{"relativeLink":"topic/reader-polls","permalink":"https://action.news/topic/reader-polls","mobileWebLink":"https://action.news/topic/reader-polls"},"url":"https://action.news/topic/reader-polls"}],"isLiveblogLive":null,"isLiveblog":false,"cetUnit":"buzz_body","bodyAds":["

\r\n\r\n","

\r\n\r\n"],"adCount":0},"isCollectionEmbed":false,"enhancements":{}}">

Lets be careful not to lump clickbait, vote-as-many-times-as-you-want polls w/ demographically representative online polls looking to solve a problem of declining phone response rates.

We, collectively, must come up w/ terminology to better explain & educate on the difference https://t.co/7Dyjxvfzt4
Nick Gourevitch (@nickgourevitch) January 17, 2019

As Democratic primary season heats up, its likely that similar types of reader polls will start to make a reappearance as well. But even when a campaign isnt making a systematic effort to rig them, they are still of extremely limited utility for understanding what Americans think.

To sum up: Any poll that does not exert some measure of control over who takes it and how many times they do so is not really a poll, and it shouldnt be treated like one.