Gallup Poll Reveals Changes To Its Survey Methodology

WASHINGTON -- In a busy week in which Gallup began reporting polling results among voters deemed most likely to vote , Gallup Editor-in-Chief Frank Newport also announced a series of updates and modifications to the firm's methodology, including a significant boost in the number of survey calls to cell phones.

In a blog post published Wednesday, Newport said that the changes reflect a process of "continually tweaking, modifying, and improving our methodology" in order to "stay consistent with changes in the communication behavior and habits of those we are interviewing." These changes include:

An increase in the proportion of cell phones called in its nightly tracking survey, from 40 to 50 percent of the completed calls (which Gallup then weights, as before, to reflect the actual percentages of Americans who use only cell phones, only landlines or some combination of the two).

A separate daily tracking survey for the final month of the campaign "focused just on the election and other political measures," which Newport explained would allow for a shorter questionnaire that will get a higher response rate. Typically, the Gallup Daily survey covers a variety of topics, including politics, economic confidence and personal well being .

Slight modifications to the questions used to select likely voters in order to account for "the increasing prevalence of early voting."

"Slight changes" to the procedures used to adjust or "weight" the adult polling samples to match U.S. Census targets for various demographic characteristics, including a "new weight for the density of the population area in which the respondent lives."

A 2005 analysis by the Pew Research Center had concluded that even after standard demographic weighting, its own adult samples were under-representing densely populated urban counties. If left uncorrected, that skew would have resulted in an understatement of the vote preference for Democratic candidates, even after narrowing the samples to just likely voters.

Newport also wrote that the changes would allow the weights that Gallup applies to cell-phone users to "be smaller, given that the actual percentage of cell phones and cell-phone-only respondents in the sample is higher."

A Huffington Post report this past June found that Gallup's weighted adult samples were missing their targets for racial composition, falling roughly 2 percentage points short of the Census estimate for the percentages of black and Hispanic respondents among all adults.

At the time, Gallup explained that the shortfall was the result of a statistical trade-off: What would otherwise be extremely large weights were being "trimmed" to avoid distorted results, but this trimming occurred disproportionately among minority respondents. By reducing the size of the initial weights, the changes announced by Newport Wednesday should enable Gallup to hit its desired racial targets more accurately.

Another minor alteration appears to address a related issue raised in the June HuffPost report. As recently as late September , Gallup had weighted to match Census figures for "telephone households," excluding from the weighting targets the tiny sliver of disproportionately non-white Americans not reachable by either cell phone or landline. As of Tuesday , Gallup had dropped the phrase "living in U.S. telephone households" from the description of its demographic weighting targets.

"As has always been the case," Newport stressed in his announcement, "we do not attempt to weight the composition of the likely voter sample in any way -- such as by political party or race or age -- to approximate some guess of what we or others think it should look like demographically on Election Day" (emphasis added). Gallup weights the larger adult sample to match Census estimates, but take no further steps to adjust the demographics of likely voters.

That said, the recent methodological changes appear to have increased the percentage of non-white respondents in Gallup's weighted samples of adults, as noted by Emory University political scientist Alan Abramowitz in a HuffPost blog item posted earlier Wednesday.

Gallup does not routinely publish statistics on the demographic composition of its samples. HuffPost asked Gallup to provide the actual weighted racial composition for the last five weeks, but those data were not available as of this writing.

In his announcement, Newport said that while the changes announced Wednesday "increase the representation of our overall samples," Gallup is still evaluating whether these modifications "may have some impact on political or other measures included in the surveys." He singled out the "average values of our presidential job approval rating" as potentially affected.

In the first week of October, as Mitt Romney erased President Barack Obama's lead in the race for the White House , Gallup's job approval rating for Obama was 52 percent. During September, Obama's average weekly approval ratings had been slightly lower: 50 percent for the first two weeks of September, 49 percent from Sept. 17 to 23 and 48 percent from Sept. 24 to 30.

Gallup has a long history as the most trusted survey brand in the world , a name virtually synonymous with public opinion polling. The changes announced Wednesday show a willingness to adapt many longstanding procedures to polling's new reality .