A memorable though possibly apocryphal religious quip dates from the days when Norman Vincent Peale was a famed author and preacher. Democratic presidential candidate Adlai Stevenson supposedly said “I find the Apostle Paul appealing and the Apostle Peale appalling.”
What he found appalling was either Peale’s criticism of Stevenson’s divorce (in 1952), or of candidate Kennedy’s Catholicism (in 1960), or both.
So are polls appealing or appalling?
Eminent sociologist Robert Wuthnow of Princeton University lays out warnings that journalists should heed in “Inventing American Religion: Polls, Surveys, and the Tenuous Quest for a Nation’s Faith,” due for October 1 release from Oxford University Press and previewed in the current First Things magazine.
Polls were never mathematically precise to begin with and are becoming ever more unreliable, even as they take up infinite airtime and column inches during the run-up to the 2016 presidential campaign. Wuthnow reports this billion-dollar industry with some 1,200 companies conducted more than 37,000 polls during the 2012 U.S. campaign. Election predictions have sometimes proven well off the mark, as recently with Britain, Israel, and America’s 2014 midterms. Public surveys involve not just politics but closely watched trends on key matters like consumer confidence and unemployment rates.
A poll’s fine print lists a “margin of error,” often ignored in the media, that can skew results. However, Wuthnow says today’s critically important crisis in reliability is that huge numbers don’t answer the phone, causing terribly low “response rates.” Back in the 1980s, two-thirds to three-fourths of those sampled would usually participate. Today it’s typically 9 or 10 percent and rarely above 15 percent. Therefore, huge majorities of “the people who should have been included in a poll for it to be nationally representative are missing.”
Turning to American religion, his own field of expertise, he asks, “Is polling even a good way to think about it?” For one thing, political polling can be critiqued and corrected on the basis of actual ballot results, but “religion does not provide the same benchmarks for recalibration.”
Then this. After Gallup began asking religion questions in 1935, “theologians doubted that something as complex and deeply personal as religion could be tapped in polls,” but Wuthnow laments that such former skepticism is now largely absent. He sees good reason to question whether polling’s ’ simple “yes” or “no” approach works with religious opinions.
Experts doubt the strict veracity of peoples’ answers to Gallup on whether they attended worship in a particular week. Or take Pew Research’s much-publicized May survey report showing Americans are becoming less religious. It boasted of a huge sample of 35,071 adults on page 1, which helped provide breakdowns for many sectors. But page 97 listed response rates of only 11.1 percent with landlines and 10.2 percent with cellphones.
On Pew, Wuthnow cautions that non-affiliation is not the same thing as non-belief, since many “nones” believe in God and occasionally attend worship. More important, he says, responses of particular individuals “fluctuate wildly. Between a third and a half of poll-ees give different responses a year later, even to relatively straightforward questions about religious preferences and attendance.”
Wuthnow’s conclusion: “Polling about religion is troubling not because it is always wrong, but because it has become difficult for anyone to know when the results are correct and when they are not.”
Yet the Religion Guy would insist that polling can add some grounding to anecdotal and impressionistic reporting. Also note this analysis by data guru Nate Silver.