What a difference a month makes. In early August, the Religion News Service ran a long list of reasons why opinion polls are often unreliable. This week, the Pittsburgh Post-Gazette turned to several surveys on Catholic attitudes toward the faith -- and toward Pope Francis, scheduled to visit the United States in a couple of weeks.
The funny thing is, both articles drew on some of the same bean-counting organizations -- and in one case, the same expert.
Last one first. The Pittsburgh paper aims at showing the challenges awaiting Francis in his first visit to the United States.
"There may be more American Catholics than ever, but they’re doing fewer Catholic things," says savvy religion writer Peter Smith in summing up the paradox. To make his point, he gathers from at least three of the usual sources: Pew Research Center, the Public Religion Research Institute and the Center for Applied Research in the Apostolate at Georgetown University.
The numbers tell him that "life-cycle events" are down: infant baptisms, first communions, church marriages and elementary school enrollment. Most Catholics affirm "basic beliefs" about Jesus and Mary, but they don’t pray the rosary, pray as families or do adoration of the Eucharist.
The Post-Gazette also cites a Pew survey that found "fewer than half of Catholics think it’s a sin to have gay sex, use artificial birth control, live with a partner outside of marriage or remarry after a divorce without an annulment. They’re evenly split on whether the church should recognize gay marriages."
But the numbers don’t overpower the human side of the piece. There’s also a nice mix of quotes and anecdotes, like this one:
And even though the Pittsburgh region lost population following the steel bust of the 1980s, the Catholic numbers declined at an even higher rate. Catholics were 45 percent of the 1980 population of the seven-county metropolitan area, which straddles parts of the dioceses of Pittsburgh and Greensburg, but only 33 percent in 2010, according to the Association of Statisticians of Major Religious Bodies.
Pittsburgh Bishop David Zubik said such numbers underscore that when Francis calls on the church to evangelize, its primary audience consists of Catholics themselves who may have only a limited understanding of the faith. He said a spiritual individualism or “me-and-God mentality” is prevalent today, making it harder for people to understand the importance of communal rites such as baptism and communion.
“We’re a church family,” he said. “It’s in the context of the sacraments that we celebrate as a community."
Things aren’t the same everywhere, the story recognizes. It points out that dioceses with large Hispanic influxes are growing -- and quotes the head of one of them, Archbishop Jose Gomez of Los Angeles. The story also recognizes the large youth rallies sponsored by the Franciscan University of Steubenville, Ohio -- then says that nationally, young Catholics are "far more indifferent to the faith."
Well, darn, all that sounds pretty bulletproof. At least until you go back and read the RNS report, which stares with narrowed eyes at the polls -- and offers several reasons for the skepticism.
Grit your teeth and get past the dig at Fox News in the lede. When Fox said it used surveys to decide on Republican presidential candidates for its TV debate Thursday, "pundits howled and poll jargon flooded the media," RNS says. Then the story settles into its theme: that some pollsters themselves now question the reliability of polling.
Here's a "yeowtch" quote by Robert Wuthnow of Princeton:
He charges that the image of U.S. religion created by pollsters is too often inaccurate, shallow and misleading. A steady parade of surveys on faith and values misses the depth and nuance of American religious life while making puffed-up claims for credibility even as the rate of response falls to record low levels.
Granted, Wuthnow is hawking his new book, Inventing American Religion: Polls, Surveys, and the Tenuous Quest for a Nation’s Faith. But RNS gets a few others to agree, to varying degrees.
"Blurry portraits," says Mark Gray of CARA, although he says blurry is better than blind.
Robert Jones of the Public Religion Research Institute says that the population's growing diversity has made polling less accurate than "when the white nuclear family was the cultural framework."
Pew's Alan Cooperman sounds a bit defensive in saying that pollsters “don’t pretend we are the only or best way.”
But RNS offers concrete reasons for skepticism about polls. One is the response rate -- the percentage of people who agree to take part -- now 10 percent or less. Another is the easy access to social media, when anyone can tweet a survey and then show how their friends agree with them.
The story adds some nuance -- saying, for one, that margin of error doesn’t always equal accuracy. It also acknowledges that some pollsters are better than others. The Barna Group, for instance, "is unique among mainstream pollsters for using a complex set of theological measures to categorize respondents rather than letting people loosely label themselves as 'evangelical' or 'Christian' or 'born-again.' "
Where to go from here? Wuthnow offers RNS a couple of recommendations. One would be "fewer and better surveys." Another:
Wuthnow would like to see pollsters turn to more open-ended questions and focus groups where people can push back against the pollster’s language and define their own terms for their religious life. Indeed, he thinks that exposure to decades of pollsters’ questions and media-touted findings have shaped the vocabulary of how people respond.
His prime example: Asking people if they are evangelical or born-again as a single category of identification. This has trained the public to think, incorrectly, that these very different theological terms are one and the same, said Wuthnow.
But the article doesn’t predict much change -- not as long as pundits and politicians load up on numbers like ammunition. And it doesn’t help us down here at the reader level, when pollsters disagree with one another even on the accuracy of polling.
Best we can do is what RNS and the Post-Gazette did: look at more than polls and get human feedback. Then maybe we can decide for ourselves who knows what they’re talking about, and who doesn’t know beans.