All the studies encountered a problem with American crime numbers: They’re based on reports from local law-enforcement agencies that may have widely different measurement and reporting standards. The Federal Bureau of Investigation strives to collate these in its Uniform Crime Reports (UCR), but as David Burnham, former spokesman for the commission that released the 52% figure and co-director of Transactional Records Access Clearinghouse, said, “There’s a warning up front that says, don’t use them” for rankings, because of variation between localities.

Arrest records are particularly incomplete, because most crime reporting focuses on crime reports, convictions and incarceration. “This is a problem in a country with 19,000-plus law-enforcement agencies that have the power to arrest and not all of them record those arrests in an easily searchable way,” said Joel Garner, chief of the law enforcement unit for the Department of Justice’s bureau of justice statistics.

Robert Friedmann, professor of criminal justice at Georgia State University, has attempted to compile incident reports for his Improving Crime Data project. “We have collected incident reports from dozens of agencies and literally no two look alike,” he said.

Then, of course, there are gaps that arise from unreported crimes. “UCR reporting is a voluntary activity and does not accurately reflect crime rates since it can only list crimes reported to law enforcement agencies,” said Cornell University statistician Martin T. Wells.

“The quality of the data across agencies varies dramatically,” said Alan Lizotte, interim dean of the school of criminal justice at the University at Albany. “There’s no easy, straightforward way to do this, at all,” he said of estimating the proportion of Americans who have been arrested.

Still, the 52% figure doesn’t appear too far off base, which may surprise people who don’t know anyone who’s been arrested (or don’t know they know anyone who’s been arrested). Part of the reason is that it’s a lifetime risk, and a lifetime is generally a very long time. Mark Rank co-wrote a recent paper finding that half of American children will be on food stamps between the ages of one and 20. “Some colleagues were quite surprised,” Rank told me. “However, the data presented in this paper are quite consistent with other work we’ve done where we’ve looked at the long-term risk for adults experiencing poverty or having to use a social safety net program. One of the reasons why these numbers are so high is that we are looking across long stretches of time (which is quite unusual in social-science research). As a result, individuals and families are much more likely to be at risk of particular economic hardships and risks occurring.” The same could be said for arrest rates.

Further reading: The arrest stat was mentioned in a Wall Street Journal article last week. I’ve written previously about murder stats, recidivism and local crime stats. And fellow numbers guy Charles Forelle assessed whether crime rises during recessions.

The digitization of data and increases in computing power are fueling this work. For one paper, Georgetown University political scientist Michael A. Bailey dug through online archives of presidential speeches and statements to see how they expressed their opinions about Supreme Court cases. And David Blei culls extensive archives of scientific journals to find connections across decades, honing techniques that could also be useful in categorizing online content for search engines.

A 1999 study comparing sportsmen of prior generations with those today conducted a decade ago was based on only 25,000 simulations — “a shamefully small number today,” Shane Reese, a statistician at Brigham Young University, said, because of increases in computing power. Nonetheless, the study drew provocative conclusions about changes in baseball, golf and hockey — though it and other team sports with multiple players involved on every play resisted analysis. “Hockey is difficult because the strength of your linemates is so important,” Scott Berry, Reese’s co-author, said. “We never did football, but to model a running back would be extremely difficult because he is such a function of his offensive lineman.”

What do you think? Do you find these inter-era comparisons persuasive? What other applications might these statistical methods have? Please let me know in the comments.

Steven Piantadosi, a graduate student in cognitive sciences at the Massachusetts Institute of Technology, arrived at a probability of slightly less than one in one trillion, as he noted on his blog. A 1960s-era compilation of American English texts known as the Brown corpus, analyzed by Linguistic Data Consortium researcher David Graff, yielded almost exactly the same calculation. Dave Thomas, a staff scientist at New Mexico Tech, used works from the Project Gutenberg database of copyright-free electronic books, and came up with one in 500 billion. Mathematician Edward Lewand of Goucher College told NPR he came up with about one in 180 billion. “I think it is very unlikely that this acrostic was just a coincidence,” Lewand said.

Schwarzenegger isn’t Jane Austen nor William Shakespeare, two of the sources for the letter-frequency numbers behind Thomas’s calculation. It’s possible that he’s more prone to use words that begin with the letters that made up the coded message. For instance, “K” is the least likely letter in the message to start a word. But the “K” in the veto message comes from a statement that the state legislature “kicks the can down the alley.” Some blogs have suggested that this phrase was included merely to make the encoded veto message work. But the phrase, meaning delaying tough decisions, improbably has become a standard one in California politics in the last year.

Also, as Thomas and others note, these probabilities are for the message to appear in any given seven lines. Schwarzenegger has vetoed about 1,700 bills, and often attaches messages to his rejections. However, these brief missives don’t add up to enough text to explain the improbable profanity. Using the text of the letters from the California government’s Web site, Graff could find no message formed by the beginning of each line that was any edgier than “espy tit,” in a veto of a bill that would have designated 211 as a social-services phone number.

If one broadens the criteria for such codes, they become easier to find. In my brief email to Brendan McKay, a computer scientist at Australian National University in Canberra, about the veto message, he found the word “measures,” by counting backwards four letters at a time from a certain spot in the email. He also found “events,” by counting backwards five letters at a time from elsewhere in the message. “The chance of this happening accidentally (even allowing for all possible starting points and all possible skip amounts forwards or backwards) is about one in a billion,” McKay said. “But I don’t believe you did it on purpose.” That’s partly because he didn’t look specifically for those words; “I fed a whole dictionary [into his computer program] and then spun a story around two of the many words that appeared,” McKay said.

Reflecting on the difficulties of calculating the probability that the veto message arose by chance, Williams College mathematician Edward Burger said, “That’s the annoying thing about coincidences. We should definitely expect them. But the annoying thing is we just don’t know the actual probability that one of these things will happen.”

Further reading: Read more about a one in 10 billion trillion long shot that cost an English journalist a job, a one in 300,000 trillion trillion long shot that riled some Pakistanis and other Numbers Guy writings on coincidences and unlikely events.

Thanks to Jason Fry for the idea.

Fear of the number 13 doesn’t extend to all parts of life. Representatives for several major U.S. airlines said all their planes contain a row 13. American Airlines customers are just as likely to choose that row as the 12th or the 14th, according to a spokeswoman. Southwest has first-come, first-seated policies, so it’s hard to say if customers prefer to sit elsewhere. However, spokeswoman Olga Romero said, “We don’t see any lower booking on Friday the 13th. Apparently our customers are not superstitious.”

Numerologist Beverly Kay agrees that fear of 13 is superstitious, but she does share concerns about the year 2012 and says 7 is “sacred and holy.” She says she loves mathematics; she just takes it a step further, reading more deeply into numbers.

Kay was very pleasant and generous over the phone, even offering to provide readings into my birth date and name. (”Your humor is so dry you should pour some water on it” and “you love your freedom a lot” were among her observations.) But I tend to agree with Colin C. Adams, professor of mathematics at Williams College, who said of numerology, “It does cause people to make choices, sometimes important choices, based not on evidence and logical analysis of the alternatives but rather on nonsensical superstitions.”

Mathematicians also may favor certain numbers, but they couch it as a fascination with their properties rather than an acknowledgment of some higher, mystical powers. George Bergman, who recently retired as a mathematics professor at UC Berkeley, recalls factoring the number of the New York City subway cars that took him to Stuyvesant High School. He also passes on the oft-told mathematical joke — not an oxymoron — that there are no uninteresting positive whole numbers, “because if there were, there would be a smallest one, and the property of being the smallest uninteresting number would itself make it interesting.”

Today, when Bergman parks at a commuter rail station, he finds it “amusing” to get the spot numbered 233 (a Fibonacci number), 235 or 238 (atomic weights of uranium isotopes), 245 (the course number of a course he’s taught) or 256 (two to the eighth power). “But noticing these things is mostly a way of amusing myself; not anything really significant,” Bergman said.

Bergman also appreciates i, which is the “imaginary” number whose square is -1. So does Phil Hanser, a statistician with the Brattle Group, who recalls encountering i at age 10 “and being completely overwhelmed by it.”

Rich Beveridge, a math instructor at Clatsop Community College, has explored the cultural significance of 13. “If we think that the world is one way and mathematics and numbers give us different information, we can explore and analyze and try to understand the information or we can be afraid of it,” Beveridge said. “If we don’t have time to explore, analyze and understand we may end up with fear by default.”

Further reading: “Dr. 13” explores the history of fear of the number. New York buildings have their own floor-numbering problems. Some dates on the calendar gain attention for their apparent numerical properties.

I wrote in 2007 that U.S. advocates’ estimates of 10,000 sex slaves brought into the country annually could be overblown; I cited a Washinton Post report. This week, the Guardian exhaustively dissected the source of estimates that thousands of women were trafficked into the U.K. each year to be foreced into the sex trade.

As often happens, the estimate started with a much smaller number that was then speculatively increased to cover all the cases that police and the press haven’t uncovered. Then follow-up reports cited the initial one, scrapping the details about the speculation and fudging definitions. For example, what about those who were smuggled into a country, but not for prostitution?

“The cacophony of voices has created the illusion of confirmation,” The Guardian’s Nick Davies writes. And with sensitive issues such as this one, it can be hard to find voices on the other side; after all, no one who wants accurate numbers wants to be seen as supporting sex trafficking.

I’m not the first to criticize the ranking. The editor-in-chief of the report, who said he wasn’t fully aware of the ranking methodology until after it was published, denounced the rankings in a journal article in 2003, drawing a rebuttal from the creators of the rankings. Glen Whitman, an associate professor of economics at California State University at Northridge, also criticized the rankings in a paper for the conservative Cato Institute.

I asked Whitman how he would have constructed a ranking. He, like many researchers, said he wouldn’t have. “I probably would not have tried, because I think the project is inherently problematic,” Whitman said. “The problem is that the quality of health care has lots of different dimensions: success in treating condition A, success in treating condition B, etc.; speed; convenience; cost; privacy; and so on. People with different values will place different weights on these factors. So any attempt to create a universal index of quality necessarily relies on value-laden assumptions.”

Another criticism noted that the rankings didn’t control for many variables it could have, such as geography (tropical climate, and distance from coast, can affect health). Controlling for such variables materially affected many country’s rankings — but not the U.S.’s. And that points out an important point that many of the WHO ranking’s critics acknowledge: Though the rankings may have been flawed, a ranking of 37th for the U.S. seems reasonable, and the U.S. may have slipped since 2000. For instance, data from the United Nations Population Division show the U.S. falling further behind its peers in life expectancy and preventing mortality of children under age five.

“When it’s good, it’s very, very good, and when it’s bad, it’s horrid,” Mark Pearson, head of health for the Organisation for Economic Co-operation and Development, the 30-member, Paris-based organization of major economies, said of the U.S. health-care system.

Some researchers have attempted to get a firmer handle on the U.S.’s proper health-care ranking by attempting to remove the effects of factors outside the health-care system’s control, such as violent crime. One such effort, which I recounted here two years ago, found that removing such factors greatly improved the U.S.’s standing in life expectancy. However, critics assailed the report for its indirect approach to the problem. One of the authors of the report, John E. Schneider, general manager of U.S. Health Economics for international health-care consultancy Oxford Outcomes, said that its intent was to show that small differences in life expectancy at birth, such as two years, “are too small to be meaningful due to differences in measurement and populations across countries.”

Researchers at the London School of Hygiene and Tropical Medicine have been tracking trends in deaths from causes deemed treatable or preventable by the health-care system, such as bacterial infections, treatable cancers and diabetes. The U.S.’s performance has slipped, falling to last among 19 wealthy nations. “Diabetes is highly correlated with obesity, and it’s not clear how ‘amenable’ obesity is to the health system,” Schneider said, in response to that study. “Can the health system make you get of your [posterior] and turn off the TV, or make you order one Big Mac instead of two? Probably not. … It’s the right general direction to go, but our old point is still relevant because this article demonstrates how methodologically challenging these kinds of analyses can be.”

The debate depends partly on how widely the health-care system is defined. “The health-care system ought to take responsibility,” for disease rates, says Ellen Nolte, director of health and healthcare for RAND Europe and co-author of the mortality-rate study. “It really is a case of integrating prevention and treatment rather than leaving the former to other sectors and only the latter to health care.”

An alternate analysis, focusing on prostate cancer and breast cancer, found signs of improvement in treatment and death prevention for those diseases in the U.S. and concluded that the country’s poor life-expectancy ranking “is not likely to be a result of a poorly functioning health-care system.” Samuel H. Preston, a sociologist at the University of Pennsylvania, pointed out that Nolte’s study focused on mortality by age 75, whereas the U.S. is strongest at later ages. Nolte counters that the Preston study examines only a select set of diseases, not the broader spectrum of her study.

Another problem with comparing countries by disease outcomes is that definitions and measurement “can vary within a country and across countries, even when the true prevalence is identical,” according to Alan Garber, an economist and professor of medicine at Stanford University. “There have been many efforts to standardize disease reporting across nations but it’s a tough challenge.”

Still other studies, though, also rank the U.S. low compared to its peers. “It still looks as if the U.S. is lagging behind,” said Jon Skinner, an economist at Dartmouth.

Further reading: The New York Times recently analyzed one of the newer comparative health-care studies. Here’s the WHO’s press release about its initial report, and a recent Politifact post evaluating the U.S.’s supposed No. 37 ranking.

Such a quirky convergence of two unrelated events is made possible by a new Web site, the Book of Odds, that has compiled thousands of probabilities and grouped them by topic but also by the magnitude of the probability. The idea, according to founder and president Amram Shapiro, is to create “a 19th-century style reference work that met the standards of scholarship” for such works, but which also taps into all the possibilities of an interactive medium. Users, according to Shapiro, will help conquer the fear of uncertainty, which he said “leads to a great deal of mischief.”

“We’re trying to make a Web site that will accomplish all of these things by being extremely interesting, and extremely easy to use,” Shapiro said.

The Book of Odds, which launched Wednesday, hasn’t quite fulfilled all its goals. It’s still a work in progress. For instance, based on its odds statements it would have been better labeled the Book of Chance or the Book of Probability. Instead, the site says things such as, “The odds an accidental death from fire will occur to a man 55 or older are 1 in 4.06.” The correct phrasing would be “The chance an accidental death from fire will occur to a man 55 or older is 1 in 4.06,” or “The odds against an accidental death from fire occurring to a man 55 or older are 3.06 to 1.” That’s because the odds against something is the ratio of the chance that thing doesn’t happen to the chance it does. A 3 to 1 underdog in a playoff series has a one in four chance of winning. (People familiar with sports-gambling lines will understand the difference.) It’s also somewhat confusing; what it means to say is that 1 in 4.06 such deaths occur to a man 55 or older, not than 1 in 4.06 such people will die accidentally in a fire.

Some of the listed probabilities also are based on rather old studies, some dating back to the 1970s, and are listed with false precision. (Is that probability really 1 in 4.06, not 1 in 4.1 or 1 in 4?) And the probabilities usually are averages based on past events, rather than predictions of the future. Surely among men 55 or older, probabilities of dying in a fire differ based on what kind of building they live in, whether they live alone and how far they live from the nearest fire department.

Similarly, on the home page, a newsy odds statement puts the probability of a Summer Olympics being held in South America at 1 in 28, based on the history of the Olympics. But the process of choosing host cities more than a century ago was very different than it is today. And that’s an example of where past events don’t guarantee future performance; in fact, quite the opposite. Cities sometimes are granted host status in part because their regions have been short-changed in the past.

Shapiro acknowledges that the site isn’t necessarily up to the standards of the most rigorous scientific journal, for instance in its use of four significant figures for many probabilities. “For us to fulfill our mission, which of course has to do with people being able to absorb what we are doing, we have to make sure we are entertaining as well as thorough and accurate,” he said. “We have to do some balancing. We will certainly not do anything we know to be wrong. But we have to have playfulness to draw in people and make them not feel they are visiting an actuary’s office or a funeral parlor.”

To the site’s credit, it has already fixed a problem in the beta version I reviewed last week. Each probability statement comes with an animation of a red circle inside a larger black circle; how much larger indicates how unlikely that event is. Initially a 1 in 5 event would be illustrated by a red circle with a radius one fifth that of the black circle. But that meant the red circle had an area one 25th that of the bigger circle, making the event seem far less likely than it is. Now the circles are scaled by area instead of radius.

Such fixes are the work of about two dozen staffers, including several with economics training, in what Shapiro calls a “multimillion dollar” venture. “What we’ve done would be meaningless if we didn’t update it,” he said. And the goal is to draw people to the free site and make money from such sources as advertising and selling T-shirts with any of the odds statements. “Make no mistake, we intend to make money,” he said. The odds of that aren’t listed on the site.

Keith Winstein contributed to this blog post.

Skeptics of such studies note that they often fail to demonstrate that spanking caused the measured effects; perhaps cognitive problems are tied to behavioral problems, which themselves cause spanking. Berlin’s study addressed this by examining whether cognitive development at age two was tied to spanking at age three; it wasn’t.

Another common weakness of spanking studies, according to Robert Larzelere, a research methodologist at Oklahoma State University and former colleague of Straus’s as a post-doctoral fellow, is that they don’t compare spanking to other forms of punishment. Berlin partially addressed that, by comparing spanking to verbal punishment, which didn’t have the same links to poor cognitive development. Larzelere said his own studies, however, show that all corrective actions — physical and otherwise — have essentially the same effects.

“Spanking researchers have been waiting a long time for a study as high-quality as this one,” Marjorie Gunnoe, a professor of psychology at Calvin College in Grand Rapids, Mich., said of the Berlin study.

However, the controversy over spanking isn’t likely to be settled over these studies. It is a subject of dispute between the American Academy of Pediatrics and a smaller, splinter group called the American College of Pediatricians that supports spanking in certain instances. “No one likes to spank a child, but for some children it is the only method that will work to gain control of the child’s behavior and ultimately the will,” said Dr. Den Trumbull, vice president of the group, and a pediatrician in Montgomery, Ala.

Adam Zolotor, an assistant professor of family medicine at the University of North Carolina, Chapel Hill, said that to control for sufficient variables such as genetics would require a complex research effort. “The best way to do this would be to get enough twins that were separated at birth,” he said. “Let’s say you need at least 500 twins separated at birth, and probably more. I don’t know if you could find that many.”

Vivian Friedman, a professor and clinical psychologist at the University of Alabama at Birmingham school of medicine also doubts that a research study could ever settle the spanking debate. “The best way to test their hypothesis that spanking lowers IQ would be to … to measure IQ in a child repeatedly over time (maybe every year or two years) while measuring the numbers of spankings administered,” Friedman said. “Even so, it would be hard to rule out other physical and emotional environmental factors for causality.”

Further reading: Some media reports injected appropriate skepticism into their accounts of the new spanking studies. Barbara Lerner defended spanking in a 1998 commentary in the Wall Street Journal. Earlier this year in the Atlantic, Hanna Rosin examined another disputed area of parenting research: breast-feeding, and its benefits.

Carl Bialik for The Wall Street Journal

Joey Chestnut may not have noticed, but Nathan’s Famous at Coney Island now reports calorie counts.

Pierre Chandon, a marketing professor at Insead, the international business school in France and Singapore, said that the study adds to prior results that are discouraging for menu-labeling advocates. “Although its results are disappointing to those (including me) who are in favor of calorie information in restaurants, they are not that surprising,” Chandon said. “We know that informing people about nutrition value and changing their food preferences are two very different things. In fact, some studies have even shown that calorie disclosure can backfire and increase unhealthy choices among people who, for example, think that unhealthy (high-calorie, high-fat) food is tastier.”

The study’s authors suggest that labeling might work if combined with other efforts to prod diners toward healthier choices. “My sense is that to really influence obesity, we will probably need a combination of interventions,” Brian Elbel, assistant professor of medicine and health policy at New York University and lead author of the study, told me. “These will have to deal with the availability of foods in a community (both health and unhealthy), the price of foods (health and unhealthy), the marketing of foods and what we provide in schools for kids.”

One measure that New York has taken since the study is an educational campaign, which might combine with calorie counts to influence choices. “I do think it would have been helpful to look at consumers who both saw the labels and were also aware of the education program, but the data were collected prior to the NYC education component,” said Scot Burton, professor of marketing at the University of Arkansas’s business school.

Kelly Brownell, director of the Rudd Center for Food Policy and Obesity at Yale University and an advocate of menu labeling, said he expected the educational program, and a proposed sugar-sweetened-beverage tax that he also supports, could boost the results. He also took it as a positive sign that 28% of New York low-income diners said the menu labels influenced their purchases. “That’s a big number,” Brownell said. (Menu-labeling advocate Marion Nestle wrote that the study doesn’t mean menu labeling won’t have positive effects.)

“At least the public has information and that’s the government’s job — to make sure that the public has information,” Mayor Michael Bloomberg told reporters, according to Reuters.

To J. Justin Wilson, a senior analyst at the Center for Consumer Freedom, the study validates his group’s opposition to menu labels. “There wasn’t any proof it would be effective,” he said — and he says the same applies to the so-called soda tax. (Brownell disagrees, citing evidence from tobacco taxes.)

Two mitigating factors make the study not quite as damning for menu labels as it seems on first glance. First, it’s possible that the effect is smaller among people with lower incomes, because healthier options — such as salads or chicken breasts — often are more expensive. And second, though New York diners’ chain-restaurant meals averaged more calories after the law took effect, Elbel points out that the difference wasn’t statistically significant.

Mathematicians said the Silver analysis — finding that certain digits showed up far more often than others in Strategic Vision polls — was troubling but want to see more evidence. Jordan Ellenberg, a University of Wisconsin, Madison, mathematician, blogged that the case isn’t as persuasive as investigations into possible fraud in the Iranian election. “It’s not so substantial that I would have gone public with it, if it were me,” Ellenberg said, but he does think it merits further investigation.

“To strengthen the argument that Strategic Vision’s (or any other polling group’s) numbers seem unusual, the next step would be to assess the observed variation across a number of similar polling organizations and see where various groups fall,” said Lance Waller, a biostatistician at Emory University.

Strategic Vision is a Republican firm, but its critics — Aapor and the National Council on Public Polls, which also issued a statement calling for disclosure — aren’t partisan, and the firm’s polls haven’t been notably favorable to Republican candidates.

Two think tanks that are clients of Strategic Vision also are seeking more details on the firm’s methods in light of Silver’s analysis. The Goldwater Institute, which calls itself a free-market think tank, and the Oklahoma Council of Public Affairs hired Strategic Vision to test high-school students’ civic knowledge in Arizona and Oklahoma, respectively.

After Silver questioned the Oklahoma results as being too bleak, both think tanks sought verification from Strategic Vision. “Although I find it very unlikely that Strategic Vision manufactured this data, I have asked for receipts from the marketing firm from which they purchased the contact data just to make certain,” Matthew Ladner, vice president of research for Goldwater Institute, said.

Brandon Dutcher, vice president for policy for the Oklahoma group, isn’t making up his mind just yet. “I have requested voluminous survey data from them, as well as answers to some methodological questions — all of which I expect they can and will provide so that they can go about defending their firm and I can go about defending this survey,” Dutcher said. “If not, however, then of course I would want my money back and wouldn’t hire them again.”

Requests for further information from Strategic Vision in the past have met with positive results. Several polling aggregators, including Tom Silver of the Polling Report (no relation to Nate Silver), said that the firm has answered their questions.

But Nate Silver remains skeptical. “I think I’ve moved the ball a lot and built a pretty persuasive case,” he said, pointing to calculations posted on his site that suggest the probability of such a digit distribution arising by chance alone is about one in 5,000. “In an intellectual sense, the burden of proof is on their side.” Silver said he’d like to see the firm disclose the name of the call center it has hired and other details about its polls.

Other pollsters argued for more disclosure. “As an AAPOR member, I’ve pledged to disclose basic methodology in any poll I release, and I think the public has an absolute right to know the basics,” said Clay Richards, retired assistant director of the Quinnipiac University Polling Institute. Added Aapor president Peter Miller, “We are asking for information that any public-opinion researcher should provide so that study findings can be interpreted and replicated.”

Pollsters are unsure of how to identify bad polls. “I’m not sure there’s a gatekeeper mechanism that works,” said Mark Blumenthal, a former Democratic pollster who now covers the industry at Pollster.com. “I propose we do a better job of scoring disclosure to create incentives for pollsters to disclose more.” His colleague at Pollster.com, Charles Franklin of the University of Wisconsin, Madison, said, “If [Bernie] Madoff could fool the SEC not to mention investors, unscrupulous pollsters could also hide details or even fake some things in a far less scrutinized and regulated industry.”

Some pollsters do take steps to check their surveys. The Centers for Disease Control and Prevention’s National Center for Health Statistics monitors its phone surveys, and recontacts 5% to 10% of respondents to in-person surveys, according to a spokesman.

Aapor plans to spotlight pollsters that use sound methodology and disclose their methods. The thinking, said Miller, is, “how can we make this a positive for organizations, as opposed to a club with which to beat organizations when they fail to be transparent.”

Tom Jensen, communications director of the firm Public Policy Polling, said the news media bears some responsibility for reporting questionable polls. “I don’t think polls should be reported if basic information to see if the numbers are valid is not given, such as party breakdowns,” Jensen said.

Further reading: Blumenthal called for more transparency from Strategic Vision, and explained why Aapor doesn’t provide more transparency of its own. Among active pollsters in Nate Silver’s ratings last year, Strategic Vision was near the middle of the pack when it came to accuracy at calling races. Aapor has information on practices to prevent interviewer falsification.