By Sherril Guthrie. Re: What’s Up With That: 27 + 83 = Huh?
Editor’s Note: The following was submitted to the Abbotsford News and a copy was provided, for publication, to Today Media as well, as it references a post on Abbotsford Today.
att: Andrew Holota, Abbotsford News
Andrew Franklin, Abbotsford News
Although I’ve “weighed in” on a few of the Abbotsford News polls in the past, (not the one currently in question) – as a researcher I have some serious concerns about how your data is gathered and tabulated. More importantly, I’m concerned about how your poll results are publicly reported.
Clearly, I’m not the only one with concerns. So here are a few recommendations:
Research Recommendations to The Abbotsford News
First, to ensure proper perspective and interpretation of your poll results, you should ALWAYS state the sample size or number of respondents. Occasionally, you have done this. But you did not report sample size on this particular poll in question. Why not? (My guess is that it was too small and you didn’t want readers to know how few readers responded. Please correct me if I’m wrong.)
Second, in the spirit of validity and transparency, you should also be stating your circulation (print and online). So let’s say your recent poll (re: the plight of Abbotsford’s homeless and whether or not your readers are now more sympathetic) reached 36,000 readers who all had an opportunity to respond. Based on this theoretical circulation number of 36,000, let’s also say 100 responded. In this case you should be indicating that this response rate represents approximately .277% of all Abbotsford News readers. In other words, this response rate would represent a tiny or insignificant sampling of public opinion- one that no one, who understands research, would bother to report.
So your bold statement (again to 36,000 readers) that “YOU RESPONDED” would in this case be considered, at best, gross exaggeration. At worst, MISLEADING!
Now, if you knew that the 100 people who responded (insignificant because of size alone) was REPRESENTATIVE of your 36,000 readers of the Abbotsford News, then you could draw some modest conclusions and share related insights – especially if you took the time to conduct some additional and legitimate FOCUS GROUPS or INTERVIEWS that accurately represent your readers. ALL READERS! NOT JUST THE USUAL SUSPECTS! In research this is called a REPRESENTATIVE SAMPLE. Which brings me to my third point . . .
Your POLLS, in their current form, cannot guarantee a representative response. (Please correct me if I’m wrong.)
For example, if a special interest or advocacy group that was not sympathetic to the homeless in Abbotsford wanted to “fix the poll results” in order to influence or juice public opinion (similar to the influence of click farms, I suppose), they could simply orchestrate a timed response by their members or followers . . . and basically, skew poll results.
Sad, but true fact of life but, of course, you know this. It’s why research has rules, boundaries, limitations, and most pollsters know this only too well.
So in future, please clearly explain the limitations of your poll results and the “approach” you have taken.
NOTE
An individual’s opinion on any given issue is valid. But PUBLIC OPINION POLLS, and the reporting of collective results, are a very different matter. And, unfortunately, your POLLS – in their current form – are not a valid or accurate representation of public opinion in Abbotsford.
Sherril Guthrie is resident of Abbotsford and a researcher for The Guthrie Consultants Group Inc. She has been responsible for the Mayor & Council Performance Review and has researched and written extensively on issues, politics and public administration in Abbotsford.