When it comes to research, online polling has an image problem. Federal Labor minister Craig Emerson was at it again this morning, saying the Essential Report was not a representative sample.
Part of the problem with online polling, is that it is often confused with the self-selecting web-based surveys that many news sites run – these, of course, are not scientific.
But evidence is emerging that professionally sourced and managed online panels are every bit as representative and accurate as the traditional method of telephone polling.
It has been estimated that more than 50% of all market and social research in Australia is conducted on line – it is relied on by major companies (and Governments!!) to make key decisions about their products, services, marketing. If it wasn’t reliable and valid they would be putting their businesses at considerable risk.
In the UK, on line panels were used by several companies for their pre-election polls – they were no less accurate than the telephone polls. The same occurred in US with the Presidential Election.
All polling methods have their problems – none are perfect. However, on line polls provide an effective measure of public opinion if the panel they are drawn from is large, if the demographics of panel members is broadly representative of the population profile, if members are selected at random and if weighting is applied to match the sample to the population profile.
For example, online panels are based on the fact that participants can be targeted to get a clear demographic spread – and that by allowing participants to fill in a survey I their own time, significant barriers are removed.
For example, research based on telephones to home numbers skews participation from a number of groups including working mums do you really have time to go through a long phone call around dinner time) and younger people (many of whom do not have a landline at all)
The counter claim is that online discriminates against the digitally poor – particularly older people and poorer people – but as access to technology increases, this is becoming less of an issue.
Anyone receiving the Essential Report can read the fine print – but for the record here is how we put the poll together:
“The data gathered for this report is gathered from a weekly online omnibus conducted by Your Source. Your Source is an Australian social and market research company specializing in recruitment, field research, data gathering and data analysis. Your Source holds Interviewer Quality Control Australia (IQCA) accreditation, Association Market and Social Research Organisations (AMSRO) membership and World Association of Opinion and Marketing Research Professionals (ESOMAR) membership. Senior Your Source staff hold Australian Market and Social Research Society (AMSRS) membership and are bound by professional codes of behavior.
Essential Research has been utilizing the Your Source online panel to conduct research on a week by week basis since November 2007. Each Monday, the team at Essential Media Communications discusses issues that are topical. From there a series of questions are devised to put to the Australian public. Some questions are repeated each week (such as political preference and social perspective), while others are unique to each week and reflect prominent media and social issues that are present at the time.
Your Source has a self-managed consumer online panel of over 100,000 members. The majority of panel members have been recruited using off line methodologies, effectively ruling out concerns associated with online self-selection. Your Source has validation methods in place that prevent panelist over use and ensure member authenticity. Your Source randomly selects 18+ males and females (with the aim of targeting 50/50 males/females) from its Australia wide panel.
An invitation is sent out to approximately 7000 – 8000 of their panel members. The response rate varies each week, but usually delivers 1000+ responses. The Your Source online omnibus is live from the Tuesday night of each week and closed on the following Sunday. Incentives are offered to participants in the form of points.
EMC uses the Statistical Package for the Social Sciences (SPSS) software to analyse the data. The data is weighted against Australian Bureau of Statistics (ABS) data.
The debate between online and phone polling
Ultimately, the relative merits will be based on results – and as the election looms over the coming months we recognize our methodology will be put to the test.
What we can say is that the trends being identified by the major phone pollsters is also flowing through our online research – and that is why more and more more people are taking our findings seriously.
As for Craig Emerson, if he thinks the Essential Report is wrong on Afghanistan, there is always another research technique he could consider employing – its called talking to the people who put him in power.
Peter Lewis, Director, EMC