Gender impact

From Quirk's Marketing Research Review, July 2009. (for more details see: www.quirks.com )

THE SUMMARY

The prompt

In February 2007, Stephan Althoff[i] reported that when women sent email invitations to participate in some German industries’ online surveys, the response rates were better than when men had sent the invitations. He also noted that this difference had not been replicated in general community online survey trials.

These initial German findings suggested two gender-related hypotheses which we subsequently tested in Australia.

The gender influence hypotheses

The first and obvious hypothesis was whether response rates from emailed invitations to participate in online surveys were higher when the emails are apparently from a woman.

The second hypothesis was that women answer online survey questions less openly than men, because of a lesser willingness to expose themselves in a more assertive Australian male culture.

If either hypothesis were valid in Australia, changes in marketing research practice would be needed for better data collection and strengthened online survey response rates.[ii]Hence, it seemed appropriate to test the hypotheses to determine the impact, if any, of gender.

No change needed!

The summary findings are that in the Australian context, the gender of the email invitation sender does not influence the response rates achieved in online surveys, and in online surveys, women answer questions as openly as do men, and.

There is room to test the hypotheses further, among specific audiences and topics and such further study is recommended.

Hypothesis 1 - Do Australians respond better to a woman’s email invitation?

On first view, the hypothesis that Australians would respond better to a women’s email appears reasonable. Australian men can appear more threatening than women – we know that four in five Australian murders and four in five Australian assaults are undertaken by men, making men actually and apparently the more dangerous sex[iii].

Conversely, gender in email invitations may be an irrelevance as the email appears on a computer screen as gender-neutral text on background, and many emails purporting to be from one person are commonly thought to have been sent by someone else[iv].

To test the hypothesis, we used clearly gender-different first names in email invitations to participate in four test surveys conducted between October 2007 and June 2008. The first three surveys were of women invitees only and the fourth test survey was with a larger female and male universe.

In accordance with the Australian Market & Social Research Society’s Code of Conduct, the email addresses used were those of real people and the gender-identifying names used are, in the Australian culture, mainstream names[v].

The first three online surveys were of samples of 3,500 women, 3,600 women and 1,000 women aged 25 to 54 years, who were invited to participate in a Shopping Centre café survey. Each woman received one personally addressed email invitation to participate, had eight days to respond and were offered the opportunity to enter a competition to win one of 13 individual $A100 cash prizes.

It was expected that the only known element in the email was a subject line note that the email was being sent to people who had completed a First Direct Solutions survey some time before receiving this survey invitation[vi].

Each sample was divided into State groups and each State’s sample was divided on an A-K and L-Z basis so that in each State, half the sample received an email from the named female researcher and the other half received an email invitation from the named male researcher. To avoid any possible alphabetical order or State bias, the alphabetical order of sample emailed by the female researcher in the first survey was emailed by the male researcher in the second survey.

The first two surveys used different survey links so the identity and gender of the email sender was clear. The third survey used one link and used the respondents’ email addresses to determine whether the female or the male researcher had invited the respondent to participate.

The gender of the email senders were confirmed when the recipients opened or clicked on from the invitation, as they were asked to advise the female or the male researcher, by name, that they had received the invitation.

The first survey (undertaken in October 2007) had one in four response rate, and the second and third surveys were undertaken shortly before or after the Christmas-New Year holiday season and had one in five response rates. The differences between the response rates from emails sent by either researcher were statistically insignificant as Table 1 shows. Each survey’s results were within standard error (2% and 3% for the first two surveys and 4% and 7% for the third survey, at the 95% confidence level).

Surveys dispatched within a month of the Christmas holiday periods (which, in Australia, begins on December 25, 2007) had noticeably lower response rates than the survey invitations sent some months before. This is definitely an Australian cultural issue as the December month is generally dedicated to finishing the year’s business, school and social activities, to shopping for Christmas[viii] and preparing for holidays after Christmas. Then, until mid to late January, more are on holidays or in caretaker mode than at other times of the year.

Almost all respondents provided their email addresses, as these were needed to enter the competitions to win one of prizes. In survey 1, 93% gave an email address; in survey 2, 96%; and in survey 3, 89%. In the fourth (male and female universe) survey, 94% of women and 94% of male respondents provided their email addresses to the male researcher and 91% of men and 94% of women gave their email addresses to the female researcher.

These high email address responses levels indicate the respondents were unconcerned about providing their email addresses to a male or to a female researcher, indicating little concern for the gender of the email invitation to participate. Once the decision to participate in the survey had been made, the sender’s gender appeared to be irrelevant.

A fourth survey test was undertaken in June 2008 with people who lived near Shopping Centres. The net 21,900 prospects universe was drawn from the First Direct Solutions panel previously used and a third of the sample were men (31%).

Each invitee was sent a one-time email invitation and in this survey, the panel source was not indicated in the subject line. Instead, the subject line stated the email was about a Food Court Survey – with cash prizes draw! The emails were again personally addressed to each prospective[ix]. The email letters were signed by either the female or the male researcher, with the same email and receipt details as before.

There was one unintended difference between the two samples in this fourth survey.

For dispatch ease, both samples had been grouped by the recipients’ home State and were sent out in mail-merged small batches of less than 2,000 emails per time. During the tedium of sending, as each batch was sent separately, the invitation line was inadvertently changed.

Three quarters of the invitations from the female researcher were sent with the planned Food Court Survey – with cash prizes draw! Subject line but a quarter were sent with the residents’ home State shown at the start and all the male researcher’s invitations were sent with the home State shown at the start – e.g. NSW Food Court Survey – with cash prizes draw!

This difference, rather than the sender’s gender, seemed to affect the response rates between the two senders, as table 2 shows.

The 3 percentage points difference in response is greater than the standard error would indicate (a 1 percentage point difference between the two would be expected). This difference appears to have been driven more by the State localisation of the survey subject line as both the male and the female senders received almost identical unsubscribe request and other email contacts from the people contacted, as the next section notes.

The Shopping Centre customers’ survey test had a larger universe of invitees and also had a larger number of responses than the earlier three surveys. The larger number was in surveys completed but also in the larger number of unsubscribe requests received and in email correspondence from invitees.

The researchers responded to all unsubscribe requests, acknowledging the request will be honoured, and responded to all other emails with appropriate answers. In doing so, it seemed that more emails came from recipients of the female researcher’s invitations and were often chattily addressed.

When analysed, there was minimal difference between the researchers in unsubscribe requests received or in email correspondence, as table 3 shows.

Table 3: Unsubscribe responses and general correspondence with male or female email senders, 2008.

This lack of gender difference in the Shopping Centre survey unsubscribe or email correspondence rates suggested we review these measures from the earlier surveys. As table 4 shows, there was essentially no difference in unsubscribe or in the email correspondence rates.

The responses to invitations from a male or from a female email address indicate no online survey response rate differences with the samples tested. This then indicates the email invitation sender’s gender is not an influence in the decisions respondents make whether to complete online surveys – at least for these topics and with these samples.

The survey results suggest that email invitations appear genderless as electronic text on electronic backgrounds on genderless computer screens. Factors other than the sender’s gender influence decisions to participate in a survey. In particular, the impact of inviting respondents to participate in a survey about their local (State) area seems to generate better responses and future testing will look to that as a way to strengthen response rates.

The further analyses of unsubscribe rates and of correspondence with the researchers also found no difference by the inviting email sender’s gender.

In testing whether women answered differently to men, we looked for a question that would be commonly asked of both and would be independent of gender-related behaviour. Bradburn’s findings that respondents in self completion surveys under-report answers to questions that make them feel uncomfortable[x] suggested such questions could be logical tests for gender differences.

As our previous online surveys had not asked which, if any, questions respondents were uncomfortable answering[xi], the “What is your total annual household income?” question was considered likely to be an uncomfortable question. This was because clients often expressed concern that it was intrusive, and Australian face-to-face interviews often use pre-coded answer sheets that give a letter or number for each income range, so respondents need not state an actual cash amount or range. The assumption was strengthened when we found that 15% of recent telephone respondents and 10% of recent mail survey respondents refused or did not answer when asked for their household income[xii].

The online surveys analysed for household income response differences all had optional household income questions. Answer places were shown but respondents could proceed to the next screen without answering (though respondents had not been told that). The results were analysed by respondents’ gender to see whether men and women responded differently to the presumed uncomfortable question about household incomes.

The first online survey results reviewed were from a sample of 6,420 Australian financial institution customers and the one-time email invitations were sent from an unknown and gender neutral) email address[xiii]. The email subject line said the email was an invitation to participate in the financial institution’s customer survey and participants could win a prize.

When the emails were opened or viewed in a reading panel, they were seen to be on the financial institution’s letterhead and signed by the financial institution’s male Chief Executive Officer. When the readers moved their cursors to another email or sought to close the email, they were asked to acknowledge receipt of the email to a male researcher.

The non-response to the household income question differed minimally between men and women. Only 3.7% of men and 4.1% of women chose not to answer the household income question – the difference of 0.4% was well within standard error (1.8% at the 95% confidence level). This study suggested gender had little influence on household income answers.

The second online survey was of 4,350 customers from another Australian financial institution but differed because a three contacts approach was used[xiv] to generate response. The same email dispatch and email receipts procedures were used and again, the email components were mostly from men.

The non-response to the household income question in this second financial institution’s sample differed markedly from that of the first study. In the second study, 32% of men but 40% of women chose not to answer the household income question – an eight percentage point difference and a significant difference (the standard error was of up to 5%).

This second study suggests women and men do answer questions that may cause discomfort differently or the women surveyed may not have known their household incomes. The survey had not asked why the household income question was not answered.

The findings of the two small-sample studies were contradictory and so inconclusive. The third study reviewed was of a large scale survey. In that study, 20,000 Australians were invited to participate in an online survey[xv]. The emailed survey invitation was sent from a gender neutral and unknown to-the-recipient email address. The subject line said the email was an online survey invitation, named the panel source[xvi], and said there were cash, iPods and cinema tickets as prizes.

When the email was opened or seen in a reading panel, the sender was identified as a male researcher and when the cursor was moved or the opened email closed, the recipient was asked to acknowledge receipt to the male researcher.

In this larger survey, 0.2% of men and 0.4% of women chose not to answer the household income question. The 0.2 percentage point difference in response from women was well within the 1.7% standard error expected. This finding in itself begins to question conventional wisdom that the household income question is an uncomfortable question, when the assumption is made in relation to these online surveys, where the household income question is asked as the one question on its own screen, as was done on these surveys.

The fourth study reviewed was the June 2007 Shopping Centre survey with its male and female universe. In that survey, 4% of men and 5% of women did not answer the household income when asked by the male researcher; and 4% of men and 6% of women when asked by the female researcher.

The two larger universe survey findings suggest no gender difference on response to a possibly uncomfortable question. So the mostly similar (with one discordant) results from these four studies suggest gender is probably not a factor in not answering questions about household income.

The first hypothesis – that Australians respond better to an email invitation from a woman to participate in an online survey – was found to be not valid for the respondent groups tested. Rather, the survey results indicate other factors influence decisions to participate in a survey, or not. In particular, the impact of inviting respondents to participate in a survey about their local (State) area seems to generate better responses and future testing will look to that as a way to strengthen response rates.

The further analyses of unsubscribe rates and of correspondence with the researchers also found no difference by the sender’s gender.

The second hypothesis – that there are gender differences in answering uncomfortable questions – was not substantiated. Rather, in general, men and women appear to answer uncomfortable questions in online surveys at the same level. IN addition, the gender of the email invitation sender was not a factor affecting the level of answer given.

Hence, the overall conclusions that gender has no impact on online survey response rates nor on the decision to answer uncomfortable questions among Australians. Other factors – time of year, localisation, topic, and not tested factors such as survey design, are more likely to influence the response rates and question answers.

The conclusions drawn are based on Australian finance industry customers, women aged 25 to 54 years, Shopping Centre customers (male or female) and on a general population sample that was researched about financial affairs. Hence, it may be appropriate to test these findings in other industry sectors, about other topics, and with senders from minority groups, to be sure of the conclusions reached.

As a more promising area for immediate response improvement seems to be named geographic proximity. That will be tested further in the future.

[i] By the numbers: Does the survey sender’s gender matter? Stefan Althoff, Quirk’s Marketing Research Review, February 2007.

[ii] The AMSRS Professional Code of Conduct requires all contact be from a legitimate and real person. Having a male or a woman sender of an online survey invitation would be, for most Australian marketing research companies, an easy matter to manage, as the professional and managerial elements of this sector are almost equally divided between males and females – 48% of market research and statistical services managers are women as are 51% of professionals employed in this sector. Australian Bureau of Statistics, 2006 Census of Population and Housing, Table 1, Persons working in 6950 Market Research and Statistical Services by Occupation and Sex, Australia.

[iii] Criminal Courts, 2006-7. Australian Bureau of Statistics, January 28, 2008. Ref. 4513.0.

Conventional Australian telephone interviewing “wisdom” is that, generally, women are more effective interviewers, again because they are more encouraging and less threatening when making an initial telephone contact, though this has yet to be expressly tested. As telephone interviewing in Australia tends to be a part time occupation, and low income occupation, it is more the province of women and the two findings on interviewer success are that length of time as an interviewer relates to better telephone interview success and that accents not consistent with the population norms tend to have lower levels of success in recruiting telephone interview respondents as the report Interviewer experience, vocal skills and the recruitment of telephone survey respondents by David H. B. Bednall and David E. Hall, Australasian Journal of Market Research, June 2004, Vol. 12, No. 1. indicates. The telephone interviewer occupation tends to be more a female domain as 66% of clerical and administrative workers (which includes the telephone interviewer category) in the Australian Market Research industry are women, according to the Australian Bureau of Statistics 2006 Population and Housing Census report Persons working in 6950 Market Research and Statistical Services by Occupation and Sex, Australia.

[iv] Exemplified by the common spam emails “Hello! I am bored this evening. I am nice girl that would like to chat with you. Email me at femalename@anemailaddress.com only, because I am using my friend's email to write this. I will reply with my pics” or the “My name is Mohamed El-Timini, I am director in the project and planning department of the Iraq Ministry of Oil, I head a seven man Tenders Board Committee in charge of contract award and Payment approvals. The reason for contacting you is to seek your assistance in the transfer of funds into a foreign account.” in which the gender as well as the motives of the senders may seem different to those apparently presented.

[v] To comply with the Australian Market and Social Research Society’s Code of Conduct, respondent contacts must be from an identifiable, contactable and real people, so placebo or dummy names could not be used. The sender email addresses used in all tests were Sue.Derham@derhamresearch.com.au (the female researcher) and Philip.Derham@derhamresearch.com.au (the male researcher), as in the Australian culture, Philip is a man’s name and Sue a woman’s name. As the researchers’ names are mainstream, the findings relate to mainstream gender names, and may or not also relate to the findings that could follow from the use of names associated with specific ethnic or religious groups and further research on this may be advisable.

[vi] It was considered unlikely that the respondents would have heard of the research company or of the researchers before the invitation email.

[vii] 2% of the total responses from the survey 3 could not be allocated to either sender as the respondents had used different response email addresses than those in the database used to determine which sender the respondent had responded to.

[viii] The December month retail sales show a marked spike when compared with sales in all other months and the December sales between 2004 and 2007 accounted for 22%, 22%, 20% and 20% of all Australian retail sales. This heavy purchase concentration indicates perhaps prospective survey participants have other matters on their minds than surveys, hence the lower response rate in the December survey. Source: Australian Bureau of Statistics publication 8501.0 - Retail Trade, Australia.

[ix] By title, first name, family name, home suburb, Australian State and postcode (zipcode) and addressed to them as Dear Title and Family Name

[x] Question threat and response bias. Norman M. Bradburn, Seymour Sudman, Ed Blair and Carol Stocking; Public Opinion Quarterly, 1978.

[xi] The absence of this does suggest a possible future project!

[xii] Based on findings from three recent Derham Marketing Research community and three customer audience telephone surveys, and four recent postal self-completion surveys.

[xiii] The sender was derhamp@derhamresearch.com.au

[xiv] The three contacts were a pre-survey advice that the survey would follow next week, the survey invitation and a “quick, the survey finishes shortly” reminder a day before the survey closed.

[xv] . The sample had been drawn in proportion to the population, by Australian State and by age groups within each State. Each sample had a 60% female component. The end sample matched the State, age and gender distributions of the original universe invited to participate.

[xvi] First Direct Solutions.