Research

Research is imperative!

Most arts promoters and organizations will need to conduct market research at some point. It is better to do this now, then wait until it’s too late!

Why Bother?

• To collect information for a specific purpose or to resolve an identified issue. For instance, finding out which method of promotion will work best or what barriers to attendance exist.
• To find out more about your current or potential audiences/ customers.  
• Often within small organizations information about audiences is kept in an informal or unrecorded way, risking losing everything when personnel changes. Keeping up-to-date research will help sustain audience development even through fundamental personnel changes.  
• Successful audience development relies upon talking to your current and potential audience/ customers, and acting upon the information they provide.  
• Research demonstrates to funders and stakeholders that you are achieving your objectives, or that you are serious about achieving them.  
• Research can also support your case for funding.  

Research Planning

All research projects should go through five stages, illustrated below.

1. Define you research objectives

2. Develop your research plan

3. Collecting the information 

4. Evaluation and Analysis 

5. Present and utilise your findings  

1: Define Your Research Objectives

Don’t conduct research just because you feel you should – you’ll waste your time and money.

The answers to these questions will guide and define your research project:

–          What do we want to know?

–          Why do we want to know it?

–          What will we do with the information?

2: Develop Your Research Plan

With your objectives set out, you now need to consider the following:

–          Who is best to answer these questions, who should be surveyed?

–          How many people should be surveyed to ensure accurate and credible results?

–          How will you choose which people to survey?

3: Collecting the Information

There a plenty of methods of collecting research data, it is up to you to decide the method. Ask yourself:

–          What will be most appropriate? What method will garner the most responses?

–          Will you be commissioning a researcher, or will you do the research in-house?

–          Will you need to recruit any help or train any staff when collecting the information?

4: Evaluation and Analysis

Evaluation and Analysis is a time consuming task, but is the purpose of all your time and effort spent on planning and conducting your research.

Identify who will be evaluating and analyzing the information from the start.

5: Present and Utilize Your Findings

–       Who you are presenting your findings too?

–       How will you present your findings?

Who will be responsible for acting upon these findings?

Picking the Right Research Method

There are 3 types of research that are available to you:

Desk Research:

Is historical information that you already have about your organization and its audiences; this may include past research and financial reports. 

Secondary Research:

Is relevant information that is available in the public domain that may relate to your current or potential audiences, or to other similar organizations or services in your area.

Primary Research:

Is the research you will need to carry out to answer the questions that you have identified in your research plan. Your should only undertake primary research when you have ascertained that the information you require is not available from Desk Research or Secondary Research.

Thorough research projects are likely to include and contrast findings from all three of these research types.

Quantitative vs. Qualitative Research

Quantitative Research: uses statistical methods to count and measure outcomes from a study – all information other than commentary i.e. ratings, rankings, yes’s and no’s.

Qualitative Research: tries to understand people’s beliefs, experiences, attitudes and behaviour and interactions, non-numerical data – audience’s verbal answers in interviews, focus groups, or written commentary on questionnaires.  

Desk Research

This includes research or information readily available in-house, and may include:

• Your customer database
• Information included in past funding applications or annual reports
• Ticket sales and financial information/ audience figures for past performances
• Past publicity material
• Your own and colleague’s knowledge of the history of your organization.

Secondary Research

This refers to publically available research, carried out by other organizations.

Sources include:

• VisitScotland.org
• Highlands and Islands Enterprise
• Scottish Arts Council
• Census Reports
• Target Group Index

Primary Research

Once you have discovered all the relevant information about your region and audiences from secondary ‘public’ information, you will most probably need to do some of your own primary research.

By comparing and contrasting your own research with secondary data you will be able to determine the most about your current and potential audience.

There are a variety of different methods of collecting primary data, it is always best to start with what is readily available to you and then collect new information.

Popular Research Methods

1. Box Office information on audiences and attendance
2. Questionnaires and Surveys
3. Focus Groups
4. Mystery Shopping

HI~Arts Audience Base software provides a free downloadable box office system for arts and cultural organizations, click through to the HI-Arts website for more information and to download.

A. Information already to hand

If you have computerized or manual booking information available from your Box Office operation you may want to collate this first, especially if it includes information on customer profiles.

 On analyzing this data you can determine:

–        Patterns of attendance

–        Your audience profile

–        The rate at which you are gaining or losing customers.

Touring Companies – you can ask the venues you visit to provide you with this information. It is advisable to agree this with the venues in advance to ensure that they are dealing with Data Protection in a way which will allow you access.

Computerized Box Office systems make is much easier to capture and analyze this type of information. If you are not doing this already you may want to consider how you can collect more information from your customers.

B. Questionnaires and Surveys

The most common form of research, however preparing a questionnaire is something of an art in itself. Customer Research Methods include:

• In-house questionnaires, completed by the customer.
• Postal surveys to your mailing list.
• Online self-completion questionnaires.
• Telephone surveys of attendees and non-attendees.
• One-on-one interviews (i.e. street research of non-attendees, in-house of attendees).

• Observing customer’s behaviour within your centre.

Costs: Relatively inexpensive.

For hard-copy questionnaires you’ll need to arrange printing and enough pens for people to fill them in.

For postal questionnaires not only will you need to arrange printing and postage costs, but you’ll need to consider how they will be returned – do you need to include postage and a return address?

Who will conduct the questionnaires? Are you going to employ additional staff to survey or assist audiences complete the questionnaire – this will certainly add an additional cost.

Online questionnaires are becoming more popular with arts and cultural organisations. There are a variety of free or low-cost websites which provide online questionnaire services, such as www.surveymonkey.com and www.questionpro.com. You’ll need to utilise your mailing list for an online approach.

Tip: When using a questionnaire bear in mind you will only get answers to the questions that you have asked.

Tip: Asking audiences to complete your questionnaire anonymously and in private will yield more honest and reliable information, as opposed to being interviewed by an organisation representative.

To bear in mind:

• Audiences are unlikely to spend more then a couple of minutes completing a survey – especially if they are not regular or committed customers. Therefore you’ll need to keep your questionnaires short and to the point.
• If opting for self-completion questionnaires it is important that your staff or volunteers proactively approach a wide range of audiences with questionnaires.
• Open question responses can provide excellent quotes from customers, which you can use for potential marketing campaigns or for funding applications.
• Questionnaires are most useful for indicating general trends, rather then exploring issues in great depth (that is, most questionnaires give a finite number of potential answers to each question that the customer can tick circle). This type of research is called quantitative research.  
• Open questions elicit more qualitative information. When it comes to analysing the data from questionnaires, qualitative information is harder to compare across a sample the quantitative information.

How to get a good Questionnaire response

• If conducting questionnaires make sure they are clearly laid out, has an obvious structure of questions, and is as brief as possible.
• Either attach an introduction or post a notice explaining the purpose of the questionnaire, or have your volunteers and staff explain to audiences beforehand.
• You could offer a prize draw as an incentive, this works particularly well with online questionnaires, and can prevent people from submitting in multiple answers.
• If not collecting completed questionnaires by hand then clearly identify a space/box for audiences to return them, or leave boxes by exits.
• Anonymous questionnaires do elicit more honest answers, however you can use this as an opportunity to expand your mailing list by adding a section for audiences to sign up and leave their details.
• Consider how respondents will return their questionnaires, they may feel uncomfortable handing it back to a member of staff, who may read it right away. Try and ensure anonymity when required.
• Make sure that staff members of volunteers are fully informed about the research you are conducting, so they can assist audiences. 

C. Focus Groups

Focus Groups are an excellent way of collecting qualitative research from a group of people on a specific issue, campaign, product or service. This is an interactive setting where audiences are encouraged to talk in an open forum.

Focus Group sessions should last no more then an hour, be held in a comfortable and informal environment, sometimes holding these sessions away from the venue up for discussion.

Choosing your focus group – the composition of your focus group is key. The dynamic of focus groups work best when you have between 6 and 10 participants– any more then this and it can become unmanageable and people will not get a chance to speak.

Leading the focus group discussion: 

• Make sure you have a structured discussion, preparing a series of 4 to 6 questions with entry level questions leading towards more focused and direct questions.
• When conducting focus groups ask open questions to stimulate debate, rather then closed, one word answer, questions.
• You should aim to spend the vast majority of the time listening not talking. Your role as lead will been to keep the conversation on topic, or steer the conversation towards areas you want to explore, without stilting the conversation or cutting it short.
• As leader you need to ensure individual participants do not monopolise the discussion, and to draw quieter participants into the discussion.
• Try to remain detached as a leader, even if they criticise your organisation, it is your role to listen rather then become defensive.  
• If you are inviting current non-attenders, perhaps you should provide a tour or invite them to a performance prior to the focus group so you can explore their experience compared to their expectations.

Tip: Avoid ‘mix and match’ focus groups. Try and include people who have the same level of participation with your venue or organisation (i.e. don’t mix frequent attendees with non-attendees.

Costs: Depending on who you aim to invite, you may need to provide a small incentive for people to take part (£10 and travel expenses for instance), often people are flattered to be asked and quite enjoy the experience.

If your focus group is to consist of supporters or current audiences, you may find that you do not need to pay participants.

It is good practice to provide drinks and nibbles. Don’t forget this is a good public relations opportunity for your organisation.

D. Mystery Shopping

Involves sneaky but enlightening research approaches of eliciting information about your organisation, which you might otherwise be impossible to get.  

1. Recruit someone to ‘road test’ your organisation. By going undercover as a customer to experiencing your venue or show, and then reporting back on their experience. The areas to cover should include, but is not limited to; customer service, staff knowledge, décor, signage, information available, quality of experience, etc.
2. Recruit someone, who is not known locally, to visit other venues (i.e. the post office, library, Tourist Information, Council, etc) to ask about your venue – Where is it? Is it worth visiting? Is it expensive? Genuine good word of mouth is worth more then any paid advertising.

Samples

When conducting a research project you it is essential you obtain information from a representative sample of your audience or non-attenders.

The challenge is getting an accurate cross-section of people – otherwise your answers will be relatively meaningless.

If you have access to your audience figures it’ll be easy to determine a representative sample for your attenders and non-attenders.

Don’t pick a certain time of day or day of the week to hand out questionnaires – even in busy times make an effort to get your audience to complete one – make sure your whole audience get an equal chance of completing one.

Sample sizes

• An accurate sample and reliable results depend on you having control over who gets a questionnaire. It is important that who is handing over the questionnaire do so without fear or favour.
• Depending on the type of venue or event you are running you will be able to best determine how you approach people. Is there a certain point where people walk into the venue you could offer the questionnaire?
• Most people enjoy having their opinion sought and will generally be happy to co-operate provided they know what you are trying to do.

How many people should you sample? In order to be able to draw conclusions with confidence fro your survey you will need a sample large enough to compare:

• Day of the week
• Type of event/exhibition
• Male/female
• Regular as well as first-time attenders
• Catchment areas.

 You will find you’ll give out more questionnaires then you will have returned.

Postal Questionnaires are lucky to return a 10% response rate.

In-house questionnaires ensure a higher response rate, as there is a higher level of motivation for the participant to fill them out there and then.

Rather then setting a number of questionnaires to be completed, which will limit your sample and your results will be incomplete. Instead plan to cover a minimum number of events or the run of an exhibition.

Online questionnaires have been found to garner a higher response rate for similar reasons, the ease and minimal effort required, as opposed to returning postal questionnaires, they tend to encourage potential participants to respond.

It’s all in the Math

The Basic Rule: the larger your sample size, the more likely it is to be representative of the market as a whole.

• You can begin by applying normal distribution statistics to sample sizes over 50 people (and non-parametric statistics to smaller sample sizes), it is usual to interview a minimum of 150 people.
• Tracking Research: if you want to know whether your results are better or worse then last time, you should really use samples of over 200 people on each occasion. This will allow you to say that one year’s results are better or worse then the previous results if you are 10% apart or more (where the scores are nearer 50%) or 6% or more (if the results are nearer 0% and 100%).
• If you’re using a sample of 1,000 people on each occasion, the results will only need to be 5% or 3% or more apart respectively

Target Markets Under 300

You can do with a sample of less then 100 – however the need to obtain a good random representative sample becomes even more critical.

Often with smaller target markets, it is easier to get as close to the entire 300 (or less).

Target Markets Over 300

You should use a sample size of at least 100 to 200.

If you want to separate the sample into several categories, you will roughly want 100 in each category. i.e. if you want to compare consumers aged 18-35 vs. consumers aged 36-55 vs. consumers over age 56, you will need to gather at least 100 responses in each category.
 

Sample types

The way you choose your sample will have an inevitable bearing on your results and what you will be able to conclude and analyse from your results. Below are a number of possible sample types you can apply:

• Random: e.g. every customer has an equal chance of being picked.
• Systematic: e.g. every third customer is chosen to be part of the sample.
• Cluster: e.g. people are chosen from a specific area (i.e. living within 5 miles of the venue).
• Quota: e.g. sample is selected on the basis of the characteristics of customer profile (i.e. first time attenders, friends membership subscribers, tourists).
• Stratified: e.g. choosing people from a specific group in the population (i.e. young people, people with disabilities).

Remember: The results of research are only representative views of the people you have surveyed, and not of the public as a whole.

Checking your sample

You should aim to get back at least 70% of questionnaires you hand out.

It is advisable to check three main points to make sure you have a good sample:

1. The number of questionnaires you give out
2. The number returned and completed
3. The number of attenders on the days you sampled.

Using Your Results

When analysing your data always start by reviewing your research objectives. This will help you organise your data and focus on your analysis.

Analysis of qualitative research

1. Read through all the data.
2. Organise comments into similar categories e.g. concerns, suggestions, strengths, weaknesses, similar experiences, recommendations.
3. Label the categories thematically.
4. Attempt to identify patterns, or associations and casual relationships in the themes.
5. Keep all commentary for several years after completion for future reference.

Analysis of quantitative research

1. Make copies of your collected data and store away the master copies, using the copy for making edits etc.
2. Tabulate the information i.e. add up the rankings, ratings, yes’s and no’s for each question.
3. For ratings and rankings consider computing a mean, or average, for each question.
4. Consider conveying the range of answers.

Interpreting you research

• It is important to put your research into perspective – compare the results to what you expected, or found in your desk research.
• What were the discovered strengths and weaknesses of your organisation/ event/ activity/ venue, and the potential opportunities and threats?
• You may need to look at why frequent attenders choose your venue/ event/ activity over other activities.
• You’ll need to compare your frequent attenders with first-time attenders, are they the same type of people? How do they differ?
• Consider the recommendations to help your staff or volunteers to improve your programme, service or activity.

Reporting your Results

The level and scope of content depends on whom the report is intended e.g. board of directors, funders, customers, the public.

Be sure that your staff or volunteers have an opportunity to review and discuss your findings. Translate recommendations into action plans, with assigned tasks to carry forward the research results.

You can Download this Tip Sheet and many more by visiting out site HI-Arts Audience Development.