Testing Advertising

CREATIVE DEVELOPMENT

Communications research is useful at two points in time: once strategy has been decided upon exploratory research of a qualitative nature is often helpful to the agency and the brand group. This is called creative development research. Its purpose is to provide feedback from target customers to rough ideas, concepts and executional directions for the benefit of the creative group. This is not testing or evaluative research, and so is not required to meet rigorous standards of protocol.

Creative development research can be done with focus group sessions, one-on-one interviews or small (mini) groups. Respondents are exposed to a variety of stimuli, rough executions, headlines, concept statements and the like, and are questioned about the reactions, feelings, associations and attitudes associated with these stimuli. It is up to the moderator and the creative people who are observing to interpret and draw conclusions from what they see and hear. There is no structured set of measurements for creative development research. It is successful when the creative people come away with new ideas and insights.

MEASURING ADVERTISING EFFECTIVENESS

The second point where communications research is more than useful is after a campaign has been executed but before it has been placed in the media. This is the time for evaluative research. In the case of pre-testing, there are formal guidelines and protocols to follow. Such issues as technique, exposure method and measurements are vitally important to the pretesting of advertising.
The goal of pretesting an ad or commercial to to determine if the proposed advertising is effective before exposing it in the media. Effectiveness is defined as the ability of the advertisement to achieve the goals for which it was created. In most cases there are three criteria that comprise effectiveness: awareness, memorability and persuasion.

Awareness, memorability and persuasion

There are many pretesting services that offer standardized techniques designed to evaluate advertising effectiveness. The three basic yardsticks of advertising effectiveness are awareness, memorability and persuasion. These can be measured individually or in combination.
Awareness refers to the ability of an advertisement to attract the attention of a reasonably large proportion of people who have an opportunity to see it. Measuring awareness is important since the first job of any ad is to get people to notice and attend to it. There are a number of ways that the awareness-getting ability of an ad can be measured. Placing the ad in an environment that contains many other ads as well as entertainment material creates a realistic test.

Measuring the number of people who attend or read the ad in this environment and expressing this number as a percentage of the total number or people who were exposed to the environment (or had an “opportunity to see” the ad) provides an “attention score”.

Memorability refers to the extent to which people who are exposed to an ad retain something about the message as a result of seeing the ad. This is important, since most ads are designed to impart information as well as have an emotional impact on readers/viewers. Memorability is particularly important in the case of a new to market introduction, where there are no residual impressions on which to build a brand image, and the product has a limited time to establish itself in the competitive environment.

Persuasion, on a macro level, refers to the ability of an advertisement to affect people’s attitudes towards buying or using the brand. On a micro level, persuasion can mean the ability an ad the change the way people perceive the brand, particularly with regard to the strategic issues being addressed by the advertising.

Measuring Awareness

This is the simplest of the three main criteria of effectiveness to measure. Awareness is usually measured by placing the test ad or commercial in the context of other advertising and editorial (portfolio or magazine) or programming (tv or cinema) without any prior notice to the respondent. Those who are exposed to the portfolio, magazine or program are generally questioned immediately after exposure.

They are asked to recall advertising they read or saw in the test vehicle, and if mention is made of the test brand or ad, they are asked to describe what the remember seeing or reading. This is considered “unaided” recall. Those who do not mention the test ad are asked if they recall seeing or reading an ad for (test brand). If they do, they are also asked to describe what they remember seeing or reading. This is called “aided” recall.

Awareness is expressed as the percentage of people who had an opportunity to see an ad or commercial who recall reading or seeing it at some time after the exposure. In a sample of 100 people, 30 say they recall reading an ad for (test brand) and are able to describe something specific about the ad, the awareness or “recall score” for this ad would be 30%. Whether this is a good, bad or indifferent result depends upon the action standards which were set prior to the test. If the brand and category have a history of test results for comparable situations, this “score” can be evaluated in the context of that history. Without such a context, there are no absolute standards against which this result can be judged.

Measuring Memorability

Memorability is a more complex dimension to evaluate than is awareness. Usually, memorability is determined by the amount of information that people can recall or “play back” in an interview some time after exposure to the ad or commercial.

The “playback” obtained in the post-exposure interview described above is the source of the “memorability” information.

The more people who can accurately recite the content of the advertising, the more effective it is in registering its message.

However, there is a catch to this issue. Memorability does not necessarily mean that people are persuaded by the message. In fact, sometimes people can recite the copy points of an ad almost verbatim, but their perceptions or image of the product that those copy points were designed to change remains unchanged! There are instances where playback of the copy content does not reveal the true impact of an advertisement on people’s perceptions of image of the brand. in other words, what people say you said is not necessarily what they feel you meant to say.

An ad designed to communicate “prestige” may successfully convey that this is its intent, but the image of the brand’s prestige could remain unmoved by exposure to the ad. Therefore, it is important to measure the impact of an advertisement on people’s perceptions or image of the brand in addition to knowing what they can recall from its content.

Measuring Persuasion

On its most macro level, persuasion is the ability of an advertisement to increase a person’s the propensity to buy the brand as a result of exposure to the advertising.

Since most people do not like to admit that advertising influences their buying decisions, direct questions will not successfully measure the persuasive power of an ad. It must be measured indirectly.

This can be done in one of two ways: one is a Pre-Post design, whereby the brand preferences of the sample are measure before exposure to the advertising and again after exposure.

Differences in the percentage of people who choose the subject brand after exposure are attributed to the test advertising. The strength of this design is its sensitivity. It does not take very large differences between pre and post levels to reach “statistical significance”.

The weakness of this design is bias caused by transparency of the test. When people become aware that the test is measuring their brand preferences, their answers are affected by more than just the impact of the test
The second design for measuring persuasion is called Test-Control, whereby separate and independent samples of people are used, one is exposed to the test ad and a second sample is exposed to a control ad or no ad for the brand at all. The level of brand selection or attitude toward buying the test brand is compared between the test and control samples.

If a larger number of people in the “test” cell favor the advertised brand than those in the control cell, the difference can be attributed to the test advertising.

In such tests, it is important to have large enough cells to make statistical comparisons (at least 100 people per cell), and to make sure that the cells are as evenly matched in critical characteristics so their pre exposure attitudes to the test brand will be comparable.

The test-control design can also be used to measure persuasion on a micro level. That is, a battery of image dimensions can be included in the interview, and the ratings of the test brand on these dimensions can be compared between the test and control cells. The dimensions should be strategically relevant to the advertising, and in this way, a more valid measure of the advertising impact on brand image can be obtained than by using verbal playback of advertising copy.

For example, by comparing the rating of the brand on a dimension such as “prestige” in the test cell with its ratings in the control cell, the impact of the ad on this dimension will be revealed.
Criteria for evaluating testing techniques

There are basically four standards that can be applied to any test design:

1) How “normal” is the test situation?

2) How normal is the exposure technique?

3) How many exposures to the advertising (film/video) are incorporated? 4) How actionable will the results be?

Taking each in turn

1) The test situation: The test situation should ideally be in the respondent’s own home or the environment where the advertising is most likely to be seen under real life circumstances. To the extent that the exposure situation is unlike “reality”, the effect on the responses of respondents may bias the results of a test.

2) The exposure technique: The advertising should be exposed in the context that most closely parallels its “normal” environment. This means in a magazine or billboard for print or in a program for video or cinema for film. Removing an ad from its normal context will cause it to be perceived differently in unknown ways.

3) The number of exposures: Most advertisements are seen two or three times before they have an impact. This is particularly true for video, which usually has too short a duration (30 seconds) to deliver its message fully on first exposure.

For this reason, tests that incorporate multiple exposure should be sought over those that rely on a single exposure. In the case of print, techniques that permit repeated and/or prolonged exposure should be favored over those that use a single brief exposure at one point in time.

4) Actionability: If a pretest is being done for a go/no-go decision, the action standards need to be outlined in advance and the ways the test will be interpreted should be clearly spelled out beforehand. To the extent that “norms” can be used, this is helpful, though norms must be viewed with a critical eye. Only those tests which most closely parallel the one being done with respect to strategy, target, category etc. should be used as yardsticks.

Some of the issues involved in choosing a pretesting service include:

Geographic location Locations of test sights may vary or be standardized. The appropriateness of the markets for standardized services should be evaluated for each brand. Cities with atypical consumption of the product or category should be avoided for test purposes.
Sampling Each pretest service has its own methods for selecting and drawing the sample for a pretest.

These include:

...Method for recruiting--Central location intercepts,Telephone, Mail, Social groups (church, school)
...Nature of the sample-Cross section of general population, Screened for specific characteristics
..Sampling method--Quota, Probability, Haphazard
...Sample size per test cell--\Range from 100-500

Test location The location of the test site is important to consider. Generally there are four alternatives offered by pretesting services: a mall/hall research suite, a commercial meeting room in a hotel or hall, in-home, or the private testing sites of the service.

Exposure method Keep in mind that the more natural the method of exposure, the more valid will be the test results. As exposure techniques become more unreal, the chances increase that the test situation will influence the results. Pretesting services offer a range of exposure techniques. Some use forced exposure with a captive audience, others use a partial “forced exposure” by inviting the audience to view, and the most desirable use a natural exposure either on-air or in-home. For print testing, some services place the test ad ina “portfolio” which is a mock magazine, while others use slides that are exposed for brief intervals, and others simply place the test ad in a folder with a group of eight to ten “clutter” ads.

For t.v. commercial testing, some services place the ad into a program along with other commercials. The program may be exposed in a studio, over cable or closed circuit t.v. or via video cassette on a monitor. Again, the more normal the context of exposure, the more valid will be the test results.

Timing of the interview For the most part, it is desirable to limit the questioning of respondents prior to exposing the test advertising. This limits the degree of “forewarning” or calling undo attention to the test subject. Basically, pre-exposure questions should be limited to qualifying matters, preferably disguised in the context of other questions.

Post exposure questioning can take place immediately after exposure, if a measure of awareness and persuasion is taken, or it may be delayed as much as 24 hours, to measure residual memorability and persuasion.