Effective performance marketing & optimization is critical to the success of every program. We feel the details of our success come down to the right people with a valid process using capable tools. Within Test and learn the gap between concept and actual implementation often seems to be the true difference between success and failure. With has years of extensive testing expertise and bening comfortable acting as the lead or support on any project. The steps of our process are outlined below:
1) Current State Assessment – Our testing process always begins with current state assessments, reviewing assets, campaigns and general data to determine what is meeting objectives or not. Baselines and benchmarks are also determined at this stage. A big part of this is understanding historical data. Most companies will try and find value from simple business experiments. It’s easier to draw the conclusions using data generated through experiments than by studying historical transactions. It is important to understand both, to avoid swim lane measurements and not look at external data.
2) Define Objectives – in this stage we identify business/financial, customer, branding and operational objectives via a testing measurement plan. Key learning objectives are defined and aligned against business objectives.
3) Idea Intake and Brainstorm – Theory & Hypothesis Creation -This leads to the 2nd phase of the process with is the idea intake after the review of data. Theories and hypotheses on why things are happening will be discussed between a variety of groups involved. At this point no idea is a bad idea, brainstorming should include easy and complicated theories. After a variety of theories are discussed and an ideas generated that include Creative, UX, content, performance CTA’s, Design style, Site structure and more, hypotheses can be put together for the tests. A hypothesis is created in three steps 1) Theory is created on what is happened 2) The quantifiable metrics in question are determined 3) Hypothesis will include the theory, quantifiable metrics and the delta determined on the test.
4) Test Plan Matrix – Once Hypotheses are determined, the test details will need to be outlined to ensure the test is actionable, significant and all details are communicated for alignment. Running a business experiment requires two things: a control group and a feedback mechanism. These will be outlined within the test plan matrix. Feedback mechanism metrics include behavioral and perceptual metrics. Behavioral metrics measure actions such as actual purchases or online engagement. Perceptual measures indicate how customers think they will respond to your actions. This form of feedback is obtained via surveys, focus groups or other forms of market research. These measures are useful in diagnosing intermediate changes in customers’ decision processes.
5) Prioritization – a unique testing scorecard will be created based on criteria important to success. We will score all tests against the pre-determined criteria and rank tests in order
6) POD Calendar Client Approval – tests will be placed in ‘Pods’ or groups of tests to be executed at a time. This is done to ensure clean and agile testing. Certain tests should not be run at concurrent times. Also, a defined 6 month schedule does not work well because out of running tests, new theories and tests may be generated and might be prioritized above others. By using a ‘Pod’ calendar, we are able to be agile and provide clients with the most impactful testing possible. This Pod calendar would be reviewed and approved on a regular pre-determined basis.
7) Execution – Go live! We have a strict Q/A process on the execution stage which will minimize error and increase launch times and quality of data.
8) Analysis & Optimization – Ongoing analysis and reporting will be communicated throughout testing. It is important to also have a clear start and end to each analysis, as structured learning is key to the data being generated.
9)Test Evaluations -Once tests reach statistical significance, an analysis which includes both quantitative and qualitative approaches.
10)Promotion of Winners – Test winners should be promoted live and used as the new ‘Baseline’.
11) Innovation – The cycle continues back to to first step. Through the evaluation and promotion of winning tests, innovative brainstorming should follow to determine new theories and tests. Innovation occurs when research and insights and intuition work as a team.
Given the suggested process, it takes three key groups which are: People, Process and Technology. It is extremely important for this to be outlined and strictly followed throughout testing. This will eliminate error and assure good testing, which is critical as any errors in testing could be costly given the time it takes tests to be completed. Test and Learn is the Core of Analysis. In addition to the process for effectively driving continuous learning, testing has a tight bound with analysis. To learn what delivers business value, the focus has to be on the business questions. Te experience to take the lead or provide support within any or all of these groups. We can provide strategic support in creating and determining the quality assurance of each test. We have the experience to collaborate with various groups to ensure the process is followed and communicated, including the key stakeholders from appropriate parties. We also have the implementation and technology experience and could execute testing and provide reporting and analysis. Testing is a method and science of optimization where the performance of variables is compared against each other real time in order to measure the effect that each version has on your conversion rate. Testing never stops, findings from each test feed into other marketing channels, improve ROI, and are leveraged when devising the next test. The expertise in any of the outlined process and the key groups to get constant testing done in required.
