Kitchn's Creative Testing Blueprint is your ultimate guide to building a system that will constantly output the best-performing ads. Learn how to structure the underlying processes within the creative testing system and evaluate your current approach. Finally, learn how to maximize results from great ads.
As you probably already realized, at Kitchn, we help advertisers automate paid social channels. Before we started doing that, we spent millions on Facebook and TikTok ads striving to achieve scale at a reasonable cost. Our team went through the same struggles as most advertisers. We were balancing out efficiency and scale, waiting hours for the Facebook Ads Manager to load, and trying to find new ways of testing ad creative that would boost the declining performance. We tested out all of the ad formats, bid strategies, and targeting options. As much as we would love to tell you the exact adset settings and the “secret sauce” to the winning ad with the highest CTRs… We won't. It's not because we don't want you to succeed, but because, with time, it's become clear that to grow, you should focus on a well-organized, structured, creative testing system.
Kitchn’s Creative Testing Blueprint o****utline:
A paid social system is a process to handle all paid social media activities. Before we go into an in-depth analysis of a good paid social system, let us explain why we prefer to think of it as a complex system rather than a single process.
First, one of the defining characteristics of digital marketing is that it is marketing, data, and creative all at once. This makes it very different from other channels in several ways. Nowhere else is the need for creativity and data-driven decision-making so closely intertwined. Is that a good or a bad thing? It depends. It does mean that paid social marketers must be prepared to wear many different hats every day. Beyond optimizing your ad campaigns for maximum return on ad spend, you also want to experiment with different types of ads. This way, you can see which ones resonate best with your target audience.
Another unique element of digital marketing is its many moving parts. Paid social is an ever-changing field with new tactics, strategies, and best practices emerging all the time. On top of that, it's unclear how all the algorithms work. As a result, we often don't know why one ad creative generates better results than the other. We cannot “crack” the algorithm and study how it makes decisions. But we can measure the input and the output and create a hypothesis for creative testing. As a result, marketers must stay on top of the latest trends and constantly output new creative testing ideas to see what works best for their brand. Technology is continually developing, too, so a good strategy will require constant monitoring, testing, and retesting to get the best results.
Unlike other channels, paid social ads are heavily tied to ad creative. Good ad creative is the core of the paid social system. Without excellent ads, it will be impossible to achieve meaningful results. When you analyze your performance, you're not just looking at the results of your previous ad campaigns; you're also analyzing your creative strategy. Is the creative you're using generating results? If so, why? If not, why not? It cannot be easy to do if you randomly test your ads and don't have a creative strategy. Have you thought about how your ad creative will affect your performance on Facebook, TikTok, and Snapchat? How can you use it to generate the desired results?
We strongly recommend thinking about it as quickly as possible. Otherwise, you will have a tough time. This ties into the previous section on creative testing. Without testing ad creative, you'll find it challenging to know what creative concepts to test and how to continue developing successful ads to boost performance.
If we had to pick one element to put at the center of the paid social system, it would be creative. So in this blueprint, we will mainly talk about testing creatives and building a functioning system around it.
In the first chapter, we will guide you through the process of building the creative testing system. We will specify its core elements and explain how they interact. In the second chapter, we will look a bit more in-depth into the underlying processes within the creative testing system. Then, we will evaluate your current system and suggest the focus areas to get you to the next level. Lastly, we will examine process standardization and discuss high-value and low-value processes. We believe that not every task requires human attention and creativity; some can be easily automated, for example, with Kitchn.io.
“A system is a network of interdependent components that work together to accomplish a common aim” - W. Edwards Deming
In this section:
According to W. Edwards Deming, a system is “a network of interdependent components that work together to accomplish a common aim.”
Systems do not necessarily have to be designed in a specific way to function. That means even if you don't explicitly come up with one, there are usually already processes and connections in place that have assembled in a way that they are working as a system working towards an undefined goal.
So here's our first clue: If you don't intentionally think about your Creative Testing goals and design your ideal system, you may end up with one doing something else entirely.
Even if we look at it in a more pragmatic way, it should be in your interest: defining each process and clarifying who is responsible for what part gives clarity to all stakeholders, increases productivity, and ultimately makes your process scalable.
The opposite of what we are looking for is one person doing everything intuitively based on our gut feeling without ever talking to anyone about it. This cannot be easily taught, makes it difficult to get other colleagues involved, and when this one person leaves your team, you have to start from scratch.
Lastly, clearly defined processes, best practices, ownership, and vertical division of labor lead to scalability and open up the space for automation. Once your creative system accomplishes the aim you set out to achieve, you will want to scale it up by either hiring more people or automating the more tedious parts. Either is impossible if you don't know how to do what you do.
The goal of the Creative Testing System should be to produce well-performing ad creatives. But, well-performing can mean something entirely different for each company.
For clarity, we will assume that we only run two types of ad campaigns in our ad accounts: Creative Testing and Scaling or Always-On, referring to an ad campaign that is always active but where old ads may be turned off or new ones rotated in.
So while the KPIs used to measure might be different, all setups have in common that once ads are declared as winners in our Creative Testing setup, it should directly translate to achieving success in our Scaling setup.
This means that we should test with representative targeting parameters and bid and budget settings.
Rolling out winning ads from creative testing into scaling ad campaigns
The success of your Creative Testing System is measured by its capability to produce well-performing creative assets and by its efficiency: How much work and effort is necessary to create one winning ad that helps us scale?
Let's call the metric which defines the above the system's hit rate.
Ultimately improving your hit rate is not as important as understanding what input is required to produce your desired output, well-performing ad creatives or “winners.”
How can we know how many winners we need?
Let's start with some assumptions:
Creative testing: CPM calculation
Assuming our target audience size is about 5M, and a “winner” ad stops working after reaching each person 2.5 times on average, we would require eight winning ads.
Required Winners = Impressions / (Audience Size * Ad Fatigue Frequency)
Creative testing: winners calculation
Assuming a modest 2% hit rate, we will have to test 400 creatives to achieve these eight winners.
Creative testing: hit-rate calculation
With sixteen new creatives for each test, we would need to run twenty-five tests, about one every two weeks.
Creative testing: required tests calculation
Of course, besides the already simplified assumptions above, the above example takes a reasonably straightforward approach to calculating how many creative assets may need to be tested. In reality, you may find ways to “stretch” your winning creatives by building close variants to the original winner. You may also find that your hit rate is better or worse. No matter whether it’s 2022 or 2023, best practices might change, but a good, functioning, creative testing system is always going to produce results.
But now that we have some numbers, we have a starting point to figure out answers to our following questions: What is needed to produce 16 new creatives every 2-week-period? And how can we improve the system’s efficiency and its hit rate?
In this section:
Now that you know the central pillars of building a successful system, let's evaluate your current creative testing best practices. You could be more advanced or just starting to test your first ads - it doesn't matter. The earlier you apply the systematic approach to creative tests, the earlier you will see your first success.
The first factor indicating the success of your creative testing strategy is best-performing creatives. Do you already have a couple of ads that are driving performance? Have you seen a significant uplift when testing certain ads? If yes, your main goal will be building the system that empowers your creative team to produce more of those. On the other hand, it can happen you spent a significant amount of resources but still haven't found any ad that will produce good results. If this is the case, let's start from scratch and re-evaluate the entire approach.
We believe the five primary processes within any creative testing are crucial for the seamless functioning of every system. They include creative concepts, sourcing, post-production, creative iteration, and creative analytics. So let's look in detail at each of them.
Creative Testing Process: Ideate, Create, Test, and Analyze
One of the essential steps to paid social excellence is figuring out a process that constantly outputs many creative concepts for creative testing. Why is it s important, though? There are several factors contributing to it.
First, the digital marketing ads market is highly competitive for a limited resource - human attention. Every advertiser tries to attract the user's attention to convey a specific message. On paid social platforms, you compete for placements with your direct competitors and sharks of e-commerce like Amazon or Apple. And with so many ads appearing daily on feeds, it doesn't become any easier. Therefore, an excellent creative concept that is relevant and attracts attention is a crucial success factor for your digital marketing strategy. Today, it's probably the only way to win over competition and user's attention.
Second, if you ever had successful creative running in your Scaling campaign, you probably noticed that it brings worse results with time. Creative fatigue is real. It's happening when Facebook users have seen your ad too many times and ultimately stopped reacting to it. Creative fatigue might affect your ads as early as a week after release, depending on the target audience size. Therefore, you must produce diverse, well-performing creative assets to sustain good results to avoid creative saturation.
To succeed, you have to plan creative concept brainstorming meetings regularly and appoint a responsible person. It could be someone from a creative team or digital marketing. You can base it on creative insights from the previous tests or audience insights from market research or product. Having an overarching direction is helpful, as there are endless directions you might take.
The key success factor is to make it interesting for everyone participating in creative concepting. We recommend starting with some ice-breakers and creative exercises before proper brainstorming. Also, ensure that everyone who is participating in this activity comes from different backgrounds and business units: from a creative team, marketing, or even product. The more diverse your team is, the better it is. This way, you increase your chances of getting unique out-of-box creative testing ideas.
It needs to be a regular cycle, and all teams must work towards the same performance and brand KPIs (for example, the previously mentioned creative hit rate). Here's an example of how a weekly cycle can be structured. Of course, it needs to be modified depending on needs, resources, and an overall marketing strategy.
Creative Testing: organizing the workflow
Where do you get the material for your ad creatives from? Do you have an in-house production team? Do you hire a creative agency? Sourcing is an essential part of the creative testing system. You'd better be sure that you get enough content to create ads from.
When we think about creative sourcing best practices, what comes to mind is a large, expensive production on set with multiple teams involved. That could be one of the ways to produce content, of course, though it's not the only one. Typically large production requires a big budget and effort and cannot be repeated very often. In addition, with short attention spans, fast scrolling, and engaging content everywhere, high-quality material alone won't do the job in the world of digital marketing. Indeed, you can create top-performing ads without super high-quality production. Some of the most engaging ad campaigns use the simplest footage & texts in their ads!
Creative Testing: Concepts & Sources Tips
Some inspiration ideas if you don't have the budget for total production or want to try something entirely different for your Facebook ad campaigns:
Whatever approach you choose, you need to source content regularly. The best way to ensure productivity in this step is to assign a responsible creative team member, as it might take some time and effort to coordinate all the stakeholders or, for the bigger projects, even a project manager.
Now that you have the content let's talk about transforming it into paid social ads. Different advertising platforms have various requirements and ad formats, and you must ensure they fit them.
We've seen that most companies have post-production to some degree, but the process doesn't stop with hiring talented graphic designers and video editors. Having the right talent is the crucial part. However, you also need to organize the workflow and ensure that your creative teams spend most of their time working on the social ads and not clarifying the briefs.
3 tips for working with designers
We recommend evaluating and structuring the following processes as a part of the overall creative testing systematization:
How to submit the right brief for creative testing
After you run a creative test, it's time to evaluate the results. It would be best if you had a performance review regularly to know whether you are reaching the goals and to set the directions for future brainstorming sessions and content productions.
A well-done in-depth analysis of a creative can say a lot about your audience and define the improvement areas. So we created this ad diagnostic overview to help you focus on those creative elements with the biggest potential for overall performance improvement.
If you want to automate creative analysis, Kitchn has a great solution for Facebook, TikTok, and Snapchat creative reporting. It's based on Gsheets, meaning that everyone can use it!
KPIs to watch: CTR, CPC, quality rank, cost per view, 2 sec. views/total views (for videos)
KPIs: Engagement rate, cost per engagement, engagement rank, # of engagements, and intent rate (the event between your main goal and click divided by clicks, f.e. add to cart/clicks)
KPIs: ROAS, CPR(cost per result), Result Rate (Results/Impressions), CVR, # of results
Ad diagnostics: how to improve your creative?
This is another often overlooked process, although it could be a fast way to improve performance on paid social channels. You probably have a couple of ads that are getting you decent results, but not the best ones. Make it perform better with some iterations!
You might not have yet found the ad that brings your dream results, but you've had some tests and defined several messages that resonate the most with your audience if you have ads performing “okay” but significantly better than the others. Go through the “okay” creative regularity (every 2-3 months) and evaluate the potential of it becoming a top performing ad.
Some social ads might get impressive CTRs but have low conversion rates to generate good results. Time to add some CTAs and focus on the product a bit earlier if it's a video. Other ads might need a change of message. If you recently tested out a new messaging and it works better, why not give a second life to the older creative with a fresh copy?
Cutting footage and combining different parts with fresher CTAs and messages can result in your new best performers if you do it regularly and give it priority. Then, test the new variations against the old ones and look at the KPIs. You will understand very soon whether you are improving or not.
Creative Iterations: tips
There are a few other components to consider when building your system. But, for that, we have to zoom out and look at two other contributors: seasonality and all other external factors that can't be planned for.
Let's look at seasonality first. We know that almost all products are at certain times more on-demand than at other times. For example, winter coats are typically not sold in spring and early summer. Ice cream instead has its demand increase precisely contrary. As marketers, we do not necessarily care about when the products of others are in demand, but we also do care about when CPMs are high, and it is more difficult to sell. We can expect rising CPMs in the build-up to black week and peak during the week itself.
This means that when we decide in what months to allocate the budget, we should plan out a full year ahead. For example, can we achieve our ROAS goal in the black week with a double CPM? When competition is low, could we spend more money in our typical “off-season”? Should we stop ads completely in months where returns are historically low?
The second part to consider is other external factors we can not plan for but prepare ourselves for. For example, has an ad ever stopped suddenly without any warning? Do competitors steal your ads, and suddenly ads fatigue faster? Or did somebody set the wrong ad destination and money went down the drain?
For this, we can only set up systems that notify us once something seems off. Of course, to be able to do that, we first need to define what is normal and what we expect to see. We recommend writing down all important KPIs that you historically can expect and building out processes to frequently check those KPIs to understand immediately when something is off.
These expected KPIs can include:
And of course, you can also set up simple rules to check for:
In this section:
Can you describe precisely how your creative testing process is done in detail? In fact, do you even have a process? Or are you just randomly running split testing without a proper strategy? If you don't put in a process and it's not measurable, then the chances of getting better results are statistically very unlikely. So let's evaluate your current strategy and give insight into which steps to build upon and which steps you are missing.
We've created this table to help you understand where you stand and find the right areas. Then, by focusing on them, you can achieve the fastest breakthrough.
Type caption (optional)
You are just starting creative testing and haven't had a successful ad until now. You are in a great position to start structuring things immediately and develop a proper creative strategy. Everyone starts somewhere, and it's OK to make mistakes figuring out how you want your creative testing process to look. However, there are two crucial processes for you to get right from the beginning: creative concepts and content sourcing. (You can learn about structuring these processes in more detail in the previous chapter).
When you just start creating social ads, we recommend you focus on collecting as much data as possible. You probably don't have many insights yet, and your main job is generating them. Focus on creative concepts - choose three entirely different creative concepts to test. Ideally, get enough budget to do a /b testing or split testing. At this stage, you want to be confident and see from the statistical significance that the concept you are moving forward with is working well for you. The cost of moving in the wrong direction can get very high. From the first tests, you will learn which visuals, messages, and ad formats work the best for your audience and can invest more in exploring the winning concept.
Another excellent starting point is experimenting with content sources. When you start creative testing, big production requiring a large budget is not something you can confidently agree to. Therefore, smaller content productions and sourcing content from user-generated platforms come in handy. It can also serve as a prototype for large-scale content productions in the following steps.
You have probably tested some concepts at this stage, and some might have even produced results, but it's still too far away from the desired goals. Do not lose hope - there are tons of creative variations to try! Slowly but steadily, continue with creative concepts and content sourcing, but add one more step - analyze data from all the past tests, too. (You can check out our ad diagnostics guide to help you understand which metrics you should be optimizing). Data-driven ad creatives are a much faster way to top performing ads!
In addition, at this stage, it's essential to understand how your creative performs against the competition. We recommend you take a look at the estimated actions ranking.
While it's clear to advertisers that conversion rates should be consistently strong and CTR high, in many cases, engagement rate is being ignored. There is no evident impact on the business, right? Who cares about the likes and comments under an ad?
The answer is - the Facebook delivery algorithm does. And to justify adding engagement rates and rankings to your dashboard, let's look at how Facebook describes the delivery system.
Type caption (optional)
The first component is Advertiser Bid. This is relatively simple: the more you are willing to pay per result, the higher your chances of appearing in the user's feed or story.
The second component is called Estimated Action Rates. The delivery algorithm will probably consider thousands of different rates, which we don't know about. What they do tell us, though, is the Conversion, Quality, and Engagement Rank. Those rankings will determine how much you will pay per conversion. Otherwise, why don't you ever get a cost per conversion that equals your bid? Looking at the "formula" below, it's fair to assume that the higher your estimated action rates, the lower your bid can be to get ad delivery.
If you have ever looked at the Facebook Investor Reports, you probably noticed that the main KPIs they report are "MAU" and "DAU" (monthly/daily active users). This is because Facebook only exists if people use it. Therefore, showing relevant ads to relevant users follows the company's business logic.
We've recently noticed that ads with high engagement rates across many different advertisers tend to get more delivery, perform better and generate a cheaper cost per result. Even if the comments below are horrifying.
If you are at this stage, you probably have a couple of good-performing creative assets that produce significantly better results than the rest. However, those ads might not be enough to achieve the desired spending and scale. Therefore, this is an excellent opportunity to explore this direction further, and we recommend you focus on two processes: creative iteration and creative analytics.
Having some social ads that show much better results than the rest is already a huge step towards success. Now, it's vital to figure out why those ads work better than others and try to develop a “formula” for an ad that works. If you use videos, one of the methods to do it will be to change an important ad element at once to understand how it affects performance. This way, you can evaluate creative elements driving the performance and build your future concepts around them. Try experimenting with:
Ad Creative Fatigue: Graph showing a rising cost per result as # of results gets higher
You've reached creative fatigue when your best-performing social ads start getting higher costs and lower CTRs. It can happen within two weeks or earlier if you target smaller audiences. There is certainly no way around it, and you will have to come up with fresh concepts (actually, it's one of the reasons effective creative testing is so important - you will constantly need new creative assets to boost performance). Therefore, focus on split testing or a /b testing out new concepts.
One difference between focusing on creative concepts earlier in the beginning and after creative fatigue is the amount of data you gathered up until creative fatigue. Collect and structure all the available insights to help you choose a direction for new concepts. Compare the worst-performing ads with the top performing ones, sort your data by different KPIs, and analyze the ad formats and placements. Those insights tell you a lot about your target audience and inspire further action.
If you regularly produce the best-performing creative assets on the scale - congratulations! You've built a creative testing machine that works for you. Now it's time to think about standardizing all processes within a system and automating the time-consuming, non-value tasks. In other words, exploiting the existing system. For example, you can think of creating ads in bulk, automating the winning ads' rollout, or implementing a quality control system to ensure your ads are delivering. In the following chapters, you will learn more about process standardization as the first step toward a fully automated creative testing system within your marketing strategy.
Paid Social: Expectations vs Reality
There is another concept we want to introduce to help you plan your budget and focus throughout the year. On a universal level, we call it “**exploitation vs. exploration.**” In marketing, this, of course, means when you should be focusing on creative testing new things and when you should be focusing on generating the highest revenue.
Let's look at the concept more broadly before applying what this could mean for us as marketers.
Every innovative company or business unit faces the challenge of balancing exploration vs. exploitation. While exploitation builds on existing assets and aims to improve them through innovation, exploration requires companies to "leave their comfort zone" and go to places they have not been before.
In digital marketing, an ever-changing field with millions of competitors bidding for the same spot and a vast amount of unknown externals, this is a constant battle between optimization and scaling of current high-performing ad sets while ensuring the backlog of new angles, creative assets, and channels is never empty.
The problem, though, is that the biggest success in both areas is achieved when focusing on one thing only for a prolonged time and digging deep. But how to do it when you constantly have to switch between both?
Our answer to this is: Build systems that do the job for you, so the job gets done, even when you don't focus on them.
So the answer to the "when" is built on the definition of exploitation and exploration. When you have built an asset in your company, it is time to build a system that keeps exploiting! In digital marketing, one of the most important systems needs to be built when there is a repeated process of testing ad creative and rolling out a winning ad. When you have done it repeatedly with similar success, it is time to build a system that does the job for you. This certainly does not mean running it on complete "auto-pilot," but implementing a system that does this job for you.
It will probably take longer than expected to have it up and running, but once done, it will pay you dividends for a long time. Then, it is time to go into exploration mode again and find new fields for innovation. Over time, balancing smartly between exploration and exploitation will leave you with superb and robust systems that always get the foundational work done.
And the current uncertain times in online marketing showcase another upside. When there is a significant shift in the market - like it is now with the announced changes on iOS 14.5 and Google Tracking, you'll have the capacity to dig deep into the implications and come out as a winner.
We hope that our Creative Testing Blueprint has given you the building blocks to implement a successful creative testing system in your workflow—the benefits you can expect from using such an approach, where to start, and how to do it. If you put these ideas into practice and stay patient with the process, you will certainly get one step closer to producing the best-performing creative assets at scale.
At kitchn, we believe that creating a clear structure is the first step to anything. A truly effective structure will make you more efficient by helping you eliminate time-consuming tasks and streamline your workflow. If you need help with automating manual tasks, contact us. Learn how Kitchn.io can improve your processes if you are a D2C brand or a marketing agency.