2024 has been the most exciting year of SEO in recent memory. I haven’t seen new opportunities and threats emerge this rapidly since the chaotic era of Penguin and Panda updates. The evolution of artificial intelligence (AI) has sown doubt and uncertainty for many. New platforms for consumer search are turning heads and raising questions. And of course, all this has occurred amidst some serious shake ups with core algorithm updates over the course of the year.
I can imagine the decision paralysis I would be facing if I were a team of one, or even on a small marketing team. Where does one focus?
- Training your department or organization on how to safely and effectively use artificial intelligence is an important investment to make. Right?
- Not so fast. Individuals can’t learn tools that they don’t have access to, which requires capital investment and opens questions around ROI. Okay, surely picking a tool is simple…
- Or is it? OpenAI emerged as an early enterprise favorite, but Google Gemini is quickly catching up. Meanwhile, Anthropic’s Claude has emerged as a great option for enterprise.
- While you’re making a decision akin to Bounty vs Brawny paper towels, you’re likely being asked to track what’s up with these new AI Overviews everyone is talking about? Unfortunately Google won’t supply us data correctly, so we’re heads down once again to figure out how to track all of this.
- Meanwhile, your boss’s boss’s boss thinks your audience is searching for your services on ChatGPT more than Google and wants to know how we optimize over there.
Exciting? Yes! Exhausting? Also yes! Beyond solidarity, I’d like to offer some insights from our small but mighty AI Council. Seer doesn’t have the same R&D firepower that some venture backed organizations tout, but what we lack in funding we make up for with a borderline unhealthy desire to win for our team and our clients. To learn more about what drives us, check out our most recent impact report.
I anticipate what follows to be a series of posts documenting the ‘field notes’ from our AI Council. What have we seen? What have we done? What do we think is coming? We’ll share our notes in the hopes that even more organizations will share those. Let’s push beyond LinkedIn clickbait and get real about what is an opportunity today, in Q3 of 2024, and when we should apply the ‘Wait Calculation.’
Follow along in this first set of field notes as we outline what we’ve learned from our Disruption Analysis, and how we’ve made progress on this initiative over the past six months.
What is a Disruption Analysis?
A disruption analysis is an A-Z review of all of the work your organization or department executes with each individual deliverable, workflow, or process scored against the same rubric:
- How generative is the output, focusing (for now) on text generation
- How predictive is the output
- How data driven is the output
- How repetitive is the process
The Marketing AI Institute can be credited with the concept, as well as the initial guidance in how to set up this effort.
At Seer, we put our own spin on it from there. We conducted this analysis across the ~170 deliverables and workflows the team is regularly scoped to accomplish. Additional data points collected include:
- How much revenue is associated with the deliverables and how many were sold in the last 12 months?
- This indicates business priority
- How much time was spent on these deliverables in the last 12 months?
- This indicates which are the greatest opportunities to drive value for the team
- How many process steps are involved in each deliverable?
- This indicates complexity, and often when we are thinking of augmenting our process with AI we are attempting to make a 1:1 replacement to ensure ease for the team
- What is the business priority of each deliverable and workflow?
- Some work is more meaningful than others. Some work is already in the process of evolving in other ways. Alignment with business leaders was crucial
We scored all of the above in order to arrive on a prioritized list of opportunities.
Perhaps one question this leads to is “Who should be responsible for the disruption analysis?” and to answer that, we’ll cover more details of how we have created our AI Council in a further post.
Balancing the Risks of Getting It Wrong with the Upside of Getting it Right
Once we had our full list of opportunities, we had further decisions to make. We are but a small team, and starting at #1 on the list of 170 opportunities didn’t make much sense. Picture this: We spend weeks working on that opportunity. Best case scenario we get it right and release one thing. Worst case scenario we get it wrong and have nothing to show for it.
Further, does the broader team have the stamina to witness one-by-one change, every other week, for the next several years? Even if we were able to update these deliverables one per week (an unrealistic target to be sure when you factor in R&D, validation testing, user acceptance testing, change management, and impact measurement), the most we could get through is about 50 per year.
If you know Wil, do me a favor and think about what his reaction might be if you told him you’d have AI integrated at Seer by ~2030 at the earliest. To put it mildly, even the best case scenario is unpalatable.
Identifying single steps found across many processes
Since a ~6 year evolution won’t do it for us, we had to find another way. This brings us to where we are today. Seer’s AI Council has identified 3 ‘base level GPTs’ that can impact over 80% of our deliverables at Seer. Before I get to those base level GPTs, I want to ensure it’s clear how you can follow this approach.
By identifying a few opportunities that impact many workflows, we ensure we are thinking about scaled solutions. We may not get it right, but if we do, the impact is immense. Equally important, the ability to implement change management is much easier. We can teach the team how to do one thing and instruct them to do it across 60 different deliverables. Sounds better than learning 60 new processes, right?
Across your organization there are likely dozens of things you and your team do. They’re varied, they’re unique, and presumably they are valuable. Once you take inventory of these opportunities and identify those that are potential opportunities for AI integration, it’s critical to break down the steps taken within these workflows. Hopefully you already have this mapped and documented as part of your operations process. If not, it’s a worthwhile activity to do so.
Prioritizing the right areas of focus for your organization
Once you’ve got all of those process steps documented, you’ll likely immediately start to see some themes emerge. Your next step is to identify a short list of themes you can prioritize impacting. The data you have collected should guide you, but ultimately your recommendation should be made by identifying your most critical need. Here are a few examples of what that might look like.
- Team Buy-in: Perhaps your team is *not* bought in on the merits of AI, and reluctant to change. In this instance I would prioritize initiatives that will help them see the light. It needs to click for each individual. Yes, there are all kinds of things AI gets wrong. But there’s power there, and you need to see it to believe it.
- Financial Performance: Maybe you’re getting squeezed on pricing and you need to make some big changes, now. In that case, I would focus on opportunities that you feel most confident will have a big impact on efficiency. Review your tasks and themes and create some hypotheses.
- Innovation: Lastly, maybe you’re an organization trying to disrupt an industry. You need to make some big bets and create something that will turn heads. It’s high risk and high reward, but the potential for domain experts to innovate has never been greater.
At Seer, we’re in a fortunate position. Our team is bought in, our financial performance is strong, and we continue to innovate rapidly. Our approach models one we observed from an unlikely source: Moderna.
In this case study, Moderna shares how they were able to integrate AI into all parts of the business and drive innovation from within. They had me at ‘drive innovation from within.’ The team at Seer is brilliant, and we know if we can upskill them effectively they will build the right solutions for the right problems.
As such, our approach to AI integration at Seer is guided primarily by a desire to help the team evolve into next gen marketers and, in doing so, ensure we’re delivering next gen performance to our clients.
Ensuring we are starting a fire, not filling a bucket
This concept was borrowed directly from Moderna’s case study. We don’t want to build a system that has the team living hand-to-mouth with change management. In other words, we can’t just position a small team to build everything. There is immense nuance in performance marketing across different industries, businesses, and marketing strategies. There will be the need for customization.
This is another advantage of approaching the problem with base-level GPTs. We are basically building open source frameworks for AI workflows (in this case, largely custom GPTs). This means the team can take what we release and customize it as they see fit. Perhaps there is a significant amount of nuance requested for a certain client’s content review. No worries, the team can start with our base-level build and add on to it.
This also helps the team use AI solutions with some training wheels. Directly prompting an LLM like ChatGPT is extremely powerful and useful. And, it can be a bit intimidating. It’s easy to go down the wrong path and get an output that isn’t helpful at all. We want to help our team avoid that as much as possible.
A digital marketer’s consideration set for base-level GPTs
With all of that background out of the way, let’s get into the base-level GPTs we have identified as opportunities at Seer. Remember, what we landed on were three base-level GPTs that will impact 80% of our deliverables at Seer. The work certainly doesn’t stop there. We don’t want to lightly insert AI into our processes, we want to transform the way we work.
Even more important to transforming the way we work is keeping our client’s data safe and secure. While we feel confident in the securities provided by our tool providers, we understand some organizations do not. To that end, we are focused on building solutions that work for all clients, both those that allow the use of AI with their nonpublic data and those that do not.
Lastly, we know due to client nuance there will be some differences between how this looks on paper vs in practice. That said, we’re targeting cultural adoption and incremental change. And not for nothing, the Wait Calculation continues to be on our minds.
Before I highlight the base-level GPTs we are pursuing, I thought it would be helpful to share the full consideration set we reviewed. We still plan to attempt to build all of these, and if others in the industry have built examples of these we would love to see what you’ve done.
Internal Reviews & Revisions
At Seer we strive to deliver fully polished, QAed, impactful work to our clients. It’s important, and it’s also very time consuming. One of our most frequently executed steps includes an individual getting feedback from their manager or project lead. There are many different flavors of this.
- Analysis Pause Point for Lead Review: We have all been there. Heads down in a time consuming analysis, moving from clean data to organized data to visualized data to insights, hypotheses and recommendations. You deliver your final product, a set of slides, with pride. You’re aghast to find that you made an error in Step 2 and all of your findings are null and void. Brutal. This step helps us avoid that issue at Seer.
- QA Checklist: Many of our core deliverables have Acceptance Criteria associated with them. We strive to find the balance between identifying the objective issues that can impact almost any work, while not smothering creativity and the need to deliver unique solutions.
- R1 / R2 Revisions: Even with a rigorous review, we almost always anticipate some Client feedback. Perhaps business priorities have shifted. Perhaps brand tone guidelines have evolved. Perhaps we just missed the mark the first time. In any event, revisions are a part of the life of the marketer.
Presentations
We make a lot of presentations at Seer. Our ideal partnership is one that is agile and fluid, focusing on work that moves the needle. And, stakeholder buy-in happens to be one of the most effective ways to move the needle. Stakeholders like decks. Such is life! Similar to reviews, there are many elements at play.
- Create the narrative: Every time we create a slidedeck we have the opportunity to build something effective, compelling, and influential. It’s surprisingly easy to attempt to do that, and instead wind up with something boring, confusing, and meaningless. The narrative is the key element that distinguishes between the two.
- QA the slides: Slides must first in the grand scheme of the narrative, but also must be able to stand alone. Ensuring the data is accurate, the content is right, and the aesthetic is on brand is important.
- Identify the potential objections: If we’re trying to convince someone to do something such as make an investment or implement a difficult initiative, we must be able to identify
Data Manipulation
Data manipulation happens across all roles and teams at Seer. It’s par for the course when being data driven is part of your ethos. This takes many shapes, and ultimately AI is not necessarily the best tool for the job. That said, we are finding that AI can often help us identify what the right tool is.
- Pull Data: For reports, analysis, and optimizations alike, we do a lot of data pulling at Seer. Luckily, much of our data is housed in Seer Signals which makes things easier. That said, we prioritize excellence over ease. For every “easy” export, there is an opportunity to locate and dig into a new data source.
- Clean Data: You simply can’t be a data driven organization and skip this step. It’s incredibly important, incredibly time consuming, and incredibly boring.
- Organize Data: As my colleague Christina Blake often points out to us, cleaning and organizing are two very distinct steps. The act of cleaning data is important to ensure you’re working with an optimized data set. It’s objective. The act of organizing data is subject to the context of your use case for the data. It can be both objective and subjective.
Analysis
Analysis is the engine that creates the insights and recommendations. It’s critical, it’s scientific, and it’s fun. If you ask folks on the Seer team what they’re favorite part of their job is, many will tell you it’s the ability to perform analysis and identify insights. But, those same people also fast follow with some opportunities where we can improve. There are two big opportunities for us here.
- Build Multi-Format Audits: Across all divisions at Seer, we create a lot of audits. Often these are built with some standard criteria in mind + some custom evaluation criteria. Our analysts work dutifully on reviewing the relevant data sources and compiling insights, evaluations, and recommendations. Then, they proudly deliver a spreadsheet to our project stakeholders. High fives commence! But the job isn’t done yet.
Next, they often must translate it all to be a more friendly document with deeper instructions for the person or team who will actually implement it. There are Google Apps Scripts that can help with this, but only to a certain extent.
Sometimes, they also must translate a third time (this time in Slides!) for the stakeholders who must approve the time spent by the folks doing the work. Efficient? Not really, but we didn’t create corporate bureaucracy, we just adapted to it. - Forecasting: We forecast and project a lot at Seer. We forecast potential impact for our recommendations. We forecast our ability to hit KPIs based on assumptions. We forecast the implications of increasing or decreasing ad spend. It’s one of the biggest opportunities to build trust by ensuring we are speaking the same language as our clients. It’s also one of the biggest opportunities to lose trust if we get it wrong.
Insight & Recommendation Generation
Data isn’t worth much unless you can do something with it. Ultimately our clients look to Seer to help them make the best possible bets with their resources and funds. We need to be able to move from analysis and research to insights and recommendations. This has many shapes and forms. This is a small sampling of our opportunities at Seer.
- Create a State of Search: Many of our clients pay for an additional element to their monthly or quarterly reports. A ‘State of Search’ section highlights the recent happenings in the industry, specifically identifying the news most relevant to their business, marketing strategy, or industry.
- Create Project Brief: Perhaps by now you see we take operations seriously at Seer. One of the best ways to execute an output seamlessly and efficiently is to ensure every stakeholder is aligned with the process, the inputs, the desired output, and the ultimate goal. Templates exist, but of course customization is required.
- Create Action Items: In any meeting or presentation there are bound to be action items. It’s a simple process. Create the list of what we said we would do and what others on the call said they would do. Organize the list and include it in our recap
Seer’s Priority Base-Level GPTs
Without further ado, I can share the base-level GPTs we are prioritizing in Q3 at Seer: Presentation Narratives, Deliverable Reviews, and Forecasting.
Beyond all of the rationale I included above, here are a few more reasons these three opportunities made our priority list.
- General Purpose Technology to the max. These three opportunities represent three very different ways we can use this technology. The Presentation Narrative GPT is a thought partner, helping an individual craft the right outline for the story they’re trying to tell. The Deliverable Review GPT leverages LLMs’ ability to reference and review content. Lastly, the Forecasting GPT will tap into our ability to perform data analysis and create/manipulate code.
- Fail fast and move on. The inverse angle to those advantages is that we’ll quickly know which concepts have legs and which don’t. For example, we’re hitting some major speed bumps with deliverable reviews. Our R&D builds are hallucinating at a concerning rate, forcing us to think about the problem differently. This is all part of the process.
- Accessibility for the win. Remember our core priority at Seer. We are enabling next-gen marketers that can drive next-gen performance for our clients. Our goal is to light a fire, not fill a bucket. These base-level GPTs are accessible for all skill levels and will serve as a great training ground for building confidence and improving the team’s technical skills.
- Buying time for the Wait Calculation. I’m a risk-averse person. That’s made this year especially challenging for me. I have a really hard time stomaching the idea of our team spending hours and hours to solve a problem that may be achievable ‘out of the box’ in the next evolution of LLMs. As such, the initiatives we are focused on are wide-ranging and varied enough that we’re learning a truly staggering amount with every new research project and MVP. Even if 3 months from now all of these base-level GPTs can be accomplished natively with ChatGPT 5, we should still be able to consider the time spent productive.
Where do we go from here?
I was most inspired to write this for those working on small teams charged with big initiatives. At Seer we have so much going for us. We have top-down buy-in on all things AI. We have budget allocated for R&D. We have (in my opinion) the smartest team in the industry. And it’s still hard for us. It’s still easy to second guess the path we’re on is the right one, and we’re investing the right amount of time and resources into the right initiatives.
If you work within an in-house marketing team you likely have far less resources and support to navigate all of this. It’s a great time to have a partnership in place that can take some of the onus of research, training, and change management off of your plates. Or, perhaps your team is on the cutting edge but you need support getting buy-in from stakeholders or other internal departments. Ultimately there is a lot of opportunity, which means there is a lot of work to be done!
More than anything we offer solidarity, and a call for more collaboration. And if you want to be a part of what we are building at Seer, keep your eyes on our careers page for open roles.
