As you may know, I am a fanatic about separating search partners and Bing/Yahoo traffic in adCenter ad groups. We've had success with search partners, particularly for financial services clients, but they need to be monitored very carefully as there can and has been fraud on Bing and Yahoo search partners for years. Fraud can take two forms - individual bad partners (my colleague Bonnie wrote about this with Yahoo) or widespread fraud by many partners. In this situation, unfortunately, I'm seeing both. There is a lot of research on this topic; PPC Hero wrote a good article in December about this.
Around Thanksgiving, one of my clients began seeing a huge spike in search partner conversions. While the conversion rate rose for search partners, the Bing & Yahoo conversion rate stayed pretty much the same. This was concerning.
I didn't want to be the wet blanket. Is it possible that MSN added great new sites to their network and this is legitimate? Of course, but I needed to do several things to be sure I wasn't wasting money.
#1 - Find the Big Fraudsters in Publisher Reports
Start with the basics. Run publisher reports to identify who the converting search partners are. In this instance, one Search Partners placement had 14 clicks and 14 conversions, yielding a conversion rate of 100%, while Bing & Yahoo converted at 2.20%. While Bing and Yahoo can convert at very different rates from search partners, this big of a discrepancy is odd. Even if a spammy site could convert at 50% (over 20x what Bing and Yahoo do) there is much less than a 1% chance (0.5^14=0.00006) that all 14 clicks will convert. I actually got an anti-virus warning when I tried to visit one of these domains. It's safe to say that conversions from those sites are fraudulent.
#2 - Find the Little Fraudsters in Publisher Reports
As adCenter professionals know, a big chunk of publisher activity may be on sites with only one or two total clicks. If you're advertising for multiple products and you don't want to pivot ad group or campaign data to get more performance data on individual publishers, it gets even worse. Individual clicks can't be analyzed like the bigger partners; however, you can look at aggregate small volume site performance.
For this client, I had 964 placements with only one click; 237 of them converted (24.58%). For a large number of very small search partner sites to convert at over 10x the rate of Bing and Yahoo is "strange." What makes this even more odd is that the biggest search partner sites for this particular client (keywordblocks, financecity) were converting at rates similar to Bing and Yahoo search properties (3.67% and 3.52% for keywordblocks and financecity respectively).
Hundreds of small search partner sites were likely recording fraudulent conversions, but the biggest search partner sites were likely not doing this. Why was this happening? I think Google Analytics data shows how smaller sites could engage in widespread over-reporting of conversions.
# 3 Check Analytics Conversion Specific Data
This client has three website conversions, all of which get volume (there is a fairly equitable mix between them). The Bing pixel can record conversions in multiple ways (Clickable does a good job describing the different MSN tracking methods - one per click and count all for the pixel). However, the Bing pixel can't distinguish between conversion types. If we had 100 total conversions, it couldn't tell us that 30 were for goal one, 40 were for goal two and 30 for goal three. This can be a problem if computer program finds a way to exploit one conversion type.
SEER always sets up Google Analytics (GA) goals in addition to the AdWords, adCenter and other engine pixels. GA goal conversion numbers won't match the Bing pixel numbers, but GA gives us a second source to compare conversion trends and we can get the breakdown of conversion types from GA. When MSN search partner conversions spiked, a large, disproportionate number of the new conversions were recorded in GA as goal type one. As you can see, goal one's conversion rate mirrors the MSN search partner conversion rate as measured by the Bing pixel.
In this case, for technical reasons as to how the site records goal one (the client's IT team can't explain how and when goal one's conversion page loads), we believe goal one could be getting exploited by computer programs aiming to maximize the number of conversions for search partner sites. It's easy to block publishers in MSN. If search partners don't convert, advertisers will drop them. My intuition is that there are unscrupulous websites looking to game advertisers by making it look like they have lots of conversions, so they won't be cut from search partners. They found a weak goal conversion page and tried to exploit it, but fortunately we noticed them.
What have we done about it? We've paused search partners, informed the client, opened a click fraud investigation and I'm trying to get form data to see if the inputted fields are legitimate. The last part is hard slogging. I'm not sure if the system the client has will be able to associate form information with MSN campaigns and referring websites. I could then cross reference referrers to publisher report data. We'll see how this finishes up; the click fraud investigation isn't done yet.
What are your best tactics for monitoring Microsoft search partners? Feel free to add comments below.
