Optimize 360 vs Optimizely: Which is Right for Enterprise Testing?

So, you've been using a free A/B testing and personalization tool and now you’re ready to put your money where your mouth is ...

A/B testing has long been a critical piece of effectively optimizing your website -- while personalization has proven to be both the present and future of what users expect on high quality sites (just ask Amazon or Netflix).

Two of the top platform options out there for digital marketers and those managing websites are Google Optimize 360 and Optimizely. 

In this post, we'll dive into the key similarities and differences between Optimize 360 vs Optimizely in each of the following categories:

• Content Experiment Types
• Audience Targeting
• Test Variation Setup
• Conversion Setup
• Reporting
• Pricing

For a full comparison chart, skip to this section of the post.

Optimize 360 vs Optimizely

Comparison: Content Experiment Types

Similarities:

• Support several types of content experiments, including A/B testing, split URL, and multivariate testing.
• Both tools support site personalization.
• Both tools allow for visual page editing (making it easy for anyone to customize test variations and experiments without coding!)
• Both are relatively user-friendly but Optimize 360 is easier to learn if you're already familiar with other Google marketing products.

Differences:

• Optimizely offers what they call the “Multi-Armed Bandit”, which aims to boost performance across variations by dynamically allocating traffic to the variation where the primary metric is doing best. This experiment type ignores statistical significance in favor of maximizing the defined goal.
• Optimize 360 offers the easy addition of banner templates to any site.
• Optimize 360 allows for the ability to prioritize customized experiments in cases where users may qualify for more than one experiment.

Comparison: Audience Targeting

Similarities:

• Target by basic demographics, such as user type, age, geography, device type, time of day, language, etc.
• Target by campaigns, query parameter, Javascript variable, cookie, etc.

Differences:

• Optimize 360 allows the importing of audiences created in Google Analytics and Google Ads, allowing for more precision in testing and personalization. Audiences are easily imported via the native integration.
• Optimize 360 can also leverage customer data from third-party sources outside of GA and Google Ads, but custom work is required.
• In Optimizely, users can create audiences and target known visitors with dynamic customer profiles that contain information from first- and third-party sources, allowing for even more precision depending on what data is imported. However, this requires more legwork, as the data import from these different platforms is not natively integrated.

Comparison: Test Variation Setup

Similarities:

• Both platforms allow changes to be made via the WYSIWYG editor, HTML, CSS, JavaScript, and jQuery selectors.
• Both provide a Visual Editor Page for the variant page where you can edit by simply clicking into elements like buttons, text, or headlines.

Differences:

• Optimizely has a more intuitive interface for completing more complex changes on test variations.
• While tracking specific user actions on a test page can be set up through Optimizely when creating an experiment, Optimize requires that this tracking be completed separately and in advance via a tag management system (think: Google Tag Manager).

Comparison: Conversion Setup

Similarities:

• Both platforms allow you to set up Experiment Objectives based on pageviews, clicks, and custom goals.

Differences:

• Optimize 360 allows you to easily set existing Google Analytics goals as Experiment Objectives once the two platforms are linked. It also allows for bounce rate and session duration to be set as objectives.
• Optimize 360 allows up to 10 objectives per experiment. You can have one primary objective, and numerous secondary objectives to boot.
• Optimizely allows for unlimited objectives per experiment. However, measuring too many objectives can often muddy your analysis of the test’s performance -- so be careful you don't set too many at once.

Comparison: Reporting

Similarities:

• Both platforms include most of the same information in their reports, such as the number of visitors that interacted with each page variation, the conversion rate for each page, which page performed the best, and the confidence level of the experimental result.

Differences:

• Optimize 360 uses the Bayesian method to calculate statistical significance in its tests, while Optimizely uses a custom frequentist method called Stats Engine.
• In addition to the standard reporting features, Optimizely allows marketers to filter through experimental results and segment them in different ways to learn more about how visitors responded to each page variation.
• Optimize 360 automatically imports test results into the Experiments report in GA, allowing you to combine test results with other GA data. A Data Studio template can also be set up to display test results in a visually appealing, shareable format.
• Optimizely has a 3rd party integration with GA that allows you to import the original and variant IDs into GA for pairing with your other website data. However, additional test performance metrics are not available.
• Optimizely has almost instant results, while Optimize 360 can have up to a 12-hour delay depending on the traffic you get to the pages you're testing on.

Comparison: Pricing

Similarities:

• Both platforms have a paid option geared toward enterprises (aka not cheap!)

Differences:

• Optimize has a less robust free version of the product that shares some features with Optimize 360. See here for the trade-offs.
• Optimize 360 can be bundled with other Google marketing products to reduce and share fees (i.e. Google Analytics 360).
• Optimizely is known to be notably more expensive than Optimize 360, and since it's a standalone option, there's no bundling to mitigate costs.

Which Enterprise Tool is Right for Your Testing Needs?

Full Comparison Chart

In Conclusion

In summary, both are excellent choices but may be better suited for different enterprise needs. 

• If you’re a Google shop already, Optimize 360 is likely the way to go, as its native integrations make it an easy win. It also offers the nice bonus of 3rd-party integration with popular heat mapping tool Hotjar. 
• However, Optimizely boasts more robust features within the platform itself. If you have a dedicated testing team already, they may prefer the more advanced sophistication of a standalone platform like Optimizely.

For more information to aid your decision-making, make sure you research both Optimize 360 and Optimizely.

Looking for an Analytics Partner to maximize Google Optimize?

Reach out to Seer today: