This was an unofficial comparison study, here's what the vendors would like you to know.
As this was an unofficial study run by the Publicis Groupe Social Media Intelligence Unit, we wanted to give the vendors an opportunity to comment on the results. Here's what they had to say...
Crimson Hexagon ForSight and Brandwatch Analytics have now combined to create Brandwatch Consumer Research and there are some significant improvements as a result of the merged functionality.
The combination of machine learning and rules is a huge advancement. Just one method to segment data only takes you so far. Once rules and machine learning classifiers are set up they are invaluable for polishing your data, especially sentiment (e.g. classifying hard to define segments like brand values, customer journey, intent to purchase, etc.). We believe that AI needs a guiding human hand and analysts need to be able to intervene when it’s important. With Consumer Research analysts finally, have both tools easily to hand.
The segmentation available within the new platform is also far more advanced. Users can go beyond analyzing sentiment at a topic level and dig deeper to see how sentiment is spread across every category set up. That’s where sentiment becomes really useful (e.g. in the example of a beer brand analyzing its products - what is it about IPAs that make people positive/negative? Is it the flavor? Is it the culture? Is it the price?).
In addition to industry-leading segmentation, Consumer Research users also benefit from the world's largest pool of social data and the smartest AI available to instantly analyze your data.
Talkwalker is known to have one of the most complete coverages on the market, including data from the Twitter firehose - the most complete Twitter connection possible. Benchmarks are never an easy task, given different parameterization and behavior of the different platforms. We are nonetheless always open to guide our prospects and clients in a fair assessment of the capabilities of our platform.
We know that data analytics is about more than just volume and sentiment. That’s why we have a detailed use case framework to support clients on their conversational data journey, to provide more actionable insights for their brand.
Pierre Detry, VP Product Management
As the authors note, this research was performed on an earlier version of our social listening platform. Since acquiring Sysomos in 2018, Meltwater Social has introduced a significantly improved social listening offering, with a brand new user interface and even more data, including news coverage from our media intelligence platform. This enables users to analyse social media mentions alongside conventional news coverage to better understand how the two are related.
It is great that the SI Lab is facilitating more and more discussions about various aspects of SML technologies. Such publications are an excellent source of information and food for thought for the social intelligence community and vendors!
As for the sentiment, indeed, there are different approaches to balancing the detection accuracy. In other words, it is impossible to create a perfect sentiment detection model which will be optimal for all use cases.
For some businesses, it is essential to be able not to miss a single negative mention (maximize recall for negative sentiment); others want to be able to consistently track sentiment trends about their brand (balance precision and recall for positive and negative sentiment), and so on.
Moreover, each model depends on data on which it was trained, and that influences for which domain the model will be best suited. So only testing on really diverse data can give more reliable results.
For this reason, a couple of years ago, we have entirely abandoned a rule-based sentiment detection approach in favor of a Deep Learning-powered, object-oriented one. It learns from users' feedback and automatically improves over time. And we keep innovating: recently, we have made our model detect post comments' sentiment better by understanding the original post context.
We also will be delivering significant improvements to our sentiment detection models shortly. They will be able to distinguish between different aspects (for example, quality, price, taste, and others) of an object being discussed, and assign aspect-specific sentiment to each of them. It will empower social media intelligence professionals to get even more value from automated sentiment detection.
It is also worth to note that the data collection requires a careful set up on the platform side. With so many options of including / excluding different regions, languages, sources, types of data, and various spam filtering rules, it is often difficult to compare the resulting datasets without a clear understanding of customer's analytical needs.
Alexey Orap, CEO