Last week, Harvard Business Review published a piece called “How to Keep Complaints From Spreading.” I have a complaint. Several, in fact.
My intent here is not (necessarily) to single out the HBR; I hold them in high regard. But the article epitomizes a systemic problem in the communications industry today: brands are susceptible to making flawed business decisions based upon misleading guidance or incomplete data. This problem has two culprits: one, some companies that traffic in the exaggerated benefits of Artificial Intelligence (AI), Natural Language Processing (NLP) and their behind-the-curtain data-stacks. And two: media who fall prey to said trafficking and don’t do their research and thereby exacerbate the problem by reporting on it. Brand managers are always on the lookout for the next magic pill and journalists rightly seek something new to report. And there are always individuals ready to sell you a dose. But, buyers beware of false solutions: A timely parallel: We do need to end the pandemic, but that doesn’t mean Hydroxycholorquine is the answer, no matter who touts its benefits, what someone’s wishful data suggests or how badly we need it.
The crux of the HBR piece is clear from the headline—it offers advice on how to stop criticisms from spreading online. What caught my eye immediately was the lede: a quick recap of the now infamous United Breaks Guitars crisis. United Airline’s brand crisis occurred in July, 2009 when a musician uploaded his music video to fledgling YouTube and expressed his frustration with United Airlines breaking his guitar in transport. In the parlance of 2020, this was Patient Zero for the viral spread of online criticisms there ever after. Yet, HBR chose to omit that historical timeline. Setting up the problem at hand with an 11-year-old case study is disappointing and extraordinarily confusing, if nothing else. That alone piqued my interest to look more deeply into what was going on.
The article is based upon one research study from the KEDGE business school in France. Dennis Herhausen, associate professor of marketing at KEDGE, sets the stage of the problem:
“Firms are using social platforms, but they don’t seem to have a coherent strategy for managing posts that convey customers’ concerns,” he says. “When someone puts up a complaint and other customers enter the conversation, a firestorm can erupt. We wanted to understand this new phenomenon.”
To be blunt, recommendations based upon this data will be inert at best. At worst, they will cause damage. Let me explain:
Strike One: 2011-2016. This study is recent only in that it was published recently (2019). But data that old is not relevant in today’s context. The fundamental challenges for brands today are: mis/disinformation, the loss of trust in media, and the mobilizations of online groups intending social harm and general disruption to business and society. To use antiquated data to make recommendations in the present day is like your doctor reviewing an x-ray from seven years ago to evaluate the leg you broke yesterday. This is analytics 101 and HBR surely knows better.
Strike Two: Facebook-only. Facebook is one data-set and, though popular, is the tip of the iceberg of online criticisms. While this was somewhat true throughout 2011-2016, it is empirically true today. Criticisms on Facebook often come after they’ve metastasized elsewhere, frequently in the deeper web on platforms such as 4chan, 8kun, Discord, Telegram, Gab, Reddit and scores of others. Trying to stop criticisms from spreading online by learning from Facebook-only is like trying to put out a house fire by dousing only the roof. Where did the fire start? How did it start? Is this arson or an accident? If we break that window will it suffocate or ventilate the fire? How is it spreading throughout the house? Without this comprehensive forensics approach, the house burns down and a lot of water is wasted.
These two concerns stem directly from the article itself. When you investigate the actual report HBR references in its article, more strikes follow.
Strike Three: text-only. The report excludes all posts that contained a video, pictures, event or external link. Translation: text-only posts. Quick: thumb through your FB feed on your phone and count how many posts you have to scroll through before you find one that’s text-only. I stopped counting at 50 because I got bored. Now, omit all those as irrelevant data because the machinery didn’t know how to analyze them.
The report also concedes emojis were excluded from analysis and this warrants special attention. The explanation for the lack of emojis is simple: the data was from 2011-2016. Facebook emojis launched later. Please don’t 😂; emojis are very valuable sources of information in evaluating sentiment, usually more accurate than words in most cases due to their lack of ambiguity and nuance. They are also the only “international” language, understood by people across the globe.
Strike Four: WTF?. The report also notes it excludes any post with fewer than three words because “every full thought requires at least a subject, verb and object to be understandable for receivers.” I don’t know if “receivers” means their NLP/AI machinery, or just people in general, but either way, a response of F@!# YOU is usually understood very clearly by those on the receiving end.
In this article, HBR is selling the latest magic pill. And the KEDGE report is likewise problematic, but at least it is very clear throughout about its methodology. Assuming no intent to mislead or harm, brands should approach AI/NLP solutions for crisis management with a critical and nuanced eye. Ask the AI salesperson to identify their data in very precise terms including what kind of content is excluded, not just what is included. Ask about Facebook while remembering a minuscule amount of the content is public and therefore “invisible.” Look out for self-serving rationalizations of data-stacks like when an AI “expert” told me “people don’t go to Facebook to complain about brands anyway,” so it didn’t matter that his data set was Facebook-free. Inquire about the deeper web: 4chan, 8kun, Telegram, Gab and especially Reddit. These are the platforms where many of the problems incubate and must be part of the analysis—but it takes humans to do this work.
Slippery data-stacks are one concern. But the bigger problem is the lack of analysis around the analytics. In the comms world, “news judgment” has been a mandatory job requirement for decades. “Social media judgment” is equally important and a little harder to define. Like news judgment, it’s more art than science, but it comes from experiencing social media, understanding human behavior, psychology, sociology, politics … all those “soft sciences” that live outside of the math of machines. Further, anything involving the complexity of human emotion and language is beyond what AI is currently capable of analyzing accurately. It’s hubris to suggest otherwise.
An illustration of the fallibilities of relying solely on AI/NLP in crisis analysis might look like this:
Sarcasm, Cynicism and Irony are the John, Paul and George of the Internet. Cat videos are Ringo.
Understanding that statement requires a lot of unstated historical context that a machine is not going to comprehend. Furthermore, AI/NLP fails to understand sarcasm, cynicism and irony effectively and, therefore it cannot truly understand conversation between humans. My Beatles example is sophomoric and anecdotal. But it is emblematic of how millions of people “talk” in social media and understanding that nuance is key to mitigating online issues.
It is my belief that identifying method and motive within conversations is the best path toward successful crisis mitigation. Understanding precisely who these people are, what channels and tools are they using to spread their complaint and, most importantly, why are they doing so, is what is needed. The KEDGE research uses the term “firestorm” 74 times in its 21-page report. This alone suggests to me a lack of understanding of the nuances in play when people complain about brands online. For one, it may not be a “firestorm”; it could be a legitimate set of complaints that the brand should address immediately. Secondly, it suggests that all complainers are “bad,” out to burn down the house. You cannot mitigate a crisis if you view everyone with a complaint as a hostile adversary. Thirdly, it self-inflicts panic which is the last thing a brand should do during a crisis. All in all, the research is clearly crafted by smart data scientists with no experience in communications. After all, the guy with the broken guitar had a legitimate complaint. It’s not a binary score of “bad” or “good” when analyzing complaints.
As a colleague who works in this field reminded me “Machines do analytics. People do analysis.” AI/NLP are incredibly powerful and necessary tools in the arsenal. But, it’s overhyped, fabricates an anxiety relievable only by purchase and can cause brands to make bad decisions.
Replace that anxiety with the hard work needed to identify method and motive, instill a balance of humans and technology, and stay off the hydroxychloroquin. Despite what anyone tells you, you do, in fact, have a lot to lose.