Tiffany Serbus-Gustaveson  0:18  

Happy Thursday everyone I am Tiffany Serbus-Gustaveson, a digital strategist with BWG Connect and we are in network and knowledge sharing group we stay on top of the latest trends challenges whatever is shaping the digital landscape we want to know and talk about it. We are on track to do at least 500 of these virtual events this year due to the increase in demand to ever understand everything within the digital space. And we'll be doing at least 100 in person small format dinner. So if you happen to live in tier one city in the US feel free to shoot us an email, we'd love to send you an invite these dinners are typically 15 to 20 people, having a discussion around a certain digital topic is always a fantastic time, we spend the majority of our time talking to brands that's how we stay on top of latest trends challenges, but love to have a conversation with you. And feel free to drop me a line at Tiffany@bwgconnect.com. And we can get some time on the calendar. It's from those conversations we generate the topic ideas we know people want to know about. And it's also where we gain our resident experts such as Yogi who's with us today. And anybody that we have to come teach the collective team has come highly recommended from multiple brands within the network. So if you're ever in need of any recommendations within the digital space, please don't hesitate to reach out we have a shortlist of the best of the best. And we'd love to provide that information to you. Also note if you have any hiring needs, we do partner with a talent agency formerly BWG Talent now called Hawkeye Search that we can put you in contact with as well. So a few housekeeping items. First and foremost, we want this to be fun, conversational educational. So put as many questions comments you have into the chat q&a bar, we'll be sure to get to them. If you feel more comfortable, you can always email me personally at Tiffany@bwgconnect.com. And we will get to them shortly. So with that, let's roll and start to learn about understanding the future of AI in commerce, focusing on Amazon review summaries and beyond the team at Yogi have been awesome partners and friends of the network. So I'm gonna kick it off to our panelists. If you guys can introduce yourself, that'd be great. And then we can dive into information.

Gautam Kanumuru  2:14  

Thinking. Awesome. Yeah, no thanks, everyone for being here, this Thursday morning or early afternoon, depending on where you are. But yeah, just to kind of give a quick introduction on myself. My name is Gautam. I'm co founder and CEO of Yogi. Prior to Yogi, I used to be a Program Manager at Microsoft working on kind of various AI technology there. So Cortana, Xbox windows, I like to say if anybody kind of has a Windows laptop, right now, you're probably running some of my code in the background. And I also like to say if I'm probably one degree removed from any like Microsoft, the owner of any Microsoft product that you're working on, or using, so if you have any bugs or issues, I can probably get you in contact with the person to make sure that they're fixed. And yeah, and was also kind of lucky enough to be a member of Forbes 30 under 30. In the enterprise, AI and or sorry, enterprise software and AI space.

Spencer Kelty  3:16  

Awesome. Thank you. Thanks for the great introduction, Tiffany. And thank you all for being here. I'm Spencer, I'm Head of Marketing at Yogi. I've been working in the AI space for a while I was the former head of marketing at constructor. And before that I worked on the agency side as a strategist and marketing lead. Really, my goal is to help companies make their customer experience their customer acquisition process and sales processes easier, but also more connected to the actual customer's journey and what they're really looking to get out of products. So with that, let's dive in. And before we dive in,

Tiffany Serbus-Gustaveson  3:56  

because it's so timely, I have some preliminary numbers from Prime Day that I can share those so hot off the press from CNBC. So Adobe analytics is reporting that online US sales between Tuesday and Wednesday, which was prime days was $12.7 billion in online sales, which was a 6% increase over last year. And Amazon though they have not taken out heavy numbers formally, but they did say on the first day of Prime Day, it was the single largest sales day in the company history. And they've sold 375 million products versus last year they sold 300 million products. And then those that competed with them was Walmart, Kohl's, Best Buy and target. We're doing prime ask events. So that was very timely information based on what we're going to talk about today and how you know, those big retailers are following the Amazon way and Prime Day has become this holiday, essentially to shop. So with that, let's get started.

Spencer Kelty  5:00  

Thank you. Yeah, that's, that's really interesting to hear. And it's, it matches what we're seeing prime days kind of become this this focal point for eCommerce. I see a lot of companies calling it Black Friday in July these days. But it definitely ties into what we're going over today. The focus of today's discussion is really that, you know, AI is quickly altering the AI, the eCommerce landscape, we're seeing companies like Amazon, embracing this technology and finding ways to use it to create better customer experiences, generate more sales, and really break down barriers in the shopping experience. And we've really seen that in the last couple of months with review summaries on Amazon. These This is really Amazon's first step into using generative AI like chat GPT like AI. And it really signals a lot of what's coming in the coming years in the eCommerce space. So we're gonna dive into that today. Talk about what brands should be doing to prepare for this. And you know, what this AI first eCommerce world will look like, are some possibilities of what that'll look like. So I'm gonna hand it over to Gautam. Yeah, I'm

Gautam Kanumuru  6:15  

just kind of diving into what we mean by Amazon sort of AI review summary. So this is a new feature that Amazon has started to beta test, you'll see it on kind of a handful of products if you use their their mobile app. And essentially, what it's doing is it's it's reading and synthesizing the 10s of 1000s, or, depending on each product 1000s to 10s of 1000s of reviews that exist, and essentially building summaries based on the most common positive and negative themes that show up in those those reviews. So we kind of have an example here on the right, that is pulled from Amazon from the Dyson Dyson air purifier. So you can see call outs of positive feedback around noise level temperature, performance, appearance. And more specific kinds of call outs about why the positive feedback is is kind of coming. And so these reviews just in terms of where they fit into the overall PDP they tend to show up directly above the review section. And, and at least kind of from the beta test perspective, it seems to be a little bit heavier on kind of electronics based product. So we mentioned the Dyson air purifier, but also the Google Pixel phone. But I think one of the important things worth thinking about or sort of keeping in mind when you think of a feature like Amazon review summaries are just a lot of these AI features in general, which is how does this help Amazon at the end of the day? Like why are they making this this move. And the important thing to note is it's all about removing kind of tensions that a shopper might have in their overall purchasing journey. And one of those those quote unquote tensions is can definitely be the process of navigating reviews. Right? We do know, reviews obviously play a heavy role in the purchasing path, not only from the star rating basis, but just what people are saying. Another statistic that we have is the most applied sort of action done on reviews and ratings, whether it's on Amazon, or other retailers, or even brand websites is people most often organize it from most recent on doubt so that they can see the recent reviews. But the long and short of it is is there is some tension that can be created from a user that has to read or feel like they have to navigate through 1020, maybe 30 reviews, before they understand whether this is a product worth by the same goes for when they have to start to do kind of side by side comparisons. Okay, I've narrowed it down to the three TVs that I want. Now, let me kind of read these reviews don't understand it. So this this AI summary at the end of the day, what it's really doing is removing that tension of having to read those those reviews. And that's that's a very important point to keep in mind when not just this feature, but all features that start to get incorporated from from this perspective.

Tiffany Serbus-Gustaveson  9:14  

Summary live? Yeah, yeah, it's

Gautam Kanumuru  9:17  

a great question. So it shows up on the actual product page, kind of as you're navigating down plat paths, the a plus content, right above kind of the reviews. Summary. So generally, what they'll give you is the breakdown of like, how many five star reviews four star reviews, three star reviews, but before they show the actual like verbatim reviews, the summaries kind of show up in the middle.

Spencer Kelty  9:41  

Now, one thing I wanted to add before we move on here is you know, the the goal here, as both alluded to is you know, Amazon is looking at their core metrics. And they're they're putting this out on test products, and they're AB test to them. So if you go on the app right now, there's maybe a 5050 chance or actually even going to see This because they are definitely a be testing this to see how it affects things like conversion rates time to purchase, maybe even things like cart size. So we're seeing them roll this out in generally high cost electronics. So far, I think there was a few items in the pet category. But basically nothing that I've seen it on so far is under like $150. It's all electronic items to some extent. And they're basically the type of things that people are generally going to do more research on, they're going to be looking at reviews, they're going to be getting as much data as they can before making that really informed purchase. So Amazon is trying it on that because those are the type of times when consumers are probably spending the most time in reviews they're seeing if it reduces that time to purchase decision increases their conversion rate. And if it does improve those metrics, we're going to see it being tested in more and more verticals, with a little bit, a little bit less research involved. So really, it's going to start rolling out into more of these verticals as those positive results come out. So one of the things that we really want to want to cover today kind of get into is get everybody thinking about, you know, if if Amazon review summaries were rolled out for your products and in your categories tomorrow, what would they say about these about your products, the summaries have a pattern to them, they usually pull up between three and five positive qualities, and between one and three negative qualities. Now these have to do with how these impact reviews and how they appear in how what frequency they appear in reviews. So there's some predictability here, we're going to talk a little bit about how we can get into that and how you can really start to understand what Amazon is looking at. So we're gonna start with some do's and don'ts. You are muted Gautam. Thank you.

Gautam Kanumuru  12:04  

Yeah. So when it comes to the kind of Amazon review summaries, I think the paradigm to really start to get an appreciation of is obviously everybody keeps an eye on their star rating, you want to have as high of a star rating as possible. But the actual content and what people are saying in reviews, has now kind of exponentially gotten more important. So it's very important to start to think of reviews as a really strategic data source to kind of inform smarter decisions downstream, whether these are decisions around what you put on your product pages, to marketing claims to even product innovations. What we tend to see a few folks kind of thing, when we look at the Donate section is you can is essentially just thinking of just worrying about generating reviews at the end of the day, right? Like okay, I need 500 reviews, what is the fastest way that I can generate them and then essentially, just move on from from that perspective, but with things like review summaries, with customers sort of getting more input in an easier way of what people are actually saying, those reviews kind of stick with you longer from from that perspective, then just looking from a PDP perspective, you really need to ensure that your PDP reflects kind of your customers actual experience with with your products, right, there's always the, what we wish was actually the case and what the reality actually is. And so as opposed to kind of treating it, kind of in this dolt section as an aspirational as aspirational content, it really is about truly understanding Hey, why do customers gravitate to your product? When do they get the best experience from it, and leaning into that, um, then there's just the monitoring of reviews, right? Things constantly changed, the world is constantly moving, it might be a change to your product, it might be a change to a competitor product, it might be something that happened out in the world that is constantly going to cause people to think, think and rethink what how they feel about your product and your competitors products. So it really isn't any more about kind of a point in time or checking in once a year or even quarterly. This is a constantly moving data source. And these review summaries are going to be generated on a kind of a consistent basis. And so it's important to really understand what's what's kind of going on. And then the final point is just really around kind of reiterating to the beginning. Focus on the review content over kind of the star rating, or at least in line with the star rating versus just purely focusing on what that Top line metric is because at the end of the day, there's a lot of nuance that can come with with these star ratings, right. So there might be a review that talks very positively about the taste of the product, the texture of the product, the instructions that come with the product, but they give it a one star review, because it showed up two weeks late, for example. Um, those are those are kind of can be sort of degrading to the overall top line star rating. And sometimes some of it is out of your control. But the nice thing that starts to happen with these review summaries is that inner depth is going to start to show up a lot more for for customers, it's going to be a lot easier for them to understand this. And so it's definitely going to start to play a role in Downstream decision making.

Tiffany Serbus-Gustaveson  15:50  

Glad you brought that up, because that was one of the questions that was lingering in the back of my head. And fairly reminder, if anybody has questions, comments, put them into the chat q&a. But how do you manage the Amazon business? I despise the fact when I got that one star, the product they loved but the logistics, they had a problem. And so have you seen them that the AI is able to take the Amazon issues per se of logistics or packaging from like an FBA fulfillment, and keep that out of the mix? When it comes to the summaries?

Gautam Kanumuru  16:23  

Yeah, yeah, generally speaking, we've gone through, like a fair amount of these summaries, and we're yet to see one where packaging shows up as like a topic of conversation. And that's a very sort of, when you think of like the engineering and algorithms side of things, that's not a hard thing for Amazon to implement on their side to ensure that the review summaries don't cover, don't cover topics that may be out of control of, of kind of the actual product, if you will,

Tiffany Serbus-Gustaveson  16:56  

we really can do the flip side and then give the brands that are star rating by extracting the Amazon issues out of their reviews.

Gautam Kanumuru  17:06  

Yep, yeah, that's a whole that's a whole different story. Yep. But then I think, like, in our head review summaries are just really the first step, we're going to start to see this AI and AI based experience show up more and more across just the entire buying lifecycle. So on the right hand side, this is a really interesting demo that's actually kind of available to a handful of folks now from Google Bart, which is Google's sort of chat GPT competitor. And so you can see here that the query is a good bike for a five mile commute with hills. And what Google is able to do is not only give you suggestions on hey, here's what you should consider when I'm like looking for a bike in this certain scenario, but will also give specific, sort of like call outs of what products do you suggest almost a buying guide is the way that we've been calling it out internally. So you can see the first option is good for hill climbing, and has a call out of some of the features that the product has, as well as what reviewers are talking about in this product. The second one is a little bit better for the commuting side of things, and also has reasoning for why you might want to select that one over potentially the first option. And so what's going to really start to happen, we also know that just kind of through some articles that have come out and some some information that we've been hearing that Amazon is also looking to potentially implement a similar type of change to their search or similar type of experience. And so you can see that these buying guides are really interesting on kind of two fronts. So just, again, going to reviews and ratings as a data source, they're going to become very core for just feeding into these these types of algorithms, right? In a lot of ways, reviews and ratings are now going to be part of your SEO story to make sure that you list sort of top or amongst the top results for queries like like this. But the second one is again, putting that hat on like why would somebody like Google do this? Why would somebody like Amazon do this? This actually removes a big tension point for a lot of customers, which is the research part of the shopper journey. I'll just talk from personal experience our kind of TV went went on the fritz about a week ago. It's been on my list to find a new TV over the weekend. I started my process and then next thing I knew I was two hours in I had like 15 tabs open and I still didn't know what TV you were gonna buy. But that's because there's the tension of kind of analysis paralysis, having to go through multiple different options reading tons of tons of reviews. I even read some data through Yogi to analyze it ourselves to make a decision but The long and short of it is is this is going to remove what ends up becoming a big tension point for customers around sort of the research phase, and identifying what product might might be best for them. And so this is definitely just when you think of it as what this helps consumers with, this is definitely something that we expect to see all kinds of platforms doing Google, Bing, Amazon, but it's going to start to become more common over the next few months and years. So with that in mind, like, I guess, kind of the next question is, why are product reviews? are kind of such a critical data source? Right? Or like, how do they, why are they continue to be important to kind of keep track of so just generally speaking, when you think of why they're becoming such a feeder into these algorithms, and these these kind of AI chatbots and stuff, it's because they they truly do give great visibility into what consumers actually feel. They're on demand or available as needed. We already know the high impact that that they can have on sort of conversion rates and sales, their public and kind of, to the point, brand defining so what what's really interesting, that kind of takes reviews, in a different light, especially for product companies compared to things like social listening, is that you know exactly what product even a lot of times what SKU folks are talking about. And so you're really able to pinpoint, hey, is this the best product for me from the consumer side or as a brand? Like? What, what product? Or what SKU are they actually talking about? It's directly in line with how consumers feel. And kind of jumping to the last bullet point. It's easily benchmarked against competing products. As a shopper, when we're deciding what TV to buy. A lot of times, you have that feeling of like, okay, hey, everything is from one to five stars. So everything is kind of scaled accordingly. So I have that confidence that a TV that's at a 4.5 is probably going to be better than a TV that that's at a 4.1. But kind of at the same time. It gives brands the opportunity to be able to benchmark themselves against competitors as well. And quickly question

Tiffany Serbus-Gustaveson  22:26  

about the summaries. So it appears the summaries for Amazon are SKU level, Google is that as well, or is that a category

Gautam Kanumuru  22:37  

level? Yeah, so their summaries are what I would what we've been seeing are kind of at a parent SKU or product level. So what what we mean by that is like two pack four packs, six packs are all going to be combined and kind of fit into sort of the same summary or the red color, the blue color, the green color of the same bike are all going to be kind of combined into into one.

Spencer Kelty  23:00  

Got it. So yeah, keeping all that in mind. I think what most brands are going to be asking themselves in the next few months is how are we supposed to apply review data? In actual practice? How do we prepare for this? How do we get ready for this and ensure that we don't miss out, we're not the last to move in our category. And the first step is really thinking about reviews as a strategic resource, it's really clear that Amazon is thinking of them as a strategic resource. All of these online retailers are because to go through this point on the last slide, they are the most plentiful customer data point on how customers feel about products. And if Amazon or Google are going to make shopping recommendations, they're not going to make them just off your PDP anymore. They're going to make them off of what actual people are saying and reacting about that product. And that goes back to the point about aligning your PDP with actual customer expectations. As reviews become leveraged as a source for for these ai, ai engines, you really have to make sure that you're thinking about them in that way. So you can align your PDP and make sure that when people scroll down from your PDP, it's not giving completely different information. You need to make sure that you're thinking about those reviews as core indicators of your customer experience. And I'm going to shout out our reviews one on one strategy course that just launched this week. I'll send a link to everybody after the webinar. So you get a chance to look at that. But it's a great way to start thinking strategically about how to apply reviews to your organization. It's designed specifically for for companies that are focused on their eCommerce and really getting ahead of what's coming next. After you start thinking strategically, it's really important to build a full strategy and understand exactly how review analysis needs To play a part in your your eCommerce in your retail, we like to always say that you have to start somewhere. And manual analysis of reviews is really where you, you're going to want to start at some point. Specifically talking about Amazon review summaries. It's not the most complicated AI system in the world, they're really looking at frequency of key words, and just when they're appearing interviews, and what they're really, really seen as those common themes, both positive and negative. So this is actually an interesting case work, manual analysis can probably tell you a lot of what you're going to see in an Amazon review summary of your products, you can look at, you know, sampling 500 reviews and seeing, okay, what are the what are the most common phrases I'm seeing here? What are the most common themes, obviously, that's a lot of work to do manually. But you can get a lot of that information and kind of start to understand what Amazon might pull in a review summary. But obviously, going through 500 or more reviews by hand is very difficult. That's a, that's a very time consuming manual analysis process. So kind of the ultimate step there is to use your own AI tool, kind of use the sort of same sort of technology that Amazon and these retailers are using to match your product reviews, to a customer experience as desired. And really understand exactly how that's going to look and what in your product reviews is going to resonate with people, or what's not resonating with people. So kind of looking at how to use review data, I'll hand it back over to Gautam.

Gautam Kanumuru  26:41  

Yeah, for sure. Um, so just to kind of set the stage for everybody now is probably a good time just to like, quickly go over sort of what what Yogi is. So we add our core, we're reviews and ratings analysis platform. And so that's why once Amazon has been putting out their review summaries, we've been seeing these AI tools kind of come out. It's it's started to have a lot of our customers ask us questions. And it's also caused us to do a lot more deep dive and understand the influence that just review data is going to have downstream. So at the end of the day, when it comes down to using review data, it was enough number of kinds of data points that are out there might seem very overwhelming, it does take less time and effort than you think. And at the end of the day, the best way to think about it is what are those low hanging fruit changes that you can make, that will quickly lead to ROI and make this process worth it. And so Spencer alluded to kind of the, at the end of the day, it just about starting right, it's about taking the first step and just prioritizing, analyzing the review data. For some folks, if it's at a smaller scale, it might just be like, hey, once a week, I'm gonna take half an hour to sort of read through some recent reviews and just see what common themes I can have. For people that are maybe at a larger scale, potentially, they need a tool, whether built internally or kind of a tool like Yogi to help out. But more importantly is what are the potential changes that you can make based on what you've learned from reviews and ratings data in order to not only influence kind of shopping as it exists now through Amazon, PDPs, and Walmart PDPs, but also how things are going to evolve over the next say, six to 12 to 18 months. And so the first one that we always like to start with on kind of the left hand side with the green box around it is just PDP and digital campaign optimization. This is kind of very low hanging fruit. And at the end of the day, what we really see this revolve around is paid based on what we're seeing people talk about in reviews and ratings, what are some quick changes that we can make to our our content. And this generally revolves around kind of two buckets, one of them is just understanding better what people care more or less about. So hey, we really emphasize the how good our product tastes. But what people really like is the fact that there is no aftertaste to the product. So that's actually what we should be emphasizing. Because that comes up a lot more in conversation. The second one is just oh, we're noticing that from a instruction standpoint, our instructions are a lot more clear compared to our competitors. So maybe that is something that we can allude to or when we kind of can, we know to target certain competitors based on some confusion that's happened with some updates that they've made, so on and so forth. And at the end of the day, just what this really is, is talking at the consumers level, like using the words that they use using what they care about, but also dealing with mismatches of expectations. Um, We have one one customer, that that makes a product that is made to last 90 days. But it ships in a bottle this big, for example. And so one of the interesting things is, when you look on the Amazon page, it seems like a big bottle. But when it gets shipped to somebody's front door, they'll open their Amazon box, and they'll see something that looks super tiny, even though it has a 90 day supply in it. And so what was really interesting is they were getting the low star rating, because everybody was like, Oh, the products great, it works fine, I just thought it would be bigger. And so that is something where they're able to kind of make some adjustments, it's just so show some some images that show like, hey, size to scale, this is what to expect. But just know it's a small, small bottle that packs a big punch, for example, that eliminate, like, eliminates a bunch of three star four star reviews that are kind of unnecessary and brings your average star rating up. But then once you start to look sort of downstream to there's other changes that you can make marketing claim improvements, just better understanding what customers care about what they like, what they dislike, how you compare to competitors, and taking those into account with kind of some of the language that you use some of the features that you emphasize. And then you have things like product updates, and new product innovation. Obviously, we know this is a lot more of a 1218, maybe even 24 month time horizon. But truly just understanding how to make your product better. And where you're potentially underperforming is a great, great example of this, we have another client that essentially ships ships a product that has a tendency to melt, when by the time it shows up at somebody's front door. And so there was a really interesting, there was something that they looked at where they actually went through a reformulation of the product. And r&d came back and said, Hey, this should be a fixed problem now, but they actually haven't seen any changes in the downstream data from an Amazon perspective. So now this is another one where you're able to go back and be like, Hey, we're fixing the right problem. But this didn't hit the way that we expected. So you're able to kind of measure that and then prioritize another product update, for example, and move like that

Tiffany Serbus-Gustaveson  32:19  

are the marketing improvements, but it also be opportunity to enhance your keyword strategy? Yeah,

Gautam Kanumuru  32:27  

that's a great question. Um, definitely. And I think it, I'll actually give the what I personally found one of the funniest examples of this is, so we have some some clients in the food space, one of them makes pasta, pretty much. Now the very, very interesting thing is is this, this company has kind of the word pasta on their box plastered kind of throughout their PDP. But it turns out that when you look at their reviews, people actually use the word noodles more than pasta to explain what the product is. And so if you did a search for noodles on on Amazon, or Italian noodles that that product would not show up. And so that's just like a very quick example of hey, that's now a new term that maybe maybe as like a devote sort of, like, Italian company, you don't want to use the word noodles, but you're going to show up in more listings if you do it. Yeah. And so, yeah, that shows up a lot.

Spencer Kelty  33:30  

Awesome. Yeah, a couple of quick things I want to want to highlight here. One is just the importance of competitive data in this process as well. Looking at your interview data is great. And you can definitely find a lot of opportunities based on only your data. But a lot of the depth and richness of this type of analysis comes when also pulling in competitor information. And looking at their reviews, especially when we're talking about things like you know, emerging technology that's going to make reviews interacted with in a different way. So I mean, for example, if you start doing in depth review analysis in your category, and nobody else's, you are going to understand the impact of things like Amazon review summaries, and be able to make changes more quickly, react faster, be more agile. And, you know, just from a basis of understanding what the actual impact of something is, rather than the relative. Let me let me demonstrate. So, you know, let's say that you've identified that 20% of your reviews mentioned packaging or shipping issues. That doesn't necessarily mean that it's a problem above the industry average, that doesn't tell you at all, what the normal ratio of shipping and packaging issues are. But if you can look and say okay, we're at 20% But our four competitors they averaged 15%. So that means that we are falling behind to your we're having more More packaging issues or more shipping problems, maybe we need to change something about our product packaging from there. But if you just look at your own 20%, that doesn't really mean anything. So being able to understand exactly where you are, in comparison to the rest of the market is really critical here. And going back to the Amazon review summaries, they're pulling the the features that appear most common in an attribute that appear most common in reviews. But the most common compared to what if you're talking about you know, let's just say like, a clothing item, for example, like fit and quality are probably going to be the most common terms that come up for pretty much any product. So if you're, if you're just a little bit behind, you might not be seen as much positives there in your reviews. So being able to benchmark the rate at which you're seeing these comments compared to the industry average compared to your top competitors. It's critical, you can't really make very informed decisions without that. I think that's really the only thing I wanted to add, Gautam. Anything you had else before we move on.

Gautam Kanumuru  36:08  

No, no, no, that's a that's a good call out on just having context kind of on on your problems and what's worth fixing.

Spencer Kelty  36:18  

Definitely. So next, I wanted to jump in really quickly and kind of kind of show a case study of a company that's using review data to really make a large impact. We work with Tylenol. And one of the products that they were really keen on using reviews to make changes with was basically a powdered version of Tylenol, Tylenol dissolve packs, it was a newly launched product when they started using review analysis in yogi and it was performing considerably lower than their standard. Tylenol is an amazing company, they have extremely high standards for how their products rate how their products performed. And this was just not up to snuff, it wasn't up to their, their benchmark that they wanted to see. So when they started using Yoki, and looking at all of their product reviews for dissolve packs, they were able to find two major themes. One was kind of a negative that was happening that they were seeing a lot of negatives in their reviews, and one was a positive. So we're going to explore kind of what it looks like to lean more into things that are working and fix things that aren't. So on the side of the of the things that weren't working, they realized that the overwhelming majority of their low reviews their one two star reviews, specifically called out that the reviewer was using the product wrong. They mentioned pouring it in water and that it wasn't dissolving correctly. When in fact, the instruction said you don't need water, you pour it right in your tongue. But it makes sense because consumers are so used to things like liquid IV, and different products like that, that you pour in a bottle of water and shake and dissolve. So consumers have been trained to use powdered products in a certain way. And Tylenol discovered that they weren't doing enough to overcome that kind of ingrained idea that people came into using the product when of how it was to be used. So what they realized was they needed to do more, they needed to put more information on their PDP, they ended up adding a you can see it on the screen here the bottle with the X over it to demonstrate no water is needed. They added that to their image gallery. They added another lines there PDP that said no water needed pour directly on tongue, they really found ways to make it very clear in the purchasing process that this goes directly on your tongue. The other thing that they saw the on the positive side was that there was a higher percentage of reviews that said it was fast acting than any of their other products that work currently under their fast acting umbrella. So from there, they added that to their PDPs They added that to their marketing claims and they started advertising this as a fast acting products. From these two changes that were you know, had nothing to do with the product itself, nothing to do with packaging, nothing to do with anything that would be somewhat long term and difficult to make make changes to just from PDP and a few marketing language changes. They had about a full star rating increase in their average reviews. And that's all just about aligning the PDP and the marketing claims to what the consumers are actually experiencing. Now I want to tie this back to Amazon review summaries you know, for example, with the with the two themes discover that you know one It wasn't dissolving in water which one wasn't what they intended for it to do, but it was the the negative experience people were having. And the positive, it being fast acting, those are two things that very well may have shown up on Amazon review summaries. So if if Tylenol hadn't used yogi and done review analysis, they could have woken up one day realize that Amazon had put review summaries on their products and discovered these things for the first for the first time and not had any time to react anytime to plan strategically. And they would have been, you know, extremely reactive at that point. With that said, I don't think they would have picked up the the dissolving in water thing that's a little bit more of a deeper analysis. But the point is really just that by getting proactive and building a review strategy, Tylenol was able to figure out these things and make a massive, massive impact to their product. Anything to add Gautam before we move on? No, no. Yeah, no,

Gautam Kanumuru  41:02  

at the end of the day, I think the fascinating thing is that these sometimes it's the simplest things that that get get mixed up. Why Tylenol call that product is all tax, I never kind of like asked them or dove into that that marketing side. But it was purely just as simple as that product is meant to be poured in your mouth, people were pouring it in water. And yeah, and that can cause an entire product to kind of kind of tank or if you can catch it early enough. You can kind of use it to your advantage. But I think it's

Spencer Kelty  41:36  

fascinating that, you know, that's the difference between a three and a half star and a four and a half star rating products. Just just the way that your PDP aligns with actual consumer expectations.

Tiffany Serbus-Gustaveson  41:47  

And just curious, do you think they change the actual packaging of the product to have that no water required?

Gautam Kanumuru  41:55  

Yeah, yeah. As far as kind of like when we've been talking to them, they made it bigger. It was always on the packaging. But they made it bigger and prominent and made multiple call outs to it on kind of

Tiffany Serbus-Gustaveson  42:06  

different it's true omni channel change of Yeah, it impacted other areas. Yep. My experience. Very cool.

Gautam Kanumuru  42:16  

Awesome. Yeah. So we can, we can kind of dive into another use case that kind of shows a little bit more on the marketing claims side side of things. And then I know, I think we've gotten a few questions that happy to jump jump to those afterwards. But another kind of customer of ours is Nestle. And they they kind of use yo yo across a multitude of different product categories and brands that they have. But there was an interesting kind of outcome that came with the coffee mate team, pretty much were one of the things that they were able to see in in yogi is that the amount of conversation around people using coffee creamer and other types of beverages was starting to increase actually not even other types of beverages. There's other use cases, teas, hot chocolates, even in baking, mixing it with your ice cream, all of that, that kind of stuff. And so by kind of discovering this early, what coffee mate was sort of able to do is incorporate this in different just marketing, messaging and marketing imagery that that they would have. So this is one from kind of the fall where it's a glass of hot chocolate versus kind of a glass of coffee. But one of the interesting things that that happened because of this is by Tate being able to take control of this conversation earlier. And recognizing it before sort of other competitors, they were able to sort of control the narrative. And as they started to emphasize this more, it started to show up in reviews and ratings board as well. So there's kind of a 60% increase in mentions of sort of using alternative beverages with your coffee creamer. Now, again, tying this to kind of how this changes, was kind of like the potential new paradigm in shopping and searching is, in a lot of ways, what we've consistently found is that what you put in your PDP does show up in reviews and ratings, right. And when you think about it, it's sort of this thing of, okay, somebody's deciding to buy your product based on what they read and what was described to them. Those are the things that they're going to keep in mind when they receive the product to decide if it was a good buy or bad buy fulfill their needs, if it didn't, and that is going to downstream show up when they write reviews. And so really at the end of the day, it's not necessarily immediate changes, and it's not like oh, let's run this promotional review program or Amazon vine program to generate kind of 100 reviews. But it really is your ability to kind of control the narrative and control what Um, people sort of the people that are purchasing your product, what they're going to talk about and what they're going to pay attention to you. And you can start to see how that flywheel will start to feed into things like your AI review summaries, your AI, buying guides and things like that, because you're gonna essentially able to control Hey, we're going to be the best in this, or this is what people liked the most about our product. So let's really, really emphasize that it's going to show up in our PDP, it's going to start to show up in reviews. And whenever we know that somebody is searching for an inflatable pool that isn't going to puncture in 20 years, like ours is going to be the top suggestion. And so that is where this entire paradigm starts to become. Super, super interesting.

Spencer Kelty  45:43  

One thing I want to add really quickly before we move over to questions, with this case, in particular, is that when Nestle started looking at Coffee mates mentions for alternative uses, they were not one of the leaders in their category. Coffee made from a market share perspective dominated the other brands. But as you can see, on the chart on the left, they only had you know 11% of the consumer mentions around other uses. So it's interesting that you know, you can see from that, that other brands are positioned themselves already around different use cases, you can see like earthstone had 55% of the volume there. So by being able to understand exactly what those opportunities are, coffee mate was able to complete expanded to a new market that they were just not addressing. I mean, when you have a brand, like like Nestle's coffee made that so ubiquitous, you're not going to be able to do much more going after coffee drinkers, coffee drinkers, either like Nestle coffee mate, or they don't. So this was a great opportunity for them to understand exactly what people were talking about. And again, I know that my last comment that I added to go from slide was was one competitor competitor analysis too, but I think it really just does sell it that looking at competitor data is as important and you really can't just ever look at your own, you have to look at the full the full picture, or you'll never have any idea if copying, they just looked at that 11% That was theirs, they would have no context that there was a brand that had five times the mentions of alternate uses, and they did

Tiffany Serbus-Gustaveson  47:29  

such a great point, and to look at our product development, you know, focusing on your competitor. And getting that Intel, when you're bringing a new product to market can save you a lot of time and a lot of money. Let them make the mistakes essentially. So we have about five minutes left. If anybody has any questions, comments, put them in the chat. And we're gonna get to them. We do have a few in queue here. So if you guys are ready, we'll go with them. And some great questions. So how does the AI percentage out the time that most customers are taking because you're more inclined to post negative feedback versus positive?

Gautam Kanumuru  48:06  

Yeah, yeah, this is a great question. So I won't I won't dive too much into the technicalities of like prompts and how kind of these these technologies work and stuff. But the general pattern that we've seen with kind of the AI review summaries to date, have been a distribution of what this does good at and what this does that at. So yeah, there is a maybe this, there's always been this general feeling that people might kind of write to the extremes in reviews, we've actually kind of been seeing that theater off a little bit because it's become so much easier to write reviews that more kind of a typical shopper demographic is more willing to write them. But generally speaking, when it comes to these actual sort of summaries, at the end of the day, it is going to cover both what people like about this product and what they don't like about this product. Now, when you dive in deeper, what the interface allows you to do is sort of get a sense of, okay, hey, if people are talking positively about the taste of this product, like what is the general ratio of it? Is it 5% of people? Or is it 25% of people, so there is more nuance that shoppers will be able to tease out as they kind of click into the experience, but high level it is going to cover both what it's generally good at and where the product might be falling short. So it's almost like the summary sort of paves over, maybe on some products, people's inclination to give more negative feedback and gives that top line of, hey, we're first going to talk about what is good about this product, and then we're going to talk about some potential shortcomings. So

Spencer Kelty  49:47  

one thing I want to add really quickly there is obviously we have a small sample size so far and most of them are category leaders. So what remains to be seen if if this will change, but as it Is there hasn't been a single product that we've looked at that had more negatives highlighted than positives. So it seems that they have some sort of a formula where it always appears to be about three to five positives and about one to three negatives. What we haven't seen is any products that they didn't highlight in negative, they always found something to highlight. And there hasn't been like I said, one that the positives didn't outweigh the negatives.

Tiffany Serbus-Gustaveson  50:27  

Have you noticed in the analysis thus far, how they're handling vine reviews, the those that are the free samples? Are that reviewers being compensated, per se,

Gautam Kanumuru  50:39  

included in this? Yeah, yeah, it's a great question. Um, we have found, I think it's been like one or two examples where some vine reviews were included. But I think there's still, there's still some more exploration that we have to do to get to the specifics of how how much weight is isn't given as an equal weight to every other review or potentially a lesser thing?

Spencer Kelty  51:02  

Yeah, I would, I would say that Amazon already does emphasize their confirmed purchasers, they, you know, they've done a lot to make sure that people who are definitely a verified buyer, are featured. So I wouldn't be surprised to see those getting more weight in these these

Tiffany Serbus-Gustaveson  51:19  

summaries. Got it? All right. Now, the question here, how does Amazon game this for their own product benefit? As opposed to competitors? Great question.

Gautam Kanumuru  51:31  

Yeah, yeah, it's a great question. The the honest truth of it is, is we haven't fully seen the playbook yet. And so we'll we'll have to see where our reasoning is, is, from a peer review summary perspective, is probably going to be the same algorithm summarizing Amazon products as it is, Dyson products or any other brand that that you can think of. So once you're on the actual PDP, like the experience should should essentially be the same. from a search perspective, once Amazon starts to introduce these kinds of buying guide type experience. I think there's, there's a chance that maybe they from like, let's say, what is the product that gives you the best value, for example, they might start to recommend their products and their private label, Amazon Basics always tends to come out and a little bit cheaper. So yeah, so I think the jury's still out on on that. But the other thing to note, and this is actually a thing that we see with with reviews and ratings as well, that's, that's very interesting, which is, if you track conversion rate to star rating, you what you actually see is a bump from 4.3 to 4.5, will always help you even a lot of times 4.5 to maybe 4.6, or 4.7 will help you but once you start to see products that have a 4.8 4.9 or five, five average star rating, conversion rates actually drop, because people don't believe that this product is perfect. They automatically feel like oh, something's wrong here. Like I'm not gonna buy this this product, they're obviously gaming the system or something like that. The same thing is going to happen with the AI review summaries. And so there won't there shouldn't be a world where they're just going to talk about what's positive, because everybody's going to be like, hey, there has to be like something wrong, the entire point of people looking at reviews, especially most recent, especially most recent one star reviews is to kind of get that sense of hey, I know this isn't going to be a perfect product. Am I okay with what might potentially go wrong with with this product? And so that's kind of an important point to note with this as well. Exactly. Yeah.

Spencer Kelty  53:48  

I want to I want to piggyback on that a little bit and talk a bit more about a few things to Gautam alluded to, you know, one of the downsides, about looking at review from a star rating perspective is and Tiffany, you mentioned this earlier, it includes stuff that has nothing to do with the product, it has to do with shipping delays, it has to do with the carrier Miss handling your package and it arriving damaged, it has to do with with fulfillment, it has to do with packaging, it has to do with things that are not intrinsic to the product experience. So, you know, one possible path is that this sort of technology and this sort of approach that Amazon is taking could very easily de emphasize the importance of star ratings. And it's interesting that you know, we brought up Amazon Basics and their own products. I think that to go through his point those will show up more more commonly in buying guides if Amazon's building those but I think the key here is buy and then analyzing reviews to understand the common themes that lets the match those themes to buyers needs. So whether or not it's an AI algorithm on the buyer side too. that shows and predicts what features and attributes a specific buyer is more likely to resonate with, it's still going to go more in a direction where those attributes are what's focused on rather than the star ratings themselves.

Tiffany Serbus-Gustaveson  55:17  

It's always a pleasure having you guys, you always bring like so much awesome information. So appreciate it. So with that, I mean, we're at time, this went really fast. So thank you all for joining today. We definitely encourage follow up conversations with the yogi team, and government and Spencer, thank you so much for joining today and providing all the great information and just to confirm, this is in beta, so Amazon will call you don't call them they'll let you know. Okay, got it. noted, as well. Have a great week. We'll take care and hope to see you at another event. Thanks, guys.

Spencer Kelty  55:53  

Thank you, Tiffany. Thank you, everyone.