Microsoft Adds AI Search Reporting to Bing. Is ChatGPT Next?

[music] Heat. Heat. [music] [music] Hello and welcome to the dojo, a search marketing news podcast by Exposure Ninja. My name is Dale Davies. I'm the head of marketing exposure ninja and I'm joined by our CEO, Charlie Martin. How you doing, Charlie? Doing great, thanks Dale. How are you? Pretty [snorts] good. I'm not experiencing as much nice weather as you are. Um, but as alongside the nice weather, there's some nice news coming out of Microsoft at the moment. Uh, or so I hear. Some brilliant news. Some brilliant news coming out from Microsoft this week, which is that they have added AI performance tracking into Bing's web master tools. It's a pretty huge revelation. Everyone of course wishes this was Google with this announcement, but it's Bing. Um, what their AI performance dashboard actually includes is reporting on what's coming from Microsoft C-Pilot and partners. The and partners bit though, we don't actually know who that is or what that is. I would make the assumption at this point that that isn't going to be chat GBT even though there is some type of partnership between uh Microsoft and ChatgPBT at the moment but potentially other integrations and potentially optimism uh that Microsoft being might be might eventually be able to report on that traffic. So I've only seen a little bit about this update this news. What does the reporting look like? Are we getting like full conversations? Is it telling us like the very first question that people ask? Are we getting that much level of detail? We're not getting that much level of detail. So the AI performance tracking inside Microsoft Bing's web master tools, what a mouthful at the moment is showing things like the total citations, which is the number of citations that are displayed as sources in AI generated answers. So how often your content is referenced by AI systems, the average cited pages. It's also showing something which is really interesting, probably the most interesting of all, which is what Microsoft are calling grounding queries. So some new vocab also thrown in here. These are the key phrases that AI used when retrieving content that was then referenced in an answer. So, this is probably the bit I'm most excited about because there's going to be some potential tells about what kind of queries you're coming up in. The key phrases means that this is sort of related to keywords, what we would traditionally associate in our Google Analytics or in being web master tools for traditional SEO showing up in the search engine results page. But now we have an idea of what kind of phrases AI was showing us up for, which is fantastic. So even though this is limited to co-pilot, which is one of the smaller AI chatbots, I think there could be potential learnings for businesses to take just by understanding the kind of queries that are going in there. particularly great for B2B because many businesses who sell to other businesses, it can be the case that Microsoft suite is the type of suite that they use, which means that if you particularly care about B2B search queries, then this could be really insightful for you. So, I think that's fantastic. The other couple of things in there, um, visibility trends over time. So, how citation activity is changing for the website, how visible your website is as a whole over time. Uh, and that's pretty much it. There's also page level citation activity. So, which specific URLs from your website make it into some of the AI answers? So, it's not a full and complete reporting, but it's the first kind of reporting we've seen at all that shows some of these insights. So, an amazing start. I think it sounds like it really matches up with what we see from a lot of the tracking platforms as well in terms of how visible your brand is for various you know prompts or queries but also the citations such a key part of it that we for ever always been used to getting you know this is understanding where I'm ranking for things and this is how much traffic I can expect from that. This is like the modern equivalent albeit not as much traffic of this is how much I can expect my pages to appear and therefore I can start to predict how much traffic I may get off the back of that as well. Yeah, absolutely. I think for predictive and general insights this is great. It's obviously only a small percentage of what the AI search pie looks like overall. The other thing I would say about grounding queries is this is the first time we're going to see the reporting actually showing us the phrases the queries where there's answers being asked in AI. Whereas a lot of the reporting tools which are absolutely fantastic like PKI, Scrunch AI profound, it's working the other way around. we have to decide what kinds of queries and prompts the business wants to go after and then it will show us those visibilities scores for those. So we're we're pre-programming whereas this is not pre-programmed. It's showing what people are actually searching for in M Microsoft Copilot, giving us some insights into what those queries look like, which I think is why it's so exciting because a lot of people are like, "Bing, that's tiny." Like, how often are people in their Bing web master tools? Not that often. And co-pilot again such a small player at the moment when it comes to AI search. What I think is interesting, exciting is the trend that this is showing for us. This is going to be putting pressure on OpenAI to show some kind of reporting. It's going to be putting pressure on Google in particular and Google Analytics to be showing some kind of reporting. And particularly because what's happening in Bing is not only is it showing Microsoft Copilot, it's also showing Bing's AI generated summaries which are Bing's version of Google's AI overviews essentially. So again, potential insights we can take from there and apply across. And we would love of course Google to be doing some kind of similar reporting in analytics. Do you think like I put in the title for this podcast that you know could Chachi be Chachi PT be next? Do you think this will force their hand a little bit to add in this level of reporting or do you think they'll be similar to Google's current position of we have no intention of bringing this to you because we know what happens when we give you this information. You try and [laughter] gain as much as possible. [gasps] I think ChachiBT will hold back. And the reason I say that is because their analytics for the ads that they've recently been testing in the US have been very very thin. So I don't think that CHBT is in the mindset of trying to give excellent and integrated analytics reporting at the moment. I don't think that's their priority. But also I do think that this is going to be quite a big shift in terms of setting some expectations. So my hope is that the long game will be CHBT introducing something similar and Google introducing something similar even if not exactly the same. I just want to come back a little bit to what you covering there in terms of groundings. This is going to be a brand new word for a lot of people and I want to make sure that people go out of this episode with a full understanding of what that is. A groundings like is this me searching and getting in my reporting uh best commuter bike or is it the topic of commuter bikes or bikes alone? Like what exactly am I getting from this? because I I appreciate that I'm not going to get the full question, but why have they gone for this groundings thing rather than just a traditional search query? Is it crucially different or my understanding from what I've seen is that it's not crucially different because in the example you just gave there it will be the topic commuter bikes that shows up very similar to how in search console in Google search console we would see the types of keyword phrases that were used and this is because being of course the answers that sorry the prompts the questions that people ask in AI can be so long the most recent data I saw on this was an average of 60 words. So absolutely huge. And what they're doing here is reducing that down to what the key phrase actually being searched for was. So to me, this feels very similar to what we see in search console when we're looking at keywords, key phrases. I think they've used the phrase grounding queries just to differentiate it from keywords in that way. And because some of these key phrases the AI is using when retrieving the content may be slightly different in length are not going to match up exactly with um data around Google search volumes for example that we have many many tools like Semrush and SE ranking to show. So, I think they're trying to signal that these are not like for like items, which is why they've got this new phrase called grounding queries. Or maybe they just wanted to make something that had a Microsoft branded name. Yeah. I mean, if you can own this kind of thing, you definitely would take the opportunity to do so. Yeah. Um, what are like your main takeaways from seeing this release? Main takeaways from seeing this release. I think it is worth dusting off your Bing web master tools to have a look to understand what the future of AI performance tracking looks like. The second takeaway is that things are going to change. We are going to be in a situation where there's more AI searches happening and there will be pressure to have insights from marketing teams to the big companies like Google and Open AI for insights like this. And the other thing is even though this is specific to Bing's AI summaries and Microsoft co-pilot AI traffic citations visibility, the thing that will be interesting is digging into this to understand if there's anything useful that applies in your broader search marketing strategy. So even though these are very specific examples, the potential especially with grounding queries to understand what queries commonly get searched in AI chat bots like Microsoft Copilot about your business is really first of its kind. Do could you imagine that this is also including the um query fan out queries as well? So for those who aren't familiar, perhaps you can better explain it than I, but there's like a a you you explain it. [laughter] This is a fantastic question. So when we use any type of AI search, so for example, AI mode or chat GBT, Gemini, Microsoft Copilot, what happens with our query is that even though we've put in just one keyword phrase, so best commuter bikes to take your example, Dale, what happens in the background is the AI can run two, three, five, 10, hundreds if it's AI mode of background queries. What it does is take that one query and fan it out into lots of other search queries that it feels are relevant to give the context that you're looking for. So with best commuter bikes, it might be looking at certain types of bike. It might ask questions about best bike models. It might ask questions about pricing, best value for money, reviews, all sorts of things, terrain type, tire type, even though you didn't specify those in your original query. And that's because what AI is trying to do is give the most complete and full answer to the user based on the intent of what they're searching for. They ultimately want to find the best commuter bike. But other important factors in that are going to be things like price, condition, brand and models, tire types, frame types, weight, all of those things that you might not have necessarily listed out in your original question or prompt. Okay? And I imagine some of that will be kind of brought into some of this reporting. But if you aren't really if you are reopening your Bing account and probably haven't done for a couple of years for some industries in some countries in the UK we don't really get a lot of high Bing usage. Um you may not find a lot of stuff in there and may not find that this report has much in it. Um what would you be using instead? Are you just relying upon search console or are you going another route to figure out like how people are looking for the the products or the services that your brand sells? This is a brilliant question. So firstly, I expect a lot of people might not have Bing web master tools set up. I would set it up because if there is some traffic coming via Bing or C-pilot, better to know and better to have the insights even if there's only a few of them. It takes about 48 hours for all of the data to populate once you set up um web master tools. And you can if you've already got your search console set up, you can simply log into Bing and then transfer across. So it uses the setup already attached to that search console account. So great firstly do that. Secondly, I think that the most reliable data is still going to be the data that we have around keyword research, search volumes from tools like Semrush, which are absolutely fantastic. Show us the average monthly searches on Google on traditional Google search results. That's still the most reliable data we have around actual search volumes. And there is no way to know exactly what people search in AI models. You can use that keyword research data that you would have had from SEO, from Google search to understand and model out what is likely to be searched during searches in AI platforms like chatbt, Gemini, Plexity and so on. The other thing I would recommend doing is jumping into a tool like Peak AI, which is an AI search tracking tool. If you don't have access, you haven't accessed it before, you can get a oneweek free trial. So, it's worth signing up for. But what you do need when you set up those tools is an idea of what prompts, what kinds of queries your business really cares about. So if you are a bike retailer, for example, best commuter bikes might be one of your queries. This should be informed by your keyword data. So you need that first. You want that before you even start the setup. And then you can add more context, more natural language around those prompts. And if this is all getting very confusing and you need to have a chat about it, please just reach out to one of us on LinkedIn or just reach out to our team and we'll have a chat about what AI search really needs to look like for your business. So, one of my favorite ways to like look for what prompts may look like that are being used by my customers if I can't like reach them directly and ask them is to just look through forums and subreddits. So, I'll just open Quora and Reddit and a few other places and just go to, you know, some of the main uh um subreddits or places and see what kind of questions people are asking about commuter packs and just see what what's reoccurring or in um Reddit, for example, often a subreddit or community will have like an FAQ area of like questions that come up a lot. Sometimes they'll have like a daily thread in some subreddits of like answer ask all your questions in here. Like there's a a barista or coffee roaster subreddit that I'm in and they have a daily thread for like people asking the same questions about like yeah what should my grind thing grind setting be on my grinder and how how long should my pore be? What should my ratio be? And the questions keep coming up. You can just compile those questions together and just add them in uh to whichever tracking platform that you use. And also, if you're in a business where you regularly speak to your customers or you receive leads rather than being in e-commerce, so you actually have conversations and you have customers self-attributing that they came to you via chat GPT or another AI chatbot, ask them what they searched for, they'll most likely remember something, particularly if it was terms like best cyber security companies in London. They're going to remember that they roughly put that into chatbt and are more likely happy to share. So you can usually get a couple of ideas, especially if you're regional or localized business. I think you'll often find that the customers who come to you that are really qualified have used localized searches in AI as well and they'll remember that they did. You could also like use your live chat on your website like they're often when chats do happen, you'll get like an output at the end. You can have a look through to see what kind of questions people are asking of your live chat or if it's AI based or human based. I suppose I never thought about it before, but when we used to incentivize people to leave Google business profile reviews, you would leave like a QR code in your reception or you try and say like you here's a Wi-Fi code and by the way, can you register? You could do things similar to that. You could incentivize your customers to share their conversation with you and say, "Hey, I can give you a 10% discount if you could share the conversation you had with me by using your public link on Chatty or whatever it is." There's probably a little bit of work that you can do there. You may not get high volume depending on what your business is, but the things like that. Like Charlie, you mentioned the other day about using clawed co-work to crawl through like a whole bunch of sales call recordings which pulls, you know, insights as well. If you just make sure that part of your sales team's prompts are to ask, "How did you find us? What did that conversation look like?" You don't have to be involved in that process. I suppose you could use well the the technique that you've used. Yeah. And in in so our calls recordings are in a fantastic um platform called grain which automatically pulls transcripts and AI generated summaries. There's plenty of other options other than grain. But if you have something like that set up with your sales team already, you can just create a folder of recent sales calls. The same if you have regular conversations with with clients, of course, it can be helpful for that. And you can use any type of AI platform provided that you have the right uh data security settings set up in your workplace as well. So you could use something like Claude Co-work, but you could also use ChatBT or Gemini or Copilot if it's in your workspace to put all of those calls in and have them review the transcripts, look for trends, and look for language trends. Really, really useful for understanding the types of questions that your customers and leads were asking very recently about the business. So a fantastic way to get some extra insights that could help with this. Yeah, I remember doing exactly that. I about two years ago we we were using grain at that point as well and I would take the transcripts from there like download them and then put them into a drive folder and then I then have to like drag them into claude or chatb whatever it is. Such a manual process looking back now compared to now it's like oh just going to click a button it's going to go in [laughter] work for me. Oh 100%. Right before we uh sign off, have you got any final thoughts or perhaps predictions um to share about you know what this may signal for Chatty and Gemini or you know this space over the next like couple of months? Prediction is that this is going to put some pressure on OpenAI and Google to do something about AI analytics reporting. And I think that what we're going to see is more and more pressure coming from the SEO community and marketers who want this type of insight and data. That's my first prediction. Uh secondly, I think uh a lot of people about to jump into Bing. So Microsoft's going to have a bit of a spike in user activity of being web master tools for the first time in a long time. Superb. Brilliant. Well, thank you so much for this week's uh information and sharing about uh Bing's latest release. I'm really excited about that. If you want to learn more about how AI search optimization works, you can head over to exposion ninja.com or go to our YouTube channel where you can find lots of videos and guidance on how to audit your brand within AI search platforms, how to create content and optimize the content you already have. Pretty much everything you could possibly think of or have questions about are going to be answered over there. So you can go to youtube.comexposurinja and find, you know, all that useful stuff. And I imagine we'll see you all next Friday for another podcast. Thanks and see you then. Bye bye.