Episode 13 - The Thinking Behind Ads in ChatGPT

Hello, I'm Andrew Maine and this is the OpenAI podcast. Today we're talking to Assad Awan about ads in chat GPT. How they'll look, who will see them, and how the company will preserve user trust. Ads are shown to people who are on the free and the goier for pro and plus and for enterprise there are no ads. Is creepy okay if it is good? It's not. We are in the business of trust. So I think if we have to say what is our core business, it's like to win users trust. So from a consumer point of view, why ads? Why now? It [snorts] goes back to our mission which is bring AGI to all of humanity and um to benefit all of humanity. Uh so when you have a consumer product which like you know 800 million plus people who are using this then how do you take the best version of that product to everyone um uh and and ads is one of the most proven models to be able to do that for consumer products. uh and I think the other part of that mission is how do you benefit all of humanity which is like you want to take the best model you want to give the highest limits usage limits to people you want to uh for the ads to be actually helpful both to the users and the businesses as well um so so I think it it's a very natural fit for a company whose ambition is actually to take the best AI to all of all of humanity it's a very interesting decision because on one hand you could say hey we're going to take what we perceive as the high road and say we're not going to do ads, but also we're not going to give a really good amount of usage for it and limit that and sort of maybe use not the most capable models, but sort of, you know, say take that approach versus embracing it. Yeah. Yeah. I think like if if if the goal is to truly democratize access, I think ads is a good model. I think maybe what is hidden in that statement is can ads be bad and and the reality is how do we think about the principles of ads? Uh how do we actually set a really high bar for what ads should be on this platform? how do we make them actually useful? So when we were starting off we we thought like hey what would be the core principles that we would announce to the world that we would be proud of that we would stand behind and as a result create a really great product. Um so so just to give example of the principles like number one the answers need to be independent from the ads both visually but also in how the models are trained and how the system works. Um so that you can always trust the answer like the whole product JBD product is based on trust. So actually it needs to feed into that. Uh [snorts] the second is your conversations are private. If you have a sensitive conversation that will never have ads in it. Um uh and and the conversations are never shared with advertise. So while we do the matching between the best ad that can be a useful thing in a conversation, the advertisers don't get to see that. We do that matching internally. And then of course like you know as as you introduce ads a big question is how did you know about this data? But like that's the difference between the user trust and just like doing something which is relevant to the user. Um uh and our goal was how do we make something which users can transparently understand how can they control and and we can go into that because I do think there is a high bar to set over there because you could have some transparency some control which most products have but what would be a really good version of that is something that we've been thinking about and finally once you add ads I think you have to set the incentives for the teams for the company in a way that actually continue to focus on user value. So you don't want to just like get empty time spent on the platform. You want to build a very useful product and automatically one good ad is good enough actually. So we don't optimize for time spent um on on the platform and focus on the user value. So these are the principles. So I think like um so connecting this back to your question is like how do you why why should we add ads and how to do it? I think a part of that is take it to all of humanity and that's the best business model to do that but prevent all the negative things that can happen if you're not doing it thoughtfully and I think being upfront with our principles being very clear with that um is is how we're starting and then how we will test how we will improve and how we become kind of a learning organization with respect to this I think that's that's our goal so you said basically there's going to be a separation so if I'm talking to chat GBT about like hey I want to start drinking smoothies and stuff. It's not going to all of a sudden blur out like well here's a blender you should buy. Absolutely. Yeah. I think like so both in terms of what the model knows. The model doesn't know whether an ad is there or not. If you ask it like hey what this ad saying it'll say I actually don't know. Uh but you can actually press some buttons and add it to the model if you want to ask. So it's totally totally totally whatever is being displayed in the ad space the model has no idea that's in there. That's right. And I think in in and both visually as well so that the user can very quickly say hey this is the answer that I got from the model. And then there's a bottom uh banner which has ad in it uh which is very clearly distinct. So visually also you don't confuse that. Of course we will learn how that experience evolves but the goal is to both keep the system the models very very separate and ads is kind of downstream of okay yeah that's I think that's a very important distinction. I think some people have sort of kind of tried to sort of spin that there's some sort of collusion between the ad part and the model part but you're saying the model's completely separate. Yeah. So it's interesting. So, as I'm having a conversation, something might come up and I can then click and say, okay, tell the model, hey, I saw this. Then it knows what's going on. That's right. Yeah. You have to go like in the in the first experience, you'll explicitly have to press a button is like, ask chat GPD about this ad. And that would be as if you took a link from the internet and asked a question about it. So, it's almost the same. We don't want to make that experience harder. But, but if you say, "Hey, what is this ad talking about?" It'll say, "I don't know." It's easy to start now and say, "Oh, yeah. We're going to do the great thing. We'll do it right." But 10 years later when there's an entire division in charge of ad revenue might say like well do we need the wall between the model and the ads. Yeah. Yeah. I think like maybe there there's multiple multiple angles to this. So one we are in the business of trust. So I think if you have to say what is our core business is like to win users trust and give amazing answers to the question that they're asking that's in the in the consumer product side. And of course on the enterprise side trust is everything which is like you're entrusting us with your most important data. We need to of course make maintain that. So, so because I think like that the ambition and the vision is so expansive. I think trust is the central point of it. We want to have devices which are helpful for you. Uh if we truly want to be your best personal assistant, then you need to be able to uh share your most uh important information but know that it will be um uh uh dealt in a way which is like how you would treat it yourself. Uh so I think our business model is trust. is very different than many other scenarios. If you're just doing like kind of a transactional stuff like again you you gave the question an answer came back and that's the end of it like a search query I think that's that's okay but it is not a long-term relationship. I think if you think of uh content discovery certainly I I mean like it it is just pushing things uh and and uh trust is not a core component of that for us. I think the whole product whether it's enterprise, whether it's consumer, whether it's anything uh devices in the future, they're all centered around trust uh trust and um um so so for us it's kind of imperative for others it could be optional uh and I think different companies are known for different things and we do want to be known for trust and so so I think like connecting this to the question which is will we drift I think you can't drift when the incentive is set up to be the best at this and this is the goal uh that we want to achieve. Uh everything else is is there to support that vision but the Uber principle is trust. Open has a very huge number of people using the free tier and then there's also paid subscribers and people who do that and pro users whatever. How are ads going to play out across the platform? Yeah, so ads are shown to people who are on the free and the goier. Um and for pro and plus and for enterprise there are no ads. Um and and I think like that's that's an important thing like the context in which which the company operates is actually like multiple missions which come all together to bring bring AI to everyone which is when enterprise use it that's a very specific context there is no ads over there and there is a specific business model around that which is very powerful uh for subscribers who want like you know the best like you know highest limits and very advanced features I think that also works but for a lot of people a lot of consumers the best way to do that is to have um have high limits and free usage uh and then add ads which are actually useful. Yeah, I've heard people talk about you know part of the goal of this is to avoid making the free tier just like the most limited thing available. Absolutely. I think like that that is that is the that is the most I think frustrating things for a lot of real users which is like you ask five questions and then it just stops in in other businesses right we we I think like we want to grow that a lot more and I think it fits with our overall goal um that uh higher usage limits is better uh and how do we fund that and be practical about it so going a little bit behind the scenes how are these decisions made like who's in the room talking about this yeah I think like this is a this is a good opportunity also to talk a little about like overall like there is a company culture and I think different companies have different cultures which results in different products and our our our company has like this DNA of a research team right uh so so we have much more I think like rigorous debates rigorous understanding of how should we make these principles how does incentives work how does the the model of this going to work in a way that it doesn't get corrupted later on so we have had both a lot of debates on that which actually resulted in these principles which resulted in this rubric with like hundreds of roundts with like folks around the company on different areas not just like working on ads but everyone on every different part of the company uh giving feedback to create uh these principles then we convert those to a very simple rubric I think like the rubric is user trust is the most important thing user trust more than user value which is then more important than advertiser value which is more important than revenue and I think while this seems very straightforward uh it's actually a very very very I think uh in-depth decision. So, we can go into a little bit this like user trust more than user value. Uh a good example of that is uh if I showed you a really good ad but and you liked it, you clicked on it, you bought something but later on you ask the question was this app listening to me and is the mic on? That that's not user trust. You you probably did provide some value. So for us our goal is like we we we cannot have that. The users need to believe and understand and control what's happening. So that's just one example. But I think once you set that right rubric up then even bottom up the team thinks like that. Uh but of course like you know as we have different decisions at different level I think we have a pretty rigorous process on how we uh discuss privacy within the company how we discuss safety within the company and there are very very clear forums for that and then of course like as we as we make decisions at leadership level we one go back to the simple rubrics because although the rubric is simple it's actually pretty pretty in-depth and actually is very discriminating if you think about these kind of questions. is like should the ad be so good but the users don't know where this data came from is creepy okay if it is good it's not so I think uh uh maybe it follows from there what am I going to see on my end what kind of controls do I have how's personalization going to work so I think like a big part of actually delivering really good ads is allow personalization so there is so when I say uh I want to uh go on a trip to yuseimity and then it shows me camping gear because that's what I like to do um but of course I Think the flip side of that how do you gain user trust which is like how did you know about this how did you learn about this? So one is transparency aspect which is like you can see what is the data that we have on you which is being used for ads. The second is the controls um uh uh which which let you see say which part of the data from your past chats can be used. Of course sensitive chats those are never used. Um and uh but you can clear your data which actually nobody else does which is kind of a crazy concept is like you can clear your data so we don't know and we won't use that. Uh you could uh say don't use my past chats if that's what you you care about or you could say turn off personalization fully. Um uh of course there is the other extreme which is like I don't want ads. That's a form of control and that's where I think upgrading to the pro plus version to completely stop ads is also there. So I think all the way from the spectrum of like I really care about this. I don't think this is the right business model. Like pro and plus is the right business model. Um for like hey I I don't know what we were talking about yesterday. I'll just clear my history. Great do that. Uh or it's like hey I'm more comfortable with like you know clicks on the ads being used but not my past conversations. You could actually do that as well. Of course people will learn hopefully experience like how it improves their experience. we have a very high bar in how we use these things but in the end the users need to know and be able to to control that what will be you know the the kind of the expectation for how many ads I'm going to see or how often is this would come up maybe the Uber principle still goes back to in that context is there a good ad to show which is useful if it is not we'd rather not show you anything in fact like you know as we roll out this test you'll see that there will be very few ads because like you know we want to be both conserv ative and we want to learn how to uh where to insert those. But uh but the the principle is a little bit more around is it useful, is it helpful, does it add uh to to what the user is doing. Uh and can we actually show a really good product as well. Um so so keep the quality of the the content very high as well. Keep the quality of the ad very high as well. Keep the relevance really high. If we can't find a good match, it's fine. We don't need to show an ad. You mentioned that sensitive conversations. How do you know what something is sensitive or not? So that's actually one of one of the big strengths of open AI. I think like both for our organic work and a lot of research in the company has gone into defining what sensitive is like it is health politics like violence like many different kind of verticals u very very in-depth uh definitions of that and then of course using some of the best models to actually predict and understand the conversation and saying marking it as sensitive or not. I think like I've actually never seen such high precision in any product so far in my career. What we have been able to build over here by taking in those policies. There's a team which works on defining those policies very very rigorously um uh and then actually also um sharing them with internal external partners for review and then of course the enforcement that comes from the prediction system that actually says hey like this is matching this policy so don't we've talked a little bit about this but I'd like to touch back on this again design so where is that going to head what are they going to look like um as we were designing this product I think like of course we set up a very clear principle that the that um uh the answers are separate from the model and then the question was like how does that actually look in the product um and on that spectrum on one side is like how do you make it look native so that it's not jarring um and on the other side there is a question which is like hey how can it be very clearly separated out um and and I think you can debate with on both of them and there is values in both of them and we start we we wanted to kind of set up the experiment in a way which is um that we can learn as we go so We take the conservative option and still keep that principle in mind and as we learn through uh building the product to getting the data um evolve it but but the idea still is how can we maintain that principle of the answers being very clearly separated from the model uh from the ads um and having a very clear understandability and visual distinction. I do think we will evolve the formats and I think they will get even more useful and better over time but that principle is constant and within the options that we had we started with something which is um which is clearly uh separated out and conservative. So, you explained kind of in a technical level how there's a separation, how the model doesn't see it, but also for guard rails and stuff, and I think you mentioned this before, but if I'm talking about, you know, you know, saying like, "Hey, I'm afraid of this trip." And it's like, "Well, how about some life insurance?" You know, uh, that's not going to happen. But how do you guys put in guardrails and how do you decide what's appropriate ads and what's not? So, so maybe there's two questions in there like what's appropriate ads or not and which context is a reasonable one. Um and the second is what are the controls in place so that like you know over time this doesn't this doesn't dissolve. So I think maybe a part of announcing our principles and being very clear internally for our rubric was to actually set that up in the first place. Then automatically a lot of the governance within the company how we make decisions follows from that uh onwards. So it's like hey I want to make this change to the product. Do this fit with this principle do they fit with this rubric that we have already set up. Uh that's the first pass. uh I think the sensitive context is is something that we take very seriously as well. Very simple things like you know conversations around health um or politics or other contexts where where uh their ads don't fit and that data is not going to be used for making ads like even even matching ads you just filter it out. So the first layer is really see does an ad belong here if it does and it can be helpful and additive then add it. I think like this goes back to the principle is like actually you don't ruin neither for users nor for businesses by showing many ads because you don't want advertisers to pay randomly for impressions. You don't want users to see too many ads. You want to share the one right ad and being one of like you know the best AI companies. I think that's hopefully something we'll do really well. Every time we do an episode we get a few people who go in the comments are like no ads, no ads, no ads. Now is your chance to talk to those people directly. Yeah. I think I think like in in some sense the when when people say no ads I I feel like there is a perception and it's not wrong that uh because I think like maybe how how the industry has evolved that there is some suspicion around how this works. So I do think it is kind of incumbent on us to come up with better principles better clarity better rules on how we're going to do this. I think there is like we like again this whole ads industry if you think about the online ad industry is like maybe 20 years old compared to many other industries which are hundreds of years old. So I think maybe we are in the third inning of this where we are saying okay we have learned from all of these questions and problems that people have. I think when people say no as I do believe that they have valid questions and concerns around privacy the it's on us to do a really good job to earn their trust uh through better transparency through better control through uh through building that is also delightful um uh I think there still be skeptics and then I think we have a way to upgrade because I think that's a valid choice as well uh but uh but enabling really good ads with good principles I think it's possible I think a big part of it is having really strong AI to power these ads also so that they are actually useful and then as a result bring this product to so many people without with with higher limits. Some of your competitors have been having a little bit of fun at the idea of ads. Yeah, I I think like um uh different different companies have different missions. Our mission is to take AI to all of humanity and and um of course we have different context. So we have the enterprise business, we have uh you know our subscription business and we have a very very huge consumer business using our product. So I think within that context we need to serve each one of them. We will have a really robust enterprise business and there will be no ads over there and then we'll have a very robust consumer business and ads will help us grow uh within that. Uh so I think if if that's not your mission maybe it doesn't make sense but our mission is to to actually build in all of these contexts and we believe they're all actually related uh in in how we build the best AI and then actually take it to everyone. And I think the good part is that we have different verticals in the business line. So it's not just an ads company. There are some companies which are purely just ads companies and there the incentives are actually different but I think we have a much more holistic view on on this and also when you're not serving hundreds of millions of free users it's easier to sort of say we don't have to do this I don't think it's like a vision vision which is which is set in abstract this is truly a vision which is like how does AI actually help people and if if there is like this elitist view that some people get to use it and some don't get to use it based on who can pay I think like that itself is a pretty fork in the road in terms of how AI can be valuable to people. Uh, and I think our position is pretty pretty much like everybody needs to have access to the best AI. I have friends at small businesses and they are always trying to figure out how to promote themselves and do that. Could you explain from that point of view like what it's going to be like for people who are actually trying to reach new audiences through ads? Yeah, I think this that's such a good question. like literally um uh I have a few friends who started this um e-commerce company um selling shoes and uh they did almost everything on their own like the founders which is like uh go to the factory get this done get the logistics done but when it came to ads they actually have to had to hire like three performance marketers to do the work because it's so so cumbersome so analytical if you don't do it right you could end up wasting a lot of money so I do think the vision has to be where almost as easy as you were prompting nowadays for questions. You could say my goal is sell these shoes more in Midwest and go and then it comes back. It's like hey I tried some experiments and I think this is the right bid given your price point. This is the right way to doing that. Do you want to spend more money on this? And then you continue that conversation and almost become an agent for that. But today literally a small business has to hire performance marketers which could be one of the biggest costs in some sense actually like you know just that cost of running ads through through that is actually one of the biggest costs in there. Um uh which which of course then makes things more expensive. So I think the vision would be that it is as easy as just steering and telling what you need from your business. So describing the what but not having to think about how it will work and how many campaigns and how much dollars and everything else is like hey I want to spend this much I want to grow my business this much these are the constraints and um and and ads are created and run to match your constraints in some sense. Yeah, it's it's a very interesting way to think about it because auctions were uh revolutionary. The idea that you just go in there, I want to put this words out there and pay for that to do it. But that created an entire ecosystem of all the sort of expertise and stuff that you have to do. And it's really hard for small businesses to try to play in that space. Yeah. I think as as as the competition on that increased, I think like the the people who had more time and money to spend on optimizing that and analyzing that data and then running the best possible ad got the benefit from that versus like if I didn't know that like hey actually um I think I I gave an example of an actual brand which is all birds. It competed with really big brands on shoes, but somehow they found that every designer in the tech company is going to love my shoe and and finding that niche and then actually being able to create your creatives, your message to focus on that made them win in this like if you go in Silicon Valley, you'll see allirds everywhere because of that. So I think like but that I think is not accessible to everyone. If you are very analytical and you have a whole team of people who think about that, you could do that. But theoretically the best products can come to life if we can find out where the right niche distribution for it is and really go go for that. Um I think an other story on this is like that there is this company which creates a vegan ram instant ramen which I love because I don't have to feel bad about eating it. Uh but it's such a weird concept vegan instant ramen like I if I was just thinking about it without knowing about this company I like this can't exist. who will want this but I think a really good product can help you find and discover those niche audience then you build a really good product so it I think enriches everyone's life from that perspective but enabling creation of those products selling of those products maybe it's not a multi-billion dollar company but that's great that's still it's like a really big SMB uh which is growing and it's sort of these people who who care about that very specific problem uh so I think it it really enriches people life if you are able to create products for these niches what Does this look like in the future where we're using things in a more agentic way? How do ads even work 10 years from now? I think a next step would be more actual conversational ads where you could truly kind of understand what this product is about. The next version would be can it work behind the scenes and actually aggregate the best discounts and best deals and the best version of the product. Like for example, if I I I know that I like ramen and let's say some somehow Chad GBT has understood that preference of mine, then it could find that for me. I didn't even know that that product exists. Then in behind the scenes, it could actually say, "Oh, actually I found this vegan ramen. Maybe that's something that's valuable." And of course, uh there is a marketplace where somebody could say, "Hey, help people who are like this to discover because because discovery goes from both directions, right? like of course I'm searching for something and then people want me to discover something and there's a match between those. So so I think it will be more agentic but uh uh in the future uh but at least the current modalities I think we start from there improve it um and make it relevant make it controllable understandable trustworthy and as I think the systems evolve the native the organic products evolve this will evolve as well with that. Excellent. Well Assad thank you for explaining this and uh look forward to seeing what's going to happen next. Awesome. Thanks for having me.