How Deutsche Telekom Is Deploying AI Voice Agents to Millions of Customers | ElevenLabs Summit

Please welcome Jonathan Abrahamson, chief product and digital officer at Deutsche Telecom along with Stan Maseras from the 11 Labs go to market team. [music] Hi, good morning everyone. My name is Stan Masuras and I'm delighted to see so many of you today. So Matty started the day by talking about innovation. So things that we shipped in 2025 and gave you a quick glimpse of what's ahead of 11 in term of new technology that we're planning to release. Marco talked about safety. So innovation and safety wouldn't be nothing without AI adoption. And so that's why today I have the privilege uh to welcome Jonathan Abramson, chief product and digital officer of Dutch Telecom, the largest telco operator in Europe, one of the largest in the world. So Jonathan, thank you for your time today. uh going to be delighted to talk about uh some of the learnings that you acquired uh along the way but first maybe if you could tell us about your current role and the scope of your your position at uh Dutch Telecom. Sure. Happy to do that and thanks for the honor of being here with you. Um yeah so I lead uh the product and technology function for Deutsche Telecom um uh in Europe. Uh so you can think about that as uh 10 countries from uh Poland and Germany in the north all the way through to uh Greece in the south. uh and we build our digital platforms, our products, our ecosystems, our engagement platforms, e-commerce platforms um also our broadband and television um uh products um uh for the group. Um obviously in the last couple of years we've shifted and pivoted our focus a little bit into focusing on industrializing AI uh for an enterprise context um at DT a big focus for us and huge opportunity then that's obviously a lot of the work that we do with you guys. Thanks. Um, so with the 11 Labs team, we spent quite a lot of time. We almost became an extension of the Dutch Telecom team. Um, so I don't know if you're curious, but if you like to meet the Magenta team, they have a booth. Uh, so you'll be able to experience a little bit of the immersive immersive experiences that they built on top of our agent platform. Um but what have been very interesting for me from my vantage point is to look at how thoughtful you were with AI adoption when most large organization were still in the starting block looking at the reporting and analyzing everything you actually you know started to to to work on testing and now you're you finished 2025 and you're ready to scale uh at large scale um in 2026. So I'll be curious to hear about some of the deliberate uh this decision that you've made last year in order to kick off 2026 and to keep your competitive advantage. Yeah, absolutely. I mean it's kind of the question the perennial question for executives what's your AI strategy? Um uh so I think the thing about that ultimately is uh you know AI in the context of an enterprise um it's sort of AI is a sort of a proxy for human intelligence which means it's applicable wherever human intelligence um can be applied with which in a place like do telecom is lots of different places. So we have to be a bit focused uh and and sort of take our our time to make sure we're clear on where we're investing and how we're investing. Um we see ourselves in uh three broad buckets. Three ways that we think about um running AI at DT. Uh the first one is ultimately making uh a better telco. Um so this is uh the work that we do in customer service. Um like obviously the work that we do with you guys which we'll talk more about today. Um this is the work that we do in network operations um in marketing automation and and the like. So ultimately growth uh efficiency and automation is is that first bucket that we like to think about. The second bucket that we talk about is um work for our employees. Um we have 130,000 employees in Deutsche Telecom across the group. Um and I can tell you each one of them is as excited about using this technology in uh their work life as they are in their personal life. Um and uh there's a huge demand for and pull for this and it's part of being a good employer that we can provide these tools uh to our employees. Um so there's a big task around making sure that we do that we have the right tools and they're brought into an enterprise context um and they're connected to the crazy amounts of uh disperate uh knowledge bases across the group that we can make um for much more meaningful jobs for our employees. So first bucket better telco, second bucket um uh more meaningful jobs for our employees. Uh and the third bucket um is actually something for our customers. Um I think you mentioned it before uh Deutsche Telecom is uh has built a very successful business on the back of connectivity providing and democratizing access to connectivity uh across Europe and the US and it's fair to say that connectivity has had a pretty um impactful uh you know impact on on society. Um, I think we would believe that and I'm sure this audience believes that artificial intelligence will have as big if not a bigger impact on society as what uh connectivity has done. And we would like to play some role on behalf um of our customers as Europe's biggest brand and and Europe's biggest telecommunications company in democratizing access to this technology for our customers. Again, we did bits of work um with you guys on this and there's much more exciting stuff coming down the path, but we see we see the role that we do across across three those three areas. Perfect. And it's been really interesting to see some of the work done around dem democratizing access to AI with the partnership with perplexity 11 apps being able to create really cool experience in the my magenta app. Um so yeah, you can all check it out at the booth after this uh session. and the first pillar which is you know to uh to leverage AI in order to uh to deliver the best possible experience uh to your to your million of customers. So now we're going to switch gear. We're going to be talking about um uh how you using our technology in order to support uh your your customer base. So we're going to switch to a video. We're going to switch gear now please. So we'll be able to see uh the the Telecom Europe's largest telecommunications company is bringing 11 Labs voice agents to customer support available through the magenta app or directly via the phone. Let me show you how that sounds like. Oh, hi there. You're so quick to pick up. You prefer to continue in English? No problem. How can I help you today? Yeah, let's continue in English. Um, I'm calling about my Wi-Fi. I'm having connectivity issues. I can look into that for you. To do so, can I get your permission to access your router to run some remote diagnostics? Please do so. And also, can you speak a bit slower? It's quite loud in here. Thank you. Connecting now. H. Okay, I have found the issue. I am rebooting your router. Okay, you should be back online now. Amazing. Thank you. Oh, and one quick tip, Louise. If your Wi-Fi signal is weaker in certain rooms, a Wi-Fi repeater can help improve coverage. Would you like me to suggest some suitable options for you? That's actually a great idea. Can you send them to me via email? I have to run. Absolutely. I will send you a short summary of what we fixed in both German and English along with information about suitable Wi-Fi repeaters to your registered email address. Perfect. Thanks. Speak to you soon. Perfect. Have a great day, Louise. Quick round of applause for Dutch Lum. [applause] I think interactions is a super uh super interesting one. And I think obviously for this audience you kind of see below the waterline of actually what's actually going on there. So obviously a very sort of uh empathic uh you know interaction with with with a voice agent but it's it's the context switching so English to German and back um tool calling um and the upsell obviously very important as well at the end. So the fact that we can sort of stitch all that together make it seamless for an uh for a customer uh is fundamentally different from from what we've been able to do up until now. Yeah. Usually people will kind of you know just judge the quality of the voice but what has been interesting is that yeah the how thoughtful you were regarding uh specific workflows super complex workflows across all the different products that you support from broadband to TV to mobile to landline etc. But then also some of the work done about u the audio production of the voice keeping the official brand voice of Dutch telecom and moving it into the next century making it emotional emotive empathetic dynamic depending of the use case. I think we still got some work to do there. Um it still sounds a little bit like a a 1980s sort of game show host. We we'll we'll we'll improve that uh over time. But I think what you guys did is is take something which um was probably recorded in the '90s uh and and and made it somehow uh much more um prevalent. But I think we we have more work to do in this context. And I think what working with you guys actually does is sort of prompts us to listen to these things again and say we need to improve on that. Yeah. Yeah. It was one of the discussion we've had last week regarding uh you know the the voice for like okay uh we might need to think about working with our branding team to think how can we take it to the next level. Correct. worked well in a kind of IVR setup and now you know can we how can we scale this to you but it comes to the question that usually people ask is what's the difference between a good and a great AI agent uh in your world I think we've got a good one now we're not yet great um although I was listening this morning to um the new release you guys did on V3 um with the expressive mode I think that's sort of another game changanger which we're incredibly excited about testing um I think we think about the path towards um a great agent. I mean ultimately what a great agent for me is uh where a customer is actually preferring to talk to the the AI agent than than a human agent and we're not quite there yet. Um but we think about it ultimately in four different ways and that path to becoming uh great when it's our four C's. So ultimately the first one is capable um so incorrect so it's able to sort of understand knowledge recite it correctly um ask for clarifications if it's not clear. Uh the second one is calm. So it understands uh context. It can deescalate if that's required. Uh the third one is capable tool use is the is the gold standard for us. Its ability to sort of make decisions, not just answer questions, but actually take actions on behalf of the customer. And the fourth C for us is continuously improving. So all our evaluability in place to make sure that we're able to take a constant sort of feedback loop and improve the model uh and improve the application as a result of that. So super super you know lots of fun on that path. Uh still a lot more work to do though. Thanks. Thanks. And spending time with your team looks like you're always at the tip of the spear of innovation testing new technology. Um putting level apps aside but how do you how did you come to the realization that uh AI voice agents were at the levels that were suitable for Dutch telecom to deploy them at scale? Yeah, asked that question and usually people think it's like oh when we got to GPT40 that was the time and it really wasn't like that. Um if we think about deploying this sort of technology in an enterprise context in front of millions of customer interactions um the model's important um but the model's been good for some time. Um it's really the I mean we're not yet at the stage unfortunately where we can just plug our call center our IVR into a large language model and the call center goes away. We're not we're not quite there yet. There's a lot of orchestration. There's a lot of plumbing. There's a lot of um applications that we need to build and wrap around those models to make them effective. And that's all your eval. That's all your observability. As ultimately, if we're going to push something in front of um that c that many customers uh in a brand um and in an enterprise context like ours, we need observability. We need the ability to um monitor um uh and and continuously improve. So, when we have all of that plumbing in place, that's kind of when we started. Um we started in digital um much easier just in text. um a couple of years ago uh and uh sort of took a lot of learnings from there and optimized from there. Um uh unfortunately well the reality is in in in DT about only about 11% of our customer interactions are happening through digital channels. The rest are in voice and that's really where the huge value for us is now and really where our focus is sort of to make sure we continue to push that sort of experience in front of all those customer interactions. Thank you. And so we met the first time almost a year ago uh when with Matty and with us that you've seen on the the video when we came to bone to spend two days with you and and the team. So I'll be curious to hear so since that day so how is that has been to to work with uh our forward deployed engineering team customer success go to market uh legal team security team etc. So I'll be curious to hear uh how how did it go with you? You put me on the spot. No, I think um listen, it's been it's been fantastic working with you guys. I would say that on stage or offstage. I think the quality of the talent that you have and you're able to track is is fantastic and it's wonderful working with you guys. I think I'd sort of call out three things here. Um I think obviously having sort of I guess we have talent here to to thank for um this sort of FD model and I think it's working fantastically in this sort of context where we're talking about sort of this kind of technology in sort of an enterprise software environment. Um the first thing obviously is sort of shared goals and alignment around how we build and and both teams sort of working together to to uh to ship amazing products um and working towards the same goal. I think the second thing um is uh co-design um of of these platforms. I think um having sort of deep domain expertise on our side of how our platforms and systems work and on your side working together to co-design systems uh makes for a more uh robust integration sort of a less brittle integration. And I think the third one and probably the most important for me um is the fact that um it's a knowledge transfer. it's the fact that you know every extra hour that my guys get to spend with your guys um is uh uh is the knowledge is transferring and that that sort of goes beyond the end of the project and I think that's something that lasts. So I mean we're super happy with the partnership and obviously the ability to have your guys sort of so so deeply embedded with us. Thank you u mutual mutual feeling. It's been a pleasure to work with with your team. Um now I'd love to to talk about uh success metrics. Uh so it's great to test and deploy but uh we'll be curious to to to see if you can share with the audience without sharing all your insight secrets but how are you planning to measure success of the the fleet of agents that uh you're deploying? Yeah, I mean I think in the short term uh we sort of look at look at it differently to how we think about sort of success in the in the longer term. In the short term it would be the metrics that you would expect. I think we're all sort of tracking um at scale at the moment. So it is um sort of containment rates with quality sort of avoiding sort of escalations. Um it's the security um metrics. So you know policy violations um uh hallucination rates and that sort of thing. Uh and it would be things like friction uh so uh you know a customer's uh sort of refusing to speak to the bot. I want to speak to a human that sort of stuff uh which is in the first weeks. I think longer term though obviously the the intention here is not just to answer the customer's question but actually solve the problem. So it's much more around sort of end to-end um workflow completion. Are we able to sort of make sure that the customer's problem is being solved or the the change that they're asking for is actually happening by the bot um uh uh and uh and being handled sort of end to end. We look at things like customer effort score and NPS at that point as well as well as cost um and and platform sort of capabilities there. So I think that uh we look at it in those two formats but ultimately the end goal is um you know customer completion not just good good good conversations. Thank you. Um yeah what what we've seen happening is that initially some of our partners and customers were uh using our platform just for deflection. Um we get you know let's say thousand of calls per day instead of putting the customer in front of a in front of a human agent we want our AI agent to deflect the conversation. So it started from there to then actually what is the time to resolution what is the win rate what is the quality of the answer what is the impact on NPS and seesat and this is where we see you know companies you know like yours really really skyrocketing because you know there specific use cases which I love to touch on there specific use cases where um customers would prefer to speak with an agent now uh because you know time you know it's super timesensitive but also they want to use it in a way that is you know very personal to them where they can share uh openly without feeling judged So I'd be curious to hear yeah what what were some of the surprises that you discovered on the way? Surprises I mean I think lots it's been it's been a quite a long journey. I think the biggest surprise for me is the expectations that we place on our human agents to disambiguate and understand uh the complexity of the businesses that we run today um in an enterprise context. Um I think when we started this we it became very quickly clear um that there was no single source of truth of how we should be uh interacting with customers. It was sort of these knowledge bases were split between training manuals um and portals uh and sort of weekly emails from the from the team leader and even tribal knowledge. Um and that doesn't work um when you want to give an instruction set to a stochastic technology like a large language model when you want deterministic outcomes. Um so it took us some time to get that um all organized. Um and I think we're up past now 15,000 SOPs which um we've have to um sort of create and then put in Girkin format so it can be sort of machine readable. Um uh and that was that was a big task and that was something which I I guess um you know when you sort of play with the models and you do your PC's you don't see um but when you want to take something to production um and you want an experience as good as what a human agent can do um uh that was the big one. I think beyond that uh you know there's always an API that's missing um there's always a data set that's that's not available. Um so it really sort of an iterative process as as as we roll this out and sort of get to get to complete scale. Okay. Thank you. And maybe I'd like to uh touch on now um the future of human versus AI agent experience. I'd be curious to to hear what's what's top of mind for you and how do you see things evolving not only 2026 but but beyond uh in the CX customer support space. I wouldn't say versus um uh I think uh where we are today is very much the sort of a hybrid node. um we have uh AI agents uh handling uh the more mundane or repetitive L1 type interactions and as things get more complicated or more heated um uh they get escalated to to a human agent um I think again with the kind of technology that you guys are displaying now and the kind of things that we'll start building into our our own um fragmenta uh bots at Deutsche Telecom I think that will uh quickly get addressed and sort of humans will be um I guess elevated at that point to more around sort of quality control um and journey design uh because I think that still humans will be will be sort of required for that but I think more and more we'll start getting to a place where the quality of of the interactions that you can have um with very good voice models uh and underlying sort of reasoning capabilities the large language models I think we we're on a pretty good path it's um been interesting to see yeah how the especially customer support agent I don't know if some of you have been a support agent uh I was when uh a few years ago ago when I was fixing printers and photocopier over the phone. Uh it's a very important job and it's um it's it's very hard you know when you have to deal with customer every single day uh repetitive task you know uh repetitive question. So that's why what we've seen happening that as you just shared which I find interesting that this role is being elevated. So now they will only focus on strategic questions that will require a sophisticated answer but also uh they will start to be more hands-on on developing a fleet of agent optimizing the workflow the knowledge base uh monitoring the quality of the answer and to um create almost an extension of themselves. Um so it's going to be interesting to see how things evolve this year when uh more and more companies like Dutch telecom uh bring to scale uh AI agent. Absolutely. Um so now I would love to talk uh about um what I shared in my introduction that uh you've been a fast mover maybe not as fast as you you wish you look but uh we we think Dutchcom is one of the fastest large scale enterprise touching millions of people uh to experiment deploy and uh put in production at scale uh AI technology uh what's what the secret sauce how do you operate internally because I think it could be great learning for the audience I think Stan you're being very polite um and and maybe we got the wrong benchmark. No, I think I mean thank you. Uh the the um the reality is um I it's constant angst for me um around uh the fact that you know you just read the press or listen to a podcast or uh keep an eye on on sort of your your Google alerts every day there's another release another announcement another breakthrough at the model at the models layer at the infra layer all the way through the application layer. So it it's you constantly got this s of are we moving fast enough? How do we move faster? um how do we continue to ship um and obviously working with partners like you guys I think it's it's sort of a big part of that in the context of a large enterprise um like do telecom is um it's super important on uh as leadership uh of sort of investing heavily to reduce uh operational overhead and and and coordination overhead um unlike a startup there's more people uh there's more opinions um uh there's more um objectives um there's more targets um and they don't always align. So focusing very clearly on what the objective is that we're trying to solve for uh and then getting broad alignment across uh across the organization um super important. So I think that's that's a big part of it is making sure that you keep alignment in in in the context of a big organization. I think beyond that um slicing uh deliveries anything that um takes longer than 3 months um to go from sort of ideation to to in front of a customer gets deleted or cut um uh down. we we don't have in the context of a large organization the kind of patience um or or ability to sort of stay focused for much longer than that now. So uh if it's if it's anything longer than 3 months we we we don't start um uh yeah I think that the probably the biggest learnings from my perspective. Great. Great. And maybe last couple of questions, but um is there any specific advice you like to share with the audience, especially teams, uh that might be, you know, large organization like Dutch Telecom, but also smaller teams? What would be the one lesson that you like to to share with them when it comes to deploying u agent KISS and especially deploying voice to your to your own customers? Lessons learned. Um now, I think um you actually asked you told me that you were going to ask me this question. Um so I I prepared on the on the SL on the on the flight on the way in. Uh I I wrote write down some thoughts to make sure I was going to hit them. Um I think uh the first one is uh around um alignment. Um, and it sounds I guess super obvious uh when it comes to um a company but actually in in this in a in a company the size of Dutch telecom or any large corporate where you have hundreds of thousands of employees uh multiple lines of business across 10 countries uh B2B consumer um multiple lines of business like a you know mobile fixed television broadband um but one product team uh and one engineering team to go and solve for all that it can be quite hard to keep such a broad organization uh on the same page. We invest heavily in DT um to make sure that we have super crisp outcome focused objectives for the organization. We just have five um and and that took I can tell you a long time to get 130,000 people onto the same page for just five things that we focus on. Uh it's super important because that keeps us um focused on what's important and sort of kills the pet projects and and and and make sure make sure that we can ship. Um but also changes the way that we work actually. um in DT there's a in my team there's a sort of required reading um by it's a Theresa Torres book I'm not sure if you read that one called continuous improvement uh and continuous learning and I think um that uh it talks a lot about um these product trios so atomized teams um three people a product person an engineer and a commercial person sitting together and and having a a core focus around um a small number of topics one topic ultimately to to solve for that's that's been the the the biggest sort of investment in learning again sounds obvious this in in the context of a smaller company and the big company I can tell you that that's the it's a big change. The second big learning for me um uh is uh we made our focus um tool execution uh not uh conversation quality. Um I think the models uh and the technology that we're able to get from um partners like you uh is solving very much for the second thing. Um but it's very much on us uh to focus and solve for that first thing. Uh and again um it's very easy in the early days um of making sure getting to a place where um you're shipping nice conversations but we're not ultimately solving uh the customer's problem and then you see customers calling back because you didn't um you didn't didn't didn't solve it to end. So that that was a that was a that was a big one. I think from our perspective also um our moat uh in this context is our learning loop. We've invested hundreds or thousands of hours um across the team in optimizing these platforms, understanding our business better, writing those SAPs um and transforming them into into a format that makes sense. And I can imagine if we were starting again from today um that would be uh a big task that was still ahead of us. Um and um AI hasn't solved that part for us yet, understanding our business and and documenting all that. So I think that time investment is is is is super critical and super important. Yeah. So they're they're the big learnings from my perspective. Thank you. gonna be greedy. One last question. Um, but yeah, what's next for for Dutchcom and for your team when it comes to uh AI adoption and AI innovation? They're asking for secrets. Um, no, I think uh um you'll hear more about this. Um, we're in a shared room. You guys can keep your secret. We'll everybody's trust you guys. So, I think um you hear more about it in MWC in a in a few weeks. I mentioned before at the start around this idea of uh democratizing access to artificial intelligence for our customers. We take that seriously. Um and I think we've got a role to play there which um we have a unique sort of opportunity there and it's um uh around how we interact with people which is through voice. And if there's a control point that Deutsche Telecom or telecommunications companies in general have, it's still the voice. Um there's 800 billion calls in Germany per year um on our networks um voice minutes. Um so there's still uh a huge amount of interpersonal sort of interaction. And I think the challenge that that we face today um more generally is that this technology um artificial intelligence is generally sort of trapped uh in an app in an app draw somewhere uh and uh isn't isn't sort of uh immediately available. People still have to think I mean maybe not this room but in general uh people have to think about I need to sort of you know open up my chat GPT I need to open up my perplexity um to to do that. Um and it's sort of controlled access to that is sort of controlled by um two West Coast uh um hyperscalers and that's not right. Um so I think um uh the idea of um pulling this technology out of the app drawer uh and into our network and having it much more easily accessible and having it just on on tap like when you're making a phone call uh is something that we're super excited to uh to um uh to to announce shortly. um and looking forward to doing with you guys. Um so it's a big part of what we see ahead of us. Amazing. Well, big round of applause to Jonathan. Thank you so much for your time. Thank you so much for sharing so much knowledge. It's been a pleasure to work with you. Thanks everybody.