What's New in VS Code: Remote, Permissions & BYOK

Hello. Uh, I'm Retant. I also go by VB and I'm a dev on the VS Code team. Yeah. And my name is Justin. I'm also a software developer on the VS Code team. And this just in, we've got a bunch of new features in VS Code in the last couple of months and wanted to talk to you guys about some of them today. So, yeah. Um, couple things. First of all, the agents window. So, you guys might have heard about this, but we've been working on this new agents window. I believe it released a couple weeks ago. But the main thing, one of the main motivations behind this is the chat in the editor in the sidebar in VS Code. It's been a pretty good thing for the last couple of years. But we noticed that, you know, it's it's we're getting into a world where we're running multiple agents at the same time, multiple sessions at the same time, even multiple repos, different workspaces, and having just a single chat in the editor may be not enough for some people's workflows. And before because it was so editor- ccentric, it was built into VS Code itself. Um, you had the editor on the left, you had the chat panel on the right, but you only had a little bit of a you just had that chat widget off to the side. And you know, this editor work, this editorcentric workflow is kind of, you know, people are writing less and less code nowadays. So now we have this agent window. It's a dedicated tople surface designed to be agent first. And this is what it looks like. We've got the sessions list off to our side. We have a bunch of different sessions from a couple different of my workspaces here. You can see I have a couple different workspaces here, a couple different sessions. All the icons on the left here um mean a couple different things and I can maybe talk about that a little bit later. We also have our changes and our files over here. So, we can go ahead and look over here. We've got all of the files in this current workspace here. If I change it, it'll change to the different files just like it does in VS Code. And it also does everything that VS Code can do as well. We've got our terminal. I can go ahead and open the terminal here. And I can navigate through this as well. And yeah, so I can also look at any changes that I might have. In this case, I actually don't have any changes, but that's something that we'll be able to see as well. So once again, it's a place for you to basically do everything that you wanted to do in VS Code, but agent first. Um, it's one place to see all the agents that you've kicked off, whether it's remote, whether it's local, whether it's cloud. And because it's chat first, you know, people will stop stop asking when they accidentally hide the chat panel in VS Code, where did my chat go? It's right here in the center. It's the main citizen in this window. Um so you know there are still a couple gaps that we have um when it comes to the agents window and one of them is regarding the loop between the user and with like working with agents kind of wherever they want to. So now I think we released this so I believe it was last week but the agent can attach to remote machines over tunnels and SSH and that's something I wanted to show off here as well. So to kind of like kind of peel back the curtain and talk a little bit about um on the surface level about how this kind of works. Um so basically every agent session is actually driven by a small process called the agent host. And you can see here I can go ahead and switch this into local agent hosts. And in this case, the agents window here, this is the UI, but behind the scenes, the agent host is the thing that's actually running the model loop, executing tools or um it's the one that owns the workspace and then ultimately streams everything back to the window here. And because AHP agent host protocol, it's just a protocol, the host doesn't really have to be on the same machine as this window. So it makes the remote story really, really, really clean because we're able to do things like close this window and I can open it up somewhere else. So to kind of show what that looks like, I'm going to go ahead and start off a request. I'll just start with something simple. I'll just say like uh hello GB dev days. And I'm starting this once again in the local agent host. Cool. Great. The all good. So you can see here it creates an isolated work tree and it says hey. So you can see here I have this little radio tower here at the bottom and it says allow remote session access. So I'll go ahead and click on this. It's going to flash a little bit now that it's yellow. That means it's active and you can see here now it says remote session access enabled via tunnel. You can see this is what the output looks like. Basically it created a tunnel that I can connect to on any device now. So, I'll go ahead and go over to here. So, now I'm over here on I'm in Microsoft Edge. Um I'm on insiders.bscode.dev/ aents. And this is just an agents window. Um but on VS Code.dev. And I'm going to go ahead and search for that tunnel that I just opened. And this is just one of the entry points into um having a remote session. So you can see now it got connected. I can go through and select a couple folders and basically this is my entire desktop. I hope I did not leak anything crazy there. Um yeah, so there's one of the main entry points, but one of the really cool entry points as well um is on mobile. And how that can also work um or I guess one of the other entry points to kind of show off what mobile would look like is with code tunnels here. So I'm going to do code-insiders tunnel. Um, if you're working in VS Code stable, you would end up doing code space tunnel. Um, either works in this case. Um, I believe this one was already set up. So, you can see here I already have a tunnel name. Um, it's working on this version. I can go ahead and go to this tunnel via this link or um, I can connect it with my phone. So, give it a second while I do this cool screen mirroring thing. So, this is actually what is on my phone at the moment. I'll go ahead and search for this host. So, it's going to search, search, search. Um, and cool. It connected to this one, which is the tunnel that I have open over here. I actually don't want this, so I'm going to kill this one for now. Go back over here. And there we go. We've connected to our devs tunnel. And you can see now the session is running. open a new client on that specific channel. And that channel is this here. And what I want to do here is I kind of want to show off um I kind of want to show off what that looks like. Um if I have both VS Code um agent window as well as mobile connected together. So over here, same thing. I can go over here and click on tunnels. In this case, I already have the tunnel connected. So I can click on this dev days and have it reconnect. Give that brief second and boom, there we go. Both our phone and the MacBook I have here are connected to this. So yeah, so to kind of kick off a session, let's go ahead and go into you can see here I'm just kind of casually browsing through um all the folders I have here. In this case, I want to be working on this repo. This is a Pokédex app. Um, fun fact or story actually. All of my PRs into the public and open-source VS Code branch as well as the previous old private Copilot chat branch or Copilot chat repo. All of my branch names were Pokemon names. So, in order to track this, I created an app that will basically it's kind of like a Pokédex me going through and catching all these little creatures and catching like PRs, I guess. But enough on that. So, I'll show this off. Let's go ahead and ask. I cannot type here. No worries. I will type on my phone here. Okay, there we go. Let me reconnect now. Cool. There we go. So, I'm going to ask it to launch the application for me. Here we go. Uh, launch this uh application for me. I cannot type and I'm not logged in. Let's try that one more time. We'll go to that folder again. actually wrong one. It's okay. Let's pivot a little bit. We'll launch it here. Launch this application for me. Yes, the demo gods really love us today. No worries. So, I'll go ahead and show off what remote looks like maybe just on my phone directly. So, unfortunately, maybe you guys won't be able to see this. um testing. Give it a second while it refreshes. okay, there we go. So, it actually ended up working, but you guys didn't get to see it. Let me reconnect here now. Cool. So, I fired off this little I just set a little hello here and I sent that from my phone. What it looks like in the application now is you can see here I have this is the one that I fired off from my phone. So, once again from my phone here, I'll try it again. Fire launch this. And you can see now in this agent session because it's all connected through the same host, it's actually going through and now reading. And you can see it's streaming on both ends. It's streaming here on my phone, but it's also streaming here in this application or in that window. So if I go through and allow or do any type of permissions check on my phone, it'll also show up and end up approving on this side. So you can see here I'm running in terminal here. I go here and hit allow one more time and it gets allowed. So let's go ahead and let that finish. It wants to do an npm install. But there we go. Because it's a React app, we can do npm starting on 43,000. Let's hope this happens quickly. Like there's a little bit of a mismatch. I'll go ahead and allow one more time here. And yeah, while that is cooking, I can kind of talk a little bit about um some of the permission modes that we're going to talk uh show off down the line as well. Um okay. Yeah. So, it looks like we're running into a little issue with the terminal, but no worries. So, one thing that you noticed is you can see that I was going through and I had to approve a bunch of things back to back to back. And in an ideal scenario, obviously with remote, if I was doing this um and I maybe stepped away from my computer, it would actually be fine because I'd be able to check and maybe be on my phone and hit allow it, hit allow, hit allow, but it can get a little tiresome and there are a lot of times where maybe we trust the agent or we trust the model to be making the correct tool calls or making the correct decisions. Okay, so you can see now it actually launched on this local host. Boom. There we go. That works very nicely. But as I was talking about permissions, um, yeah, so with permission modes that we added here, now there are a couple of different new permission modes that you can go ahead and try out. And we actually released this maybe a month or so ago, but we wanted to kind of clarify what a couple of them meant because there was a tiny bit of confusion about the differences between default, bypass, and autopilot. um autopilot which is currently in preview but will eventually be released um and enabled by default in stable on June 1st um as it goes out with our UBB changes. But I have a brief thing here that kind of talks about the differences. So to kind of talk about the main differences. So default approvals is probably what most of you guys are seeing here. So in the default approvals mode, there will never be any auto approve unless the tool unless the settings themselves have been explicitly set. And these settings look like these below here. We have the chat.tools edit auto approved settings. And these are related to the um the files that can be edited. We have the terminal auto approve that's related to the terminal tools that can be run. Uh we have like some fetch related ones as well as um rules just generally about about like when things can be um auto approve ignored when things shouldn't be. So you can kind of search for these by searching for auto approved in the settings. As you can see that's kind of a lot. What if we just want to blanketly allow everything? And yes this is a little bit of a dangerous thing but we have this new thing called bypass approvals. And it's essentially equal to the existing yolo mode/global auto approved setting that we have. But all tools are automatically approved. It doesn't really care about the settings that you have. So any settings that you previously had set with global auto approve with uh with terminal commands with anything like that. It'll all be ignored in favor if you have this version or if you have this picker set. It will auto re-trigger on certain errors. Um but it will not automatically retry if you hit your quota limits or if like you're hitting rate limits or any off-topic filters. And the big thing here is it will not answer questions or provide any user input. If it requires some kind of terminal input, um it will ask you if it requires some kind of uh if it runs through the ask questions tool, it will ask you and that's when it will actually pause the the response. And that's starkly different from autopilot where it's everything that auto approve has or everything that bypass approval has but a little bit more as well. So, on top of auto replying to your questions, kind of just letting the agent take the wheel, it also has a new thing called the task complete tool. And this is a tool that we added as well as a couple like system prompt additions where you can check if the task complete was actually called. Um, and that's when we actually know if um the model thinks it's already done. And once again, it will automatically retry on certain errors. And then it will also have an iteration loop um if task complete is not found. So, it will continue to loop and loop and loop um if it thinks that the task isn't found or if it thinks that the task complete isn't called and that the task isn't done. And another really cool thing here um is if you're in plan mode, you can start a session off in plan mode and it'll automatically hand off to the agent once it's done. And just really quick on a couple of the next steps. So, as I mentioned before, it's currently insiders only. it can be enabled by setting but once um the um unit based billing lands generally we'll also turn this on at stable. A couple new things in this last release in 1.22 actually is a couple improvements on sandboxing. So previously we were seeing some issue people run into issues with sandboxing where um in the bypass or auto approval modes it would run a command in sandbox but then it would see that it would fail and immediately run it again outside of sandbox um because it would be automatically approved. So now we have a couple of better improvements where sandbox is now only in default. Um as I talked about before, some customizations on the iteration loop numbers. If you want um the agent to loop a 100 times, be my guest. Um that's something that we're going to work on where you can force it to loop a certain amount of times or cap it to a certain number. And then a kind of more fun thing, um having an option to do a force to call at the end, which is code review. That's something to take a look at. As well as something that's more true to I guess the term of autopilot where we have a classifier on each tool called to determine whether or not it's safe to run instead of just blanketly approving everything. And then the last thing here for a couple next steps um is kind of like a goal mode where instead of using task complete, we're actually giving the model a transcript and saying, "Hey, take a look at everything that's been run here and determine if the task is done that way." And I guess how that kind of ties into the entire story of us being agent first as you saw here. I'm going to reconnect here. So it kept asking for approvals, but if I go here and I switch to autopilot, which it will not let me do on my phone, so I will do it here. So on my phone, I'm actually hitting that button. I switched it to autopilot. I hit enable and you can see here it actually changed here without me touching it and I'll go ahead and type something along the lines of like um we'll do this here of that reconnect again change all instances of Tom Nook to nom. So, while that runs, um, if I had this in default approvals, I'll actually just start up a new session and show off what that looks like in default approvals is pomnook to nom. And we'll kind of see the differences where sometimes it's doing a like a grub search or it's doing some kind of string replace. It might actually ask for approvals. Meanwhile, in this autopilot mode, it'll go through, it'll make all the changes. It's doing an MPM run build, it it did all these changes here without asking me. Um, but in some cases in default approvals, it will actually ask for approvals. Um, let's see if that happens. There's a chance it actually doesn't happen. But as you can see here, we went through it made all these changes and then it ended up using this task complete tool and it gives us a little bit of a summary of what actually happened with that task complete. You can see here it's got that summary of renamed all visible references and nomuk. Um and then this is the output. So yeah, kind of working on a couple different things to make this a little bit better in the seal in this local agent host um as well where maybe we want this output to actually be streamed out um instead of being in this collapsed section. But um yeah, you can see here it's trying to make an edit to package JSON and it thinks it's unsafe. I have a couple confirmations here pending. It also wants to make a change to the package lock. I'll go ahead and allow this and allow this. That wouldn't happen in autopilot. So that's something that um can be improved on and um if you want to go ahead and try that out, make sure you are being a little bit safe. Um and that's what the sandboxing for is in default approvals. Um but yeah, cool. There you go. It went ahead and worked on both work trees. But yeah, that was just a quick demo on how remote works and how that can be applied into some permissions and how the agent window as a whole. And yeah, I guess now we can pass off to VB to talk a little bit about BY. Okay. Yeah, thanks Justin. So with what Justin showed it it's kind of evident that VS code is becoming more sort of a AI first code editor and we still want to continue meeting developers where they are uh in the sense we want to let you use the models of your choice with the provider of your choice even though GitHub copilot has many models to do so we want to we are taking many steps to ensure that VS code lets you use your model of choice with our init harness. So, I'm going to walk through a few of the different ways we support it today and a few of the different capabilities we've added. Uh, first off, I have in front of me VS Code uh open with GitHub Copilot not signed in. Um, I'm going to go ahead and add a open up our manage language models. Manage language models view. This is where you would normally see all of your models. There's none over here, but I'm going to go ahead and add the anthropic provider quickly. Going to switch to get my secret and paste that in. And now I have all of these models coming straight from Anthropic without sending to Copilot. Um, when you do so and you do it for the first time, you're going to see a notification here. This is a notification to let you know that utility models are not configured. These are modeled used in the background by various features in VS Code which involve setting titles to your chat pane or maybe generating commit messages, renaming things using AI. And you can go ahead and now select a model from your own BY models or bring your own key models to do so. And when I go ahead and ask what's in this workspace, we're now making a request. Fun tip, you can go ahead and open the chat debug view. This lets you see all the calls being made to the models from VS code. And um over here you can see that this call was made to API. Anthropic directly uh and you can see what was returned. So now you can go ahead in these ad models and see all the native providers that are supported. If these are one of the ones you want to use, uh you can go ahead and set it up. One specific one I would like to point out here is custom endpoint. Uh these are natively supported where if you put an API key, you'd most probably get all of the models you have in your account. But if you have something custom like Mistrol for example, uh let me go ahead and add my API key for that. And you can then select what kind of API the provider supports. U Mistral in this case supports the chat completions API. Anthropic would support messages and many some support responses. Um going to the um option it takes you to a JSON file. This is the chat language mo models JSON file which contains configurations for all the custom models that you would add. Uh and you can configure them specifically over here. So I'll copy what I had previously configured for mistrol and paste it right here. And what does it not like? It does not like that quote. Um you would see various different options available here like constraining the maximum input tokens allowed or the output tokens you would like to allow as well as whether you want to enable tool calling vision or even thinking. Uh once you've configured this and close it and save it, you would should be able to see Mistral or your custom model show up below that you can select and continue the conversation with uh how do I make changes here. It would just continue to do so. Uh this was previously available as open AI compatible which we've deprecated and will be available uh in stable in this coming release. Um the next thing I'd like to point out is third party extensions. The language model providers is an extensible API which can be supported by extensions and if you make your own. I'm going to go ahead and install the hugging face provider. I'll trust them and install it. So if I now go back to the manage language models view, I will now see hugging face over here which I can select. I will add hugging faces API key to pull the repos I have available over there and see them below. Now hugging face comes with a lot of models and you don't really want to see all of them in the picker. So you can toggle visibility and just select the one you want to see. I'd like to just see Minax and go ahead and open the picker here. And I see hugging face over here. And then I can go ahead and use it. Um the few things I've shown over here have happened completely without signing in to copilot. So, in a sense, you could have a local model running on your machine like with LM Studio and use this on a plane or maybe in an aircapped environment if you'd like. Um, and the last thing is everything you've seen here will be available in the agents window as well. Um, I'm going to then hand it back to Justin. Yeah, cool. So, I thank you guys all so much for tuning in today. We have a lot of documentation um about everything that we talked about just now. on code.visisualstudio.com/doccks if you want to read more about by if you want to read more about permissions as well as the agents window at remote. If you have any questions, feel free to reach out to us on X, I believe, x.comcode. And once again, if you have any issues, um, you can find us on github.com/microsoft-vscode/issues, file an issue. We'll definitely see it. Um, it'll get assigned to the right person. And yeah, once again, thank you guys for tuning in and hope to see you guys soon.