Pursuit of wicked smartness in VS Code
Join VS Code at Microsoft Ignite or catch the sessions online. A year of VS Code, wrapped. Explore the highlights. On this page If you tuned into GitHub Universe last week, you saw a tremendous amount of progress, innovation, and vision for artificial intelligence across the entire developer workflow. What we want to do in this blog post is focus in on the advancements around Visual Studio Code over the past few months helping to realize this broader vision.
"Wicked smart" One of my favorite lines in Matt Damon and Ben Affleck's seminal film "Good Will Hunting" is "my boy's wicked smaaahtt" (read it with a Boston accent for the full effect). The line is delivered by Morgan (Casey Affleck, Ben's brother) after Will (Matt Damon) breaks up a confrontation between Chuckie (Ben Affleck) and an overly confident "1st year grad student" by recalling facts about American history page by page, word for word.
You could say Will was trained by all the books he read and was able to recall them based on the conversation. AI is similar to Will – it knows about a lot of text. But what is missing from AI – what humans have that AI does not - is the context of a particular interaction in order to give the best possible answer. And in Will's case, because he can also "read the room", he can use his book smarts to return a carefully crafted takedown.
Large language models (LLM) are trained on public repository data at a point in time. That means that they know nothing about your current code. They know about code in general, but they don't have the necessary context to accurately answer questions about it, or to suggest new code that follows the form and function of your workspace. To work around this, GitHub Copilot Chat sends snippets of code that help the model better answer questions (this is called Retrieval Augmented Generation, or "RAG").
Answers get better by seeing the most relevant code. But there are limits in the amount of code (and guidance through the prompt) that can be sent to the LLM. For a small project, this usually isn't a problem. But consider any large source code repository out there, and you'll quickly realize it's impossible to send the contents of every file to the model. The solution to getting better answers is to send the relevant context using the right amount of resources in a reasonable amount of time.
To help with this and to unlock many other scenarios, we added the concept of participants to Copilot Chat. Participants Chat participants are domain experts that can answer the user query however they want - by fully using AI in the query processing or in a traditional way by forwarding it to a backend service. Participants can also provide the large language model access to domain specific tools. With the help of the LLM, the participant may select a tool and define how to invoke it.
One example of such a chat participant is @workspace . The @workspace participant knows about your workspace and can answer questions about it. Internally, the participant is powered by different tools: GitHub's knowledge graph combined with semantic search, local code indexes, and VS Code's language services. Chat participants can be contributed by the client or the service. At GitHub Universe, there was a demo of a service-side participant, a "Docs Agent" in the github.com Chat experience that knows how to search documentation found in repositories (coming soon to VS Code).
Client-side participants can be contributed through traditional VS Code extensions. More on that in the Extensibility section, but let's take a look at two chat participants that are available in VS Code today: @workspace and @vscode . @workspace The @workspace participant knows how to gather context about the code in your workspace, can help you navigate it, find relevant classes, files, etc. Imagine you are in the VS Code repository and you want to find out more about the service in charge of the current ICodeEditor; you can use the participant like this: Natural language is used to ask the @workspace participant "what service class do I use to get the current ICodeEditor".
From there, the participant does the following to get just the right amount of context to send to the LLM: The vscode repo has been indexed by the GitHub Search Blackbird service . The @workspace participant uses this index as a tool to tap into the repository knowledge graph. The @workspace participant runs a semantic search that returns relevant code snippets and meta data. GitHub search service has indexed the top 10K GitHub repositories, with plans to add more.
The next tool that the @workspace participant uses is the lexical text search over the local index to find extra code such as local, uncommitted changes, and the Copilot conversation history. Then @workspace uses a final tool - VS Code"s language intelligence to add crucial details like function signatures, parameters, and even in-line documentation. All of these pieces of context are ranked, sliced, and summarized by the @workspace participant and then sent off to the LLM to answer the question.
Because it has all of the necessary context, the @workspace participant can answer the kinds of questions that developers are much more likely to ask. For example, questions that pertain to how different parts of the code interact: " @workspace how are notifications scheduled?" Or questions that require knowledge of related code, dependencies, and design patterns: " @workspace add form validation, similar to the newsletter page" @vscode VS Code can be customized in so many ways that even the members of the VS Code team get pleasantly surprised when they discover some hidden functionality.
To help our users and team members alike to unlock the full power of VS Code, we created the @vscode participant. This participant knows all about VS Code and can help you bridge the gap between natural language and VS Code commands and customizations. @vscode participant internally uses tools that give it access to the index of all the settings and commands and we are in the process of adding a tool so that this participant can also use the VS Code documentation .
Now you can ask vague questions like " @vscode the name of that thing when vscode fake opens a file? And how to disable it?". Note that there is a Show in Settings Editor button in the response. This is because the @vscode participant knows not only about how VS Code works, but also has a tool to invoke the Settings editor or the Command Palette. In addition, the Command Palette now supports similarity search, so you no longer need to know the exact name of the command when you search for it.
You no longer need to speak the unique VS Code jargon to unlock all the goodness the team ships month over month. This is just the beginning of the @vscode participant. We plan to support more and more scenarios that will allow users to better understand and fully control VS Code. Slash commands Chat participants can also contribute what we call slash commands , which are shortcuts to specific functionality provided by the participant.
One of the tasks when answering questions is to determine the intent, understanding what you want to do. We can infer that "Create a new workspace with Node.js Express Pug TypeScript" means that you want a new project, but " @workspace /new Node.js Express Pug TypeScript" is explicit, concise, and saves you time typing. Once the intent is clear, the @workspace participant has a much better chance of addressing the user's needs, despite the inherent ambiguity of natural language. @workspace can propose a directory structure and users can click on the proposed files to preview them.
There is a Create Workspace button that will generate these files in a new folder. Extensibility "VS Code is just a shell, you need extensions to make it shine!" – the usual meeting anthem of Microsoft teams working on VS Code extensions, proudly flaunting their role in VS Code's success. We, as the VS Code core t
The rest of this page is on the live site.