Language Model API | Visual Studio Code Extension API
AI agents: See llms.txt for documentation discovery and navigation. Documentation articles are also available as Markdown by appending .md to the article URL. Join VS Code at Microsoft Ignite or catch the sessions online. A year of VS Code, wrapped. Explore the highlights. Language Model API View as Markdown Ask in VS Code Ask in VS Code Insiders The Language Model API enables you to use the Language Model and integrate AI-powered features and natural language processing in your Visual Studio Code extension.
Use the Language Model API to add model-powered behavior to a language or debugger extension, or to an extension command or task . For example, a language extension might suggest names during a rename operation. To extend Copilot chat workflows, start with agent plugins or tools instead of implementing a chat participant. The process for using the Language Model API consists of the following steps: Build the language model prompt Send the language model request Interpret the response The following sections provide more details on how to implement these steps in your extension.
To get started, follow the editor annotations tutorial . Build the language model prompt To interact with a language model, extensions should first craft their prompt, and then send a request to the language model. You can use prompts to provide instructions to the language model on the broad task that you're using the model for. Prompts can also define the context in which user messages are interpreted. The Language Model API supports two types of messages when building the language model prompt: User - used for providing instructions and the user's request Assistant - used for adding the history of previous language model responses as context to the prompt Note : Currently, the Language Model API doesn't support the use of system messages.
You can use two approaches for building the language model prompt: LanguageModelChatMessage - create the prompt by providing one or more messages as strings. You might use this approach if you're just getting started with the Language Model API. @vscode/prompt-tsx - declare the prompt by using the TSX syntax. You can use the prompt-tsx library if you want more control over how the language model prompt is composed. For example, the library can help with dynamically adapting the length of the prompt to each language model's context window size.
Learn more about @vscode/prompt-tsx or explore the chat extension sample to get started. To learn more about the concepts of prompt engineering, we suggest reading OpenAI's excellent Prompt engineering guidelines . Tip: take advantage of the rich VS Code extension API to get the most relevant context and include it in your prompt. For example, to include the contents of the active file in the editor. Use the LanguageModelChatMessage class The Language Model API provides the LanguageModelChatMessage class to represent and create chat messages.
You can use the LanguageModelChatMessage.User or LanguageModelChatMessage.Assistant methods to create user or assistant messages respectively. In the following example, the first message provides context for the prompt: The persona used by the model in its replies (in this case, a cat) The rules the model should follow when generating responses (in this case, explaining computer science concepts in a funny manner by using cat metaphors) The second message then provides the specific request or instruction coming from the user.
It determines the specific task to be accomplished, given the context provided by the first message. const craftedPrompt = [ vscode . LanguageModelChatMessage . User ( 'You are a cat! Think carefully and step by step like a cat would. Your job is to explain computer science concepts in the funny manner of a cat, using cat metaphors. Always start your response by stating what concept you are explaining. Always include code samples.' ), vscode .
LanguageModelChatMessage . User ( 'I want to understand recursion' ) ]; Send the language model request Once you've built the prompt, select the language model by using the selectChatModels method. This method returns an array of language models that match the specified criteria. Then, send the request to the language model by using the sendRequest method. To select the language model, you can specify the following properties: vendor , id , family , or version .
Use these properties to either broadly match all models of a given vendor or family, or select one specific model by its ID. Learn more about these properties in the API reference . Note : Currently, gpt-4o , gpt-4o-mini , o1 , o1-mini , claude-3.5-sonnet are supported for the language model family. If you are unsure what model to use, we recommend gpt-4o for it's performance and quality. For interactions directly in the editor, we recommend gpt-4o-mini for it's performance.
If there are no models that match the specified criteria, the selectChatModels method returns an empty array. Your extension must appropriately handle this case. The following example shows how to select all Copilot models, regardless of the family or version: const models = await vscode . lm . selectChatModels ({ vendor: 'copilot' }); // No models available if ( models . length === 0 ) { // TODO: handle the case when no models are available } Important : Copilot's language models require consent from the user before an extension can use them.
Consent is implemented as an authentication dialog. Because of that, selectChatModels should be called as part of a user-initiated action, such as a command. After you select a model, you can send a request to the language model by invoking the sendRequest method on the model instance. You pass the prompt you crafted earlier, along with any additional options, and a cancellation token. When you make a request to the Language Model API, the request might fail.
For example, because the model doesn't exist, or the user didn't give consent to use the Language Model API, or because quota limits are exceeded. Use LanguageModelError to distinguish between different types of errors. The following code snippet shows how to make a language model request: try { const [ model ] = await vscode . lm . selectChatModels ({ vendor: 'copilot' , family: 'gpt-4o' }); const request = model . sendRequest ( craftedPrompt , {}, token ); } catch ( err ) { // Making the chat request might fail because // - model does not exist // - user consent not given // - quota limits were exceeded if ( err instanceof vscode .
LanguageModelError ) { console . log ( err . message , err . code , err . cause ); if ( err . cause instanceof Error && err . cause . message . includes ( 'off_topic' )) { stream . markdown ( vscode . l10n . t ( "I'm sorry, I can only explain computer science concepts." ) ); } } else { // add other error handling logic throw err ; } } Interpret the response After you've sent the request, you have to process the response from the language model API.
Depending on your usage scenario, you can pass the response directly on to the user, or you can interpret the response and perform extra logic. The response ( LanguageModelChatResponse ) from the Language Model API is streaming-based, which enables you to provide a smooth user experience. For example, by reporting results and progress continuously when you use the API in combination with the Chat API . Errors might occur while processing the streaming response, such as network connection issues.
Make sure to add appropriate error handling in your code to handle these errors. The following code snippet shows how an extension can register a command, which uses the language model to change all variable names in the active editor with funny cat names. Notice that the extension streams the code back to the editor for a smooth user experience. vscode . commands . registerTextEditorCommand ( 'cat.namesInEditor' , async ( textEditor : vscode .
TextEditor ) => { // Replace all variables in active editor with
The rest of this page is on the live site.