Understand language models
AI agents: See llms.txt for documentation discovery and navigation. Documentation articles are also available as Markdown by appending .md to the article URL. Join VS Code at Microsoft Ignite or catch the sessions online. A year of VS Code, wrapped. Explore the highlights. Understand language models View as Markdown Ask in VS Code Ask in VS Code Insiders Visual Studio Code uses large language models (LLMs) to power its AI features.
A model interprets the information provided to it and generates a response, such as an explanation, code, or a request to use a tool. This article explains what models can and can't do, how tokens and context limits affect your work, and how to balance quality, speed, and cost when choosing a model. For setup steps, see choose and configure language models . How language models work A language model generates a response based on patterns learned during training and the information in its current input, or prompt .
In VS Code, the full prompt includes more than the message you type: it can contain conversation history, file contents, tool outputs, and custom instructions. Some models also accept images and other input formats. Language models don't execute code or access files directly. Instead, they generate text that the agent loop interprets as actions. When a model requests a tool call, VS Code executes the tool and feeds the result back to the model for the next iteration.
Some key characteristics of language models include: Nondeterministic : the same prompt can produce different results. A successful response once doesn't guarantee the same result on another attempt. Context-dependent : the quality of the response depends on the quality and relevance of the context provided in the prompt. Limited training knowledge : training doesn't automatically include your current codebase or the latest API changes.
Tools can retrieve current information for the model to use. Fallible : a model can produce a confident but incorrect answer, sometimes called a hallucination, even about familiar topics or code it has read. Tool results help ground an answer in evidence, but don't guarantee correctness. Review explanations against their sources and validate generated code. A convincing response or a detailed reasoning summary is not a substitute for checking the result.
Tokens Models process text in units called tokens . A token can represent a word, part of a word, punctuation, or another piece of text. Token counts are not the same as word or character counts and vary by model. Input tokens represent information sent to the model. Output tokens represent what it generates. Tokens help measure both the model's context limits and the cost of requests . One user request can trigger many model requests as the agent reads files, calls tools, and checks results.
Each step can process additional tokens. Context window The context window is the amount of information a model can process in a single request, measured in tokens. Instructions, conversation history, file contents, tool outputs, and your message consume that budget, and room is also needed for the response. The available input and output limits depend on the model. A larger window can accommodate more information, but doesn't automatically include your whole repository or guarantee that the model uses every detail correctly.
Focused, relevant context still matters. As a conversation approaches its limit, supported harnesses compact older content into a summary to make room. Details can be compressed or omitted, so a long conversation is not a perfect record of everything the model can still use. Learn more about how VS Code assembles context and context compaction . Thinking and reasoning Some models use additional computation, often called thinking or reasoning , before producing a response.
This can help with tasks that require comparing approaches or working through several dependent steps. Reasoning can be useful for multi-step debugging, architectural planning, and complex refactoring. For a small edit or straightforward question, extra reasoning might add time and cost without improving the result. It does not remove the need for review. Thinking effort Thinking effort controls how much reasoning a supported model can apply to a request.
Higher effort can help with complex problems, but is not a guarantee of higher quality. Lower effort can reduce latency and token usage. The available effort levels and their default values vary by model and provider. Some models also support adaptive thinking , where the model dynamically decides whether and how much to reason based on the complexity of each request, rather than always using a fixed thinking budget. VS Code sets default effort levels based on evaluations and online performance data, and has adaptive reasoning enabled where supported.
For most use cases, the defaults work well without changes. Thinking tokens Models that use thinking tokens spend part of their generation on reasoning rather than the final answer. A visible thinking summary isn't necessarily the full reasoning output. How these tokens affect context limits and billing varies by model and provider. Learn how to configure the thinking effort level in VS Code. Choose the right model Each model has different strengths.
Some are optimized for speed and work well for quick edits and simple questions. Others have larger context windows or better reasoning capabilities, making them ideal for complex tasks. As a general guideline: Fast models are best for quick code edits, boilerplate generation, and straightforward questions. Reasoning models excel at complex refactoring, architectural decisions, multi-step debugging, and tasks that require analyzing trade-offs.
Large context models can accommodate more source material or conversation history. The agent still needs to find and include the relevant information. You can switch models at any time, based on your needs for a particular task. For a detailed comparison, see Choosing the right AI model for your task in the GitHub Copilot documentation. For a practical workflow, compare and choose models for your coding tasks using Fusion, Auto, or individual models.
Auto model selection Auto model selection in GitHub Copilot selects a model for each request based on task complexity and model availability. Use Auto when you want the service to make that choice rather than selecting a model yourself. In Copilot harness sessions , the same routing entry can appear as Copilot Fusion instead of Auto , depending on your settings and the tiers offered to your account. Fusion adds multi-model orchestration alongside single-model routing.
Auto selects from multiple models and respects your organization's model access settings . Auto won't select models excluded by administrator policies or models restricted by data-residency policies. For more details, see About Copilot auto model selection in the GitHub documentation. Copilot Fusion Copilot Fusion is a model orchestration system, not an individual language model. It chooses models and a workflow for your task, so you don't need to coordinate each step yourself.
Unlike Auto, which selects one model per request, Fusion can use one model or coordinate multiple models. For example, one model can draft a solution, another can critique it, and the original model can revise it. You receive a single final response. Consider Fusion for substantial, well-scoped coding tasks where additional review can be worth the extra time and AI credits . Availability depends on your account and organization policies.
Learn how to use Copilot Fusion . For details about orchestration, see GitHub Copilot Fusion in the GitHub documentation. AI credits and model costs GitHub Copilot plans use AI credits to measure usage. Consumption depends on the model's rates and the tokens processed, not only the number of messages you send. Check your plan for its allowance and usage limits. Copilot Fusion can use mult
The rest of this page is on the live site.