Text of code.visualstudio.com › blogs › 2026 › 01 › 15 › docfind as Styx crawled it on Nov 18, 2026. The page may have changed since. Only its words are kept, without pictures or layout, and only the first part of a long page.

Open the live page

Building docfind: Fast Client-Side Search with Rust and WebAssembly

Join VS Code at Microsoft Ignite or catch the sessions online. A year of VS Code, wrapped. Explore the highlights. On this page If you've visited the VS Code website recently, you might have noticed something new: a fast, responsive search experience that feels almost instant. Behind that experience is docfind , a search engine we built that runs entirely in your browser using WebAssembly . In this post, I want to share the story of how docfind came to be: a journey that took me from a decade-old blog post about automata theory to patching WebAssembly binaries.

The problem I'm currently a Software Engineering Manager on the VS Code team, so these days I don't get much time to write code. When I do, it's rarely in unfamiliar territory. But some problems just nag at you until you do something about them. Until recently, our website still had that basic search experience: you'd type a query, and it would redirect you to search results powered by a traditional search engine. Not quite what developers are used to today.

I wanted those search results to appear instantly as you type, similar to many other websites out there. It should be something as snappy as VS Code's Quick Open ( Ctrl+P ). Together with my colleague Nick Trogh , we researched the alternatives. The landscape looked something like this: Algolia : State of the art search-as-a-service. But I wanted a pure client-side solution. TypeSense : Powerful open-source search, but requires server-side code, just like Algolia.

Plus, it'd be another service to maintain and monitor. Lunr.js : Client-side search in JavaScript, which sounded promising. We tried it with our docs (~3 MB of markdown), but it produced index files around 10 MB. Too large. Stork Search : WebAssembly-powered client-side search with a nice demo. But when we tested it, the indexes were still quite large, and the project appeared to be unmaintained. None of these options hit the sweet spot: fast, client-side, compact, and easy to host and operate.

I started to wonder if we could build something ourselves. The inspiration Thinking about client-side search reminded me of a blog post I'd read years ago. It was written by Andrew Gallant (burntsushi), the creator of ripgrep, and it's titled Index 1,600,000,000 Keys with Automata and Rust . Published nearly a decade ago, it explains how to use Finite State Transducers (FSTs) to index massive amounts of string data in a compact binary format that supports fast lookups, including regex and fuzzy matching.

The key insight is that FSTs can store sorted string keys in a state machine that's both memory-efficient and fast to query. Better yet, Andrew had published a Rust library called fst that implements exactly this. What if we could use FSTs to index keywords extracted from our documentation? The user types a query, we match it against keywords using the FST, and we get back a list of relevant documents, all in the browser, with no server round-trip.

But how could we get these document keywords? And wouldn't this just create a very large index file, given all the strings would need to be in memory? Could we use compression to create the smallest possible index? This led me to two more pieces of the puzzle: RAKE (Rapid Automatic Keyword Extraction): An algorithm for extracting meaningful keywords and phrases from text. Feed it a document, and it returns keywords ranked by importance.

FSST (Fast Static Symbol Table): A compression algorithm optimized for short strings. Since we'd need to store document titles, categories, and snippets in memory, compression would help keep the index small. With FST for fast keyword lookup, RAKE for keyword extraction, and FSST for string compression, I had the technical foundations. Now I just needed to build it in Rust, a language I'm not particularly experienced with, during the limited time I could carve out from my day job.

The solution I ended up creating a single CLI tool, docfind, meant to create an index file from our website documents whenever we build the website itself. Users of this CLI tool shouldn't need any external dependencies other than docfind itself in order to create index files. That index file ended up being a single WebAssembly module, easily served to visitors via HTTP. When visitors come to our website, their browser downloads the WebAssembly module in the background and is used to empower the search functionality.

Building the index Here's a diagram of how docfind transforms a collection of documents ( documents.json ) into the respective index file ( docfind_bg.wasm ): Docfind first reads a JSON file containing information about your documents (title, category, URL, body text). For each document, it extracts keywords using RAKE, assigns relevance scores, and builds an FST that maps keywords to document indices. All the document strings are compressed using FSST.

Both the FST and the compressed strings are then packed into a binary blob, representing the actual index. The data structure representing the index is surprisingly simple: pub struct Index { /// FST mapping keywords to document indices fst : Vec < u8 >, /// FSST-compressed document strings (title, category, href, body) document_strings : FsstStrVec , /// For each keyword index, a list of (document_index, score) pairs keyword_to_documents : Vec < Vec <( usize , u8 )>>, } The index stores keywords and maps them to indices into keyword_to_documents .

Each entry there points to the relevant documents with their relevance scores. Document strings are stored compressed and decompressed only when needed for display. Now, we could dump that index data structure to a binary file, serve it up to our website visitors and have some WebAssembly module on the site which would parse it and use the FST library to perform the search operations. But here's where things get interesting.

Rather than shipping the index as a separate binary file, docfind embeds it directly into the search library WebAssembly module, allowing visitors to fetch a single HTTP resource whenever they intend to search on the website. Searching the index So what happens client-side? When the user types a query, the WebAssembly module is loaded in memory (code and document index) to execute that query as a search operation by going through the FST data structure.

We've found it useful to use a Levenshtein automaton (for typo tolerance) and prefix matching, to get more relevant matches. Finally, search results are produced by combining scores from multiple matching keywords, decompressing the relevant document strings on demand, and returning ranked results as JavaScript objects. The challenge The trickiest part of this project wasn't the search algorithm or the keyword extraction, it was embedding the index into the WebAssembly binary.

The naive approach would be to use Rust's include_bytes! macro to bake the index into the WebAssembly module at compile-time. But that would mean recompiling the WebAssembly module every time the documentation changes. Instead, I wanted a pre-compiled WASM "template" that the CLI tool could patch with an updated index. This meant I needed to statically create a WebAssembly module template, with an empty index, and embed that in docfind.

Then, docfind could: Parse the embedded WebAssembly module to understand its structure Find the memory section and calculate how much additional space the index needs Add the index as a new data segment, updating the data count section accordingly Locate placeholder global variables and patch them with the actual index location Write out a valid WebAssembly module The WebAssembly module template declares two placeholder globals with a distinctive marker value: #[ unsafe (no_mangle)] pub static mut INDEX_BASE: u32 = 0xdead_beef ; #[ unsafe (no_mangle)] pub static mut INDEX_LEN: u32 = 0xdead_beef ; At run-time, the search function uses these to locate the embedded index and parse it from the raw bytes: static INDEX: OnceLock < Index > = OnceLock :: new ();

The rest of this page is on the live site.