# Languages (/languages)

A language in Lumis is a Tree-sitter parser: a WebAssembly module that Lumis
compiles and loads. You have two ways to get one, and you can use both.

|  | On demand | Preloaded |
| --- | --- | --- |
| What you write | nothing | a list of languages, once |
| First document in a language | pays the compile | already loaded |
| Every document after | full speed | full speed |
| A language you did not expect | works | works, on demand |

**On demand is the default and needs no setup.** Highlight a document and Lumis
loads whatever that document turns out to name. **Preloading is the same work,
moved off your users' first request** — it is the "pick a language" step done
ahead of time instead of during a request.

Both end in the same place. Preloading only decides *when* you pay.

<span id="highlighting-loads-what-a-document-needs" />

## On demand

Nothing to declare. Lumis resolves and loads a language the first time a
document names it, then reuses it for every later document.

Where the bytes come from depends on the runtime. In Elixir and Node a parser is
an ordinary dependency: `mix deps.get` and `npm install` deliver it, and
highlighting reads it off the code path without touching the network. The CLI
has no manifest to read, so it downloads and verifies a parser the first time it
needs one. A browser loads the parser packages the application imported, and
downloads nothing unless a resolver is configured.

:::note
A package is named after the WebAssembly parser it carries, not after a
language, because the parser is what Lumis loads. Most parsers
serve one language, and the package's default export is that language. A few
serve more than one, and export each by name:
`@lumis-sh/wasm-embedded-template` has `ejs` and `erb`, and
`@lumis-sh/wasm-markdown` has `markdown` and `mdx`.
[Language catalog](/reference/languages) lists the package for every language.

A grammar split in two is a dependency of the package that needs it, on npm and
on Hex alike: installing `@lumis-sh/wasm-markdown` or `lumis_wasm_markdown`
brings `markdown_inline` along.
:::

That includes languages **injected inside** another one, in the same pass: a
Markdown file with a fenced Rust block highlights that block, and an HTML page
highlights its `<style>` and `<script>` contents, without any of them appearing
in your code.

A language that cannot be loaded costs its own block, not the document. A
thousand-line file still highlights when one fenced block names something the
project never installed; that block stays plain. A parser package installed
after the process started is picked up when the process restarts.

The cost is a Wasmtime compile: seven parsers take about 8 s to compile, against
1.3 s once their compiled forms are stored. Registering an already-compiled
parser is 3-15 ms, and about 0.3 ms after that. So the penalty is real but it is
paid once per language, by whichever request arrives first.

That is fine for a CLI, a script, a build step, or anything where one slow first
render does not matter. Preload when it does.

:::note
Browsers are the exception. A browser cannot download a parser in the middle of
highlighting, so a language has to be loaded somewhere on the page before a
document injects it, including one from a registered bundle. It does not have
to be loaded into the highlighter doing the work: once any highlighter on the
page has loaded a language from its package, every other highlighter without a
resolver uses it for injected blocks too. A global resolver set with
`languages.configureWasmResolver()` or `languages.configureLanguagePackageResolver()`
counts as a resolver for every highlighter, so none of them borrow. Node does not
have this limit: it highlights through a native addon.
:::

## Preloaded

Name the languages you want. Which call depends on whether the process doing
the work is the one that will serve:

| Runtime | In the process that serves | In a process that exits |
| --- | --- | --- |
| JavaScript / TypeScript | `languages.load([...])`, not awaited | `languages.load([...])`, awaited, in a build step |
| Elixir | `Lumis.Languages.async_load([...])` in `start/2` | `Lumis.Languages.load([...])` in a release task |
| Browsers / CDN | `createHighlighter({languages})` with imported packages | — the bundle already carries them |
| CLI | — the process exits, so there is no later to hold for | `lumis languages download <names>` |
| Rust | — parsers are Cargo features compiled into the binary | — |
| Java | construct `Lumis` at startup and retain it | — parsers ship inside `lumis4j` |

**Loading** compiles the language and keeps it in this runtime, so nothing
compiles it again. It also writes the compiled module to the data directory,
where it survives restarts. A later process that loads the same parser from that
directory reuses it instead of compiling.

A process that will serve loads in the background. A process that is not
serving, such as an image build or a release task, runs the same load, waits for
it, and exits. What it leaves behind is the compiled modules, which the serving
process picks up. Compiled modules only carry over to the same platform and
Lumis version, so run that step where the server's image is built, with the same
parser packages installed.

The CLI's `lumis languages download` fills the CLI's own store. Node and Elixir
load parsers from the packages a project installed, not from that store, so for
them the CLI saves only the compile, and only when it fetched the same parser
version the project installed. Run the application's own load instead.

:::note
Preloading is an optimization, never a prerequisite. An application that starts
cold serves correctly the whole time it is loading, so preloading must never
delay a boot or fail one — every example below keeps both true.
:::

### Which languages to name

Name your root languages and anything they can inject. Markdown may need the
languages used in fenced code blocks, and HTML may inject CSS and JavaScript.
Prefer a focused list or a small bundle. Anything you miss still loads on
demand.

### `bundle-full` is not the safe default

Every language in a process shares one 128 MB Tree-sitter Wasm store, and
nothing is ever reclaimed from it. The bound is the total of everything loaded,
so the published set can exhaust it, and the languages that fail are whichever
were asked for last.

Preload what you serve. Installing a parser is cheap; loading one is what
spends the budget.

`bundle-full` leaves out big parsers so it fits under 80 MB: `arduino`, `d`,
`fortran`, `fsharp`, `julia`, `lean`, `nim`, `ocaml_interface`, `systemverilog`,
and `tlaplus`.

### Add languages a bundle leaves out

A bundle is a starting set, not a limit. Install the parser package for each
language you need and pass it next to the bundle:

### JavaScript / TypeScript

```bash
npm install @lumis-sh/wasm-bundle-full @lumis-sh/wasm-julia
```

```ts
import { createHighlighter } from "@lumis-sh/lumis";
import full from "@lumis-sh/wasm-bundle-full";
import julia from "@lumis-sh/wasm-julia";

const highlighter = await createHighlighter({ languages: [full, julia] });
```

### Elixir

```elixir
# mix.exs
{:lumis_wasm_bundle_full, "~> 0.2"},
{:lumis_wasm_julia, "~> 0.26.0"}
```

### CLI

```bash
lumis languages download bundle-full julia
```


[Language catalog](/reference/languages) has every name, the package each one
resolves to, and what each one costs: **Size** on disk and **Memory** against
this budget. Neither column is what you download — a package manager fetches
the parser gzipped, which is several times smaller.

All of these operations are safe to repeat. A normal run reuses valid files;
the CLI's `--force` is an explicit update that resolves the compatible package
range again.

## Preloading, per runtime

### JavaScript / TypeScript

Start serving, then load in the background:

```ts title="bootstrap.ts"
import * as languages from "@lumis-sh/lumis/languages";

await startServer();

// Not awaited, so a slow compile delays no request. The
// `.catch()` is required: an unhandled rejection
// terminates the process, which is exactly the failure
// loading in the background is meant to avoid.
languages.load(["javascript", "html", "css"]).catch(
  (error) => {
    logger.warn(
      { error },
      "Lumis preload failed; languages load on demand",
    );
  },
);
```

`languages.load()` warms the same default runtime the module-level `highlight()`
uses, so nothing has to be threaded through your application. It accepts catalog
names, aliases, and bundle names such as `bundle-web`. Every name is attempted;
if any fail it rejects with an `AggregateError` naming each one, after the rest
have loaded.

`createHighlighter({languages})` is the same load into an instance you keep
yourself, and is what browsers use.

On the native Node addon, loading validates the parser and queries and persists
the compiled Wasmtime module. Bun and Deno, or a Node installation without the
addon, call the same API, but the portable `web-tree-sitter` fallback cannot
persist compiled modules across processes.

To compile in an image build or a prestart step instead, run the same call
there and await it, with `LUMIS_DATA_DIR` pointing at the directory the server
will use:

```bash
LUMIS_DATA_DIR=/app/lumis node --input-type=module -e '
  import * as languages from "@lumis-sh/lumis/languages";
  await languages.load(["javascript", "html", "css"]);
'
```

A language that fails to load exits non-zero, so the build fails rather than a
request. On the portable fallback there is no compiled module to keep, so the
step only helps the native addon.

### Elixir

Call `Lumis.Languages.async_load/1` from your application's `start/2`:

```elixir title="lib/my_app/application.ex"
def start(_type, _args) do
  Lumis.Languages.async_load([
    "markdown",
    "elixir",
    "javascript"
  ])

  Supervisor.start_link(children(),
    strategy: :one_for_one,
    name: MyApp.Supervisor
  )
end
```

It returns before the compile starts, so the boot never waits on it. The return
value is deliberately not matched: the work runs under a `:temporary` child of
Lumis's own supervisor, so a failure is logged and never retried, never
escalates, and cannot stop the application from starting.

The API accepts language names and `:bundle_*` names such as `:bundle_web`.

Loading is global to the VM: one `Runtime` lives in the NIF, so the first
process to need a language pays and every process after it does not.

Use `Lumis.Languages.load/1` when the caller does want the result and can wait
— a release task, or a smoke test that should fail if a parser is unusable.
A release task that runs it during the image build writes the compiled modules
into the data directory. Keep that directory in the image and the first boot
doesn't compile.
For release and container details, see the standalone
[Elixir deployment guide](https://lumis.hexdocs.pm/deployment.html).

### CLI

Use this when the CLI itself does the highlighting, such as a script or a service
that runs `lumis highlight`, and you want its store filled ahead of time. For a
Node or Elixir application, run its own load in the build step instead; see
their tabs.

```bash
lumis languages download rust javascript elixir
lumis languages download bundle-web
lumis languages download --all
```

Refresh compatible versions already in the store with `--force`:

```bash
lumis languages download rust javascript --force
```

For an image build, name the destination explicitly and copy that same
directory into the runtime image:

```dockerfile
FROM node:22-bookworm-slim AS parsers
RUN npx --yes @lumis-sh/cli --data-dir /app/lumis \
      languages download markdown elixir javascript

FROM debian:bookworm-slim
COPY --from=parsers --chown=nobody:root \
  /app/lumis /app/lumis
ENV LUMIS_DATA_DIR=/app/lumis
```

The first stage only needs a Node image because `npx` fetches the CLI; nothing
from it reaches the runtime image except the directory. Install the binary
instead — see [Installation](/installation) — if you would rather not reach the
network during a build.

### Browsers / CDN

A browser loads only the parser packages you import, so preloading is passing
them to the highlighter you keep:

```ts
import { createHighlighter } from "@lumis-sh/lumis";
import css from "@lumis-sh/wasm-css";
import html from "@lumis-sh/wasm-html";
import javascript from "@lumis-sh/wasm-javascript";

const highlighter = await createHighlighter({
  languages: [html, css, javascript],
});

await renderApplication({ highlighter });
```

A bundle registers lazy entries; call `highlighter.loadLanguage()` for the
bundle members the application must have. `languages.load()` takes names, which
carry no package, so in a browser it only works with a
[configured resolver](/usage/javascript#custom-resolvers).

Parser bytes a resolver fetched persist in CacheStorage, with an IndexedDB
fallback for WebKit, but never a compiled module: WebAssembly compilation
belongs to the browser and cannot be persisted by Lumis. Browsers do not use
`LUMIS_DATA_DIR`.

### Rust

The Rust crate has no parser download or runtime compilation step, so there is
no on-demand mode and nothing to preload. Select the languages at build time
with Cargo features; their parser code is compiled into the application:

```toml title="Cargo.toml"
[dependencies.lumis]
version = "0.16"
default-features = false
features = ["lang-bundle-web", "lang-rust"]
```

There is no preload command, parser data-directory environment variable, or
runtime parser-cache configuration for the Rust crate.

### Java

`lumis4j` ships its Lumis WebAssembly module and languages with the Java
artifact, so there is nothing to download. Construct the `Lumis` instance during
application startup and retain it for the application lifecycle, so module
initialization happens before the first request:

```java
import io.roastedroot.lumis4j.core.Lumis;

var lumis = Lumis.builder().build();
startServer(lumis);

// Call lumis.close() from the application's
// shutdown lifecycle.
```

There is no parser cache command, `LUMIS_DATA_DIR`, or parser-store
configuration in `lumis4j`.


## Point every process at one directory

The CLI, Elixir NIF, and native Node addon keep compiled modules in the same
directory, so a parser one of them compiles is already compiled for the others,
as long as it is the same parser version. Name the directory explicitly when one
process prepares it for another:

| Host | Explicit setting | Environment fallback | Default |
| --- | --- | --- | --- |
| CLI | `--data-dir /app/lumis` | `LUMIS_DATA_DIR` | platform user data directory |
| JavaScript / TypeScript | — | `LUMIS_DATA_DIR` | platform user data directory |
| Elixir | `config :lumis, data_dir: "/app/lumis"` | `LUMIS_DATA_DIR` | the Lumis application `priv/lumis` directory |

Elixir configuration takes precedence over the environment. Set `LUMIS_DATA_DIR`
for the deployed highlighter when it must read a directory a build step
prepared.

[CLI behavior](/cli/behavior#data-directory) has the directory layout.