LLM

Implementation: exploratory · Proposal: draft (status checked 2026-10-31). Early published draft has binding and mock-runtime tests; provider behavior and API design remain exploratory. All capabilities remain subject to change. Maturity and opt-in policy.

User-selected language models through the runtime

This non-normative guide follows the living NAP-LLM proposal. When the runtime exposes window.napplet.llm, a napplet can ask the user to pick a model connection and submit stateless generations against it. The napplet owns its prompts, conversation, and output validation; the runtime owns connection selection, credentials, disclosure consent, and provider access. Domain presence does not imply a configured connection or a consent grant: handle unsupported and user-cancelled as ordinary outcomes and keep the rest of the napplet usable without inference.

Import llm from @napplet/sdk or @napplet/sdk/llm; root exports include prefixed helpers such as llmSelect and llmGenerate. The dedicated subpath also provides bare aliases such as select and generate.

import { llm, isLlmFailure, llmText } from '@napplet/sdk/llm';

if (window.napplet?.llm) {
  const model = await llm.select({ json: true });
  const generation = await llm.generate({
    handle: model.handle,
    messages: [
      { role: 'system', content: [{ type: 'text', text: 'Return a JSON object with a short label.' }] },
      { role: 'user', content: [{ type: 'text', text: note }] },
    ],
    format: 'json',
  });
  if (generation.finish === 'stop') applyLabel(JSON.parse(llmText(generation)));
}

Operations and results

Method	Result
select(requirements?)	LlmModel
describe(handle)	LlmModel
generate(request, options?)	LlmGeneration, with requestId on the pending promise
cancel(requestId)	boolean

select asks the user to choose a connection meeting the optional model, images, json, and tools requirements; a user default that meets them may satisfy it without a chooser. The returned handle is opaque, scoped to this endpoint and account, and never a URL or credential. describe revalidates a handle without a chooser; a changed connection, model, credential, account, or endpoint lifetime invalidates handles and rejects with selection-expired, so select again. route is a human-readable disclosure path including intermediaries; "local model" does not promise data stays on the device when the runtime is hosted. tools and toolChoiceModes say whether the connection calls tools and which of auto, none, required, and tool (named forcing) it supports; a request for an unlisted mode fails with unsupported instead of being substituted.

generate is one stateless request: the supplied messages are the entire conversation, and the runtime does not append or drop anything. Optional maxTokens (a positive integer the runtime may cap) and temperature (unsupported values fail with unsupported) tune the request. The terminal content is a list of text, thinking, and toolCall parts; llmText(generation) concatenates the text parts. Only finish: "stop" is normal completion; length and refusal may carry partial or empty content, carry no tool calls, and must not be presented as complete. With format: "json", the text of a stop result parses as a JSON object, but its schema, accuracy, and suitability remain the napplet's checks. thinking parts are display-only: the runtime omits them when history is replayed. usage is provider-reported token counts (including optional cache and reasoning counts), not a price.

Tool calls

Pass tools (portable names matching ^[A-Za-z0-9_-]{1,64}$, an optional description, and a JSON Schema whose root is type: "object") and an optional toolChoice. The model may answer with finish: "toolUse" and one or more toolCall parts. The runtime never executes them: a call is a request, not authorization, so validate its arguments and obtain any separately required capability before running it. Then send the whole conversation again, with the assistant turn as returned and one toolResult message per call immediately after it, reusing the call's toolCallId and name. Set isError: true when the tool failed. Tool results re-enter the model as untrusted input.

import { llm, llmText, llmToolCalls } from '@napplet/sdk/llm';
import type { LlmMessage, LlmTool } from '@napplet/sdk/llm';

const tools: LlmTool[] = [{
  name: 'get_weather',
  description: 'Look up current weather',
  parameters: { type: 'object', properties: { city: { type: 'string' } }, required: ['city'] },
}];
const messages: LlmMessage[] = [{ role: 'user', content: [{ type: 'text', text: 'What is the weather in Berlin?' }] }];

let generation = await llm.generate({ handle: model.handle, messages, tools });
while (generation.finish === 'toolUse') {
  messages.push({ role: 'assistant', content: generation.content });
  for (const call of llmToolCalls(generation)) {
    const outcome = await runToolIfAllowed(call); // the napplet's own validation and authority
    messages.push({ role: 'toolResult', toolCallId: call.toolCallId, toolName: call.name, isError: !outcome.ok, content: [{ type: 'text', text: outcome.text }] });
  }
  generation = await llm.generate({ handle: model.handle, messages, tools });
}
if (generation.finish === 'stop') show(llmText(generation));

Each consent prompt discloses tool definitions and results as well as prompts, so every round trip is a new disclosure and may be a new charge. History with completed calls and results may be replayed without tools, which continues the exchange without enabling new calls.

Streaming and cancellation

The pending promise returned by generate exposes requestId. Pass onDelta to receive ordered provisional block events and signal to map an abort to llm.cancel; the generation still settles through its own result, because completion may win the race. Each block (text, thinking, or toolCall, identified by blockIndex) streams one start, zero or more delta events carrying text fragments (JSON argument fragments for a tool call), and one end; a toolCall start carries the call's toolCallId and name. Deltas are optional, so render provisional output as provisional and replace it with the terminal content rather than appending. Never execute a tool from a delta or a block end. The binding drops deltas that do not match the schema, stops provisional delivery after a sequence gap, and still resolves the terminal result.

const controller = new AbortController();
const pending = llm.generate(request, {
  signal: controller.signal,
  onDelta: (delta) => {
    if (delta.event === 'delta' && delta.block === 'text') showProvisional(delta.blockIndex, delta.text);
  },
});
cancelButton.onclick = () => controller.abort();
try {
  showFinal(llmText(await pending));
} catch (error) {
  if (isLlmFailure(error) && error.code === 'cancelled') showCancelled();
  else throw error;
}

cancel(requestId) resolves true only when an active generation owned by this endpoint was terminated; unknown, terminal, or foreign ids resolve false. The runtime bounds generation time, including the consent wait, and reports timeout. Neither cancellation nor timeout proves the provider stopped computing or charging, so never retry an ambiguous outcome automatically; a user-initiated retry is a new request and a new consent.

Failures

A runtime Error record rejects the promise with an Error whose code is the specification category (unsupported, invalid-request, selection-expired, user-cancelled, cancelled, timeout, quota-exceeded, provider-error) and whose message is a diagnostic for people, not for matching. Unknown codes are still failures. Use isLlmFailure(error) to recognize one across bundle boundaries. A lost response is different from an explicit error: the binding's own transport timeout rejects with a plain Error and no code. Runtime limits are policy, not fixed maxima; a request that exceeds them fails explicitly instead of truncating messages, dropping images, or changing models.

Declaring the domain

Declare llm as required (R) only when inference is the napplet's core task; otherwise declare it optional (O) and check window.napplet?.llm before use. The vite plugin infers the bare domain from @napplet/sdk/llm and @napplet/nap/llm imports and from direct window.napplet.llm reads. Declaring the domain configures no connection and grants no consent. Model output is untrusted data: render it as text, never as markup, and never treat it as authorization to execute tools, sign, pay, or fetch.

Conformance scope

Conformance validates the four request schemas, including tool definitions, tool-choice rules, part placement by role, and adjacent tool-result matching, rejects result and delta envelopes emitted by napplets, and runs the reference shell's deterministic mock: one connection (reference-model, with images, JSON, tools, and every tool-choice mode), endpoint-scoped handles, exact-model requirement matching, a start / delta / end block lifecycle before the authoritative result, echoed text or JSON output (trailing tool results echoed, with the specified Tool execution failed. prefix for errors), a toolUse result for required or named choices, and fixture markers in the last user text ([[decline]] for a declined prompt, [[refuse]] for a provider refusal, [[stall]] for a generation that stays active until cancel, [[think]] for a thinking block, [[tool]] for a call to the first supplied tool). The mock never executes a tool call. It exercises protocol shape, not live inference, consent prompts, provider adapters, or billing; verify those in an LLM-capable host.