# Single-batch local AI build (2026-11-04)

This build replaces the upstream batch-128 model library with a batch-1 library. Qwen3.5 0.8B q4f16_1 weights, tokenizer, 1024-token prefill chunk, live 4096-token context, history limit and sampling options are unchanged. The browser TVM module is paired with the compiler ABI; the WebLLM 0.2.87 engine and tokenizer/grammar modules are unchanged.

`config/ai-assets.json` pins the compressed and unpacked model/runtime artifacts. Normal `npm run build` checks both hashes before using them, keeps model transfer integrity checks, and substitutes only the pinned WebLLM bundle's TVM module. Visitors download these bytes from this site; they do not execute downloads from compiler/upstream hosts.

## Native build provenance and reconstruction

This is an optional developer build, separate from ordinary site builds and CI. The recorded build environment is macOS arm64; other native hosts have not been verified. It requires Git, Python 3.13, uv, CMake, Ninja, a native C++ compiler, Node and npm dependencies installed with `npm ci --ignore-scripts`. Build dependencies and source clones remain in an ignored workspace; none is installed globally. On macOS, the script ad-hoc signs invalid upstream nightly dylibs only inside that build environment.

- Python compiler wheels: `mlc-ai-nightly-cpu==0.26.dev246`, `mlc-llm-nightly-cpu==0.26.dev6`. Other Python packages are pinned in `compiler-requirements.txt`.
- TVM compiler/runtime source: `837cb9de1127b48ce48e4cefe09e83215b9d4ba7` from https://github.com/mlc-ai/relax (the MLC TVM fork).
- TVM FFI: build from its pinned submodule `21e30c3b1d4421f95fd69a6ba3fd0285c7b69e9c`, version `0.1.13.dev47+g21e30c3b1`. The newer PyPI `0.1.15.post2` wheel crashes this compiler during schedule printing. The working build uses the exact corresponding source, not the yanked PyPI 0.1.13 wheel used during initial investigation.
- MLC runtime source: `2f78caa4db0f90730a11ee3bb5cbd5f23bf67f9f` from https://github.com/mlc-ai/mlc-llm , with pinned submodules except the explicit TVM override above.
- Emscripten 6.0.13 supplies all runtime bitcode and the generated WASI adapter. The adapter returns a readiness promise and TVM's asynchronous instantiate path awaits `onRuntimeInitialized` before accessing the FFI registry. This accommodates Emscripten 6's asynchronous initialization; the pre-existing synchronous ordering failed to find registered FFI functions. The source changes are included in `tvm-browser-memory.patch`. `DEFAULT_TO_CXX` links the C++ runtime; path-prefix maps remove developer paths from diagnostics.
- Compiler compatibility changes in `compiler-compat.patch`: use expression bindings (`I.meta_var`) for RNN sequence/history indices; use `func.buffer_map.values()` in low-batch specialization; restore the buffer-allocation lift pass API from the pinned MLC source. Tensor shapes and model arithmetic are unchanged.
- Runtime patch: include `3rdparty/tvm-ffi/src/ffi/custom_allocator.cc` in TVM's `web/emcc/wasm_runtime.cc`; build the three TVM WebAssembly bitcode files and MLC bitcode before exporting the model.
- Browser module: esbuild CommonJS bundle of that TVM revision's `web/src/index.ts`, with Node-only `fs`, `path`, `ws`, `perf_hooks` imports excluded. The WASI adapter is generated from the same TVM source and Emscripten version as the model runtime.

From a repository checkout:

```sh
npm ci --ignore-scripts
npm run build # download and verify the original pinned model inputs
bash scripts/ai/rebuild-model.sh
npm run build
```

The script creates a fresh `.runtime/ai-rebuild` directory and refuses to overwrite an existing one. Set `AI_BUILD_DIR` to another absolute, unused path when necessary. It applies the recorded patches, builds the native FFI and WebAssembly runtime, compiles with `--device webgpu --overrides max_batch_size=1`, and invokes `package-model.mjs` to update the two compressed artifacts and hashes. Compiler-generated code may differ between host toolchains; review changed hashes and validate actual inference before shipping rebuilt artifacts. The exact artifact hashes in the manifest, rather than a promise of cross-platform bit-for-bit compilation, identify the delivered build.

The `/sources/` downloads provide this recipe, patches, requirements, packaging code and model manifest. To reconstruct outside a checkout, restore the downloads to the `scripts/ai/` names used here and obtain the public pinned compiler/runtime sources. The normal site builder additionally requires its existing npm lockfile and model manifest; all model-weight URLs/revisions/hashes are in `ai-assets.json`.

## Loading-memory repair and acceptance

The first batch-1 preview still exhausted memory on physical iPhone after download. TVM prefetched all 404.30 MiB of weight shards into the Worker facade before GPU upload; cloned Response bodies remained until initialization finished. `tvm-browser-memory.patch` changes loading to fetch/upload each shard sequentially and passes tensor records as byte views rather than copying them. The Worker uses one-shot `arrayBuffer`/`json` adapters that remove cache references on consumption and return transferred buffers directly. Parent integrity validation, transfer ownership, persistent model-part caching, network denial and conversation boundaries remain.

The accepted preview's Mac diagnostic measured GPU allocation peak 556.59 MiB, versus 1426.13 MiB for the original batch-128 library on the same hello workload. Ready allocation remained 482.75 MiB. Transferred asset-body retention peaked at 121.25 MiB (one largest shard), versus whole-model prefetch. These are instrumented allocations, not total process RSS or physical iPhone memory measurements. The owner confirmed successful loading and response on physical iPhone on 2026-11-04, then authorized delivery through production.

A separate broad recall question in an earlier diagnostic was answered incorrectly. No general model-quality or perfect recall claim is made; the change is intended to reduce loading and inference memory without changing weights or conversation settings. Regression and production verification results are recorded in the associated PR/development record.