Skip to content

Recipes

For a first real decision, follow the complete CPU walkthrough (中文) and its recorded LAYA demo.

  • Laya text worker: start the external Python worker, connect the Rust frontend and compare direct and proxied responses.
  • Laya native CUDA worker: build the Hopper bundle and serve English text decisions with Rust and CUDA.
  • Laya on Apple Silicon: serve Laya on the Mac GPU with the Laya worker, put the frontend in front of it and run the benchmarks.
  • Cua-S1 4B 0.2 text worker: download the pinned weights, start the worker and connect the Rust frontend.
  • Valen-Preview-0923 reference worker: prepare the pinned Preview checkpoint and Qwen base, start the text-or-image choice worker and verify the frontend.
  • Cua-S1 4B 0.2 native text worker: build the CUDA library and the Rust worker, export the merged weights and start the worker.
  • Open-Jev native text worker: export the merged text backbone and trained decision head of Open-Jev-27B-v1.1 or Open-Jev-9B, then serve with Rust and CUDA. Open-Jev-9B validation compares the 9B worker with the reference.
  • JEV-27B-VL experimental worker: native text and preencoded image decisions; bounded H800 validation with explicit deployment limits.
  • OmniJev-4B v1.1 preparation: export the checkpoint and regenerate the CPU fixtures; the native worker is not available yet.
  • CLM behind the frontend: run CLM's own server behind the frontend on CPU with a stub encoder, and what the response comparison has to allow for.

  • D3 reference worker: download the pinned vllm-sr/d3 package, verify its runtime manifest, and serve choice/noul/score text decisions through the Python worker and the Rust frontend (serving and parity validated on A100).

  • LFM2.5-350M choice worker: score text decisions with independent questions, hybrid cache branching and configurable candidate batches.

Recipes contain setup, launch commands and examples. Reusable implementation code belongs under src/.

compare_with_backend.py checks that the frontend returns what the worker returned, for any recipe; test_compare_with_backend.py covers it without a model.