Supported models and hardware¶
This page distinguishes validated serving paths from experimental integrations. Start with the CPU decision walkthrough. The rolling model tracker (#83) records available paths separately from proposed integrations and hardware evidence. Models that are being added are also tracked in issues labeled new model.
| Model | Worker | CPU | NVIDIA CUDA | Apple GPU | Requirements |
|---|---|---|---|---|---|
| LAYA, English checkpoint | External worker running the upstream Laya runtime, for text requests | Validated (#2); CPU demo | Unverified (#39) | Use the dedicated MPS worker below | Python 3.12, pinned CPU dependencies |
| LAYA, English checkpoint | Python MPS/CPU worker in this repository | Contract checks documented (#67) | No validation recorded for this path | PyTorch MPS validated on M1 Pro; M4/M5 checks documented (#30, #67) | Python 3.12, MPS dependency versions; native Metal execution remains planned |
| LAYA, English checkpoint | Native Rust worker | Processing/checkpoint checks; no CPU inference | Hopper sm_90a; validation scope |
Not supported | CUDA toolkit, TileLang for AOT generation, pinned checkpoint and rotary tables |
Cua-S1 4B 0.2, text adapter |
Reference worker on Transformers and PEFT | Unverified | Validated (#13) | Unverified | Python 3.12, the versions in requirements-text.txt |
Cua-S1 4B 0.2, text adapter |
Native Rust worker on the Qwen3.5 CUDA kernels | Not supported | Validated on compute capability 8.9 (#19, #52) | Not supported | Compute capability 8.0 or newer, the CUDA toolkit to build, weights merged with export_text_merged.py |
Cua-S1 4B 0.2, multimodal adapter |
Reference worker on Transformers and PEFT, src/frontend/cua_s1.py; no recipe yet |
Not supported | Validated (#17, #18) | Not supported | The state is one PNG or JPEG image; upstream's weights.lock.json next to the base weights |
| Valen-Preview-0923 | Python reference worker; text state or one inline PNG/JPEG image; choice questions |
Not supported | Author-run on RTX 5060 Laptop (sm_120, WSL2): one text and one 2×2 PNG request; input and readout parity ≤1e-6 against reference (model outputs not compared). Owner: community. | Not supported | Python 3.12, pinned Valen source/checkpoint, Qwen3.5-2B base, BF16 CUDA |
| Open-Jev-27B-v1.1 | Native Rust/CUDA worker on the shared Qwen3.5/3.8 executor | Not supported | Validated on H200 (sm_90) for the 74 single-candidate workload | Not supported | Compute capability 8.0 or newer, CUDA toolkit to build, exported merged weights and trained head |
| Open-Jev-9B | The same native Rust/CUDA worker | Not supported | Validated on compute capability 8.9 against the reference for 253 requests | Not supported | Compute capability 8.0 or newer, CUDA toolkit to build, exported merged weights and trained head |
| autotrust/JEV-27B-VL | Experimental native worker, single-question API | Contract tests only; no CPU inference | H800 frozen-corpus check; experimental scope and limits | Not supported | Pinned merged checkpoint, rebuilt ABI 6 CUDA library; image inputs require offline Transformers preencoding |
| CLM-v0.1-8B | External clm-serve recipe with a CPU stub embeddings server |
Stub-encoder contract checks only (#23); not real Qwen3-8B decisions | Real encoder unverified by the merged recipe | Unverified | Python, upstream CLM and head checkpoint; a real encoder requires a separate embeddings server |
| LFM2.5-350M | Transformers choice worker | Tiny-model regression tests; full checkpoint unverified | L40S FP16 recorded execution checks | Unverified | Python 3.12, pinned dependencies, explicit candidate batch size |
| CLM-v0.1-8B | Native Rust engine, clm-run, in front of a Qwen3-8B /v1/embeddings server |
Engine-only agreement 4.5e-06 on identical vectors (#29, validation); contract checks documented (#23) | Validated on compute capability 8.9 (#29, validation) | Unverified | Rust to build and a separate Qwen3-8B /v1/embeddings server for the encoder; the engine itself has no device-specific code |
| OmniJev-4B v1.1 | None yet: checkpoint export and CPU contract; the native worker is in progress (#114) | Request contract, preparation, heads and finishing checked against the reference (#120); no inference | Not yet | Not supported | Python 3.10 reference environment for the export and fixtures |
| D3 (Decision 3.0) | Python reference worker running the checkpoint's pinned runtime | Unverified | Author-run on NVIDIA A100 (sm_80): three-kind serving smoke with byte-identical frontend forwarding and reference parity (exact decisions, max delta 0.0, 348 == 348 tokens); evidence in the integration PR. Owner: @MrDongsls. | Unverified | Python 3.12, the pinned runtime requirements, the pinned package (~52 GB) |
- Validated: covered by the recipe on
mainor by the checks in the linked merged pull request. - Author-run: recorded by the recipe author on the stated hardware and not independently reproduced; the row names its owner.
- Unverified: the worker accepts this device, but no recipe or merged pull request covers it.
- Planned: not implemented yet; the linked issue tracks it.
The Cua-S1 workers answer choice questions only.
LAYA's English worker, Open-Jev and CLM support choice, score, and noul text questions.
D3's reference worker serves the same three kinds for text state; its media
inputs stay refused for now and will be integrated soon.
CLM has a row per path: the merged clm-serve recipe exercises those answer shapes with stub
embeddings and says nothing about decision quality, while the native engine's CUDA row is the
path through a real Qwen3-8B encoder. Its engine-only figure comes from the same comparison run
against the deterministic stub, where both sides get identical vectors; neither number validates
decision quality. MPS validation above is for a Python/PyTorch worker, not a native Metal backend.
The architecture contracts describe the native target. Shared processing orchestration and dynamic batching remain planned. Native workers reuse serial admission. Qwen workers run independent single-prompt prefills with CPU heads; Laya packs questions within one request and runs its scorer/action head on CUDA. These target layers do not expand the validated model or hardware coverage above.