The browser implementation measured against the same Python reference.
33%parity
How much of nanochat survives the move to the browser?
12 of 36 capabilities
fully present · baseline nanochat (Python) → target NanoChat Web
· 36 compared
assessment v1 · 2026-08-02 · authored match
36 runtime capabilities: 12 aligned, 18 partial, 4 missing in Web, 2 intentional differences. UI excluded. Target is codex/foundation at c950592 with recorded dirty runtime manifest; no commit made.
Different 2
Both sides implement the capability — but not the same way. This is the verdict a
simple has-it check cannot express, and it is where a port quietly diverges from its reference.
Arbitrary Python executioncatalog:F26→ Arbitrary Python execution98%Matches: Generated Python stays inert.Differs: Execution is excluded pending an approved isolated executor.
HumanEvalcatalog:F30→ HumanEval98%Matches: Programs are prepared and fingerprinted without execution.Differs: Actual execution requires an isolated executor.
Missing in NanoChat Web 4
The work queue: capabilities the baseline has and the target does not.
CUDA, dtype, compilation, and FP8catalog:F1198%Matches: Diagnostics exist, but training remains CPU-only.Differs: No CUDA training, dtype scaling, compilation, or FP8.
GSM8K rollout policy optimizationcatalog:F1898%Matches: GSM8K evaluation exists but no rollout-policy training path exists.Differs: No rewards, advantages, policy-gradient update, RL resume, or RL package.
DDP training and coordinated overflow and checkpoint statecatalog:F3298%Matches: Static rank partition helpers exist; no torch.distributed/DDP path exists.Differs: No process group, DDP wrapper, synchronized overflow/metrics/checkpoints.
Official pipeline and run scriptscatalog:F3598%Matches: Individual services/workers exist.Differs: No official speedrun/miniseries/scaling/CPU pipeline orchestration.
Partial 18
Started, not finished — present but not equivalent.
Named prepend and append encodingcatalog:F02→ Named prepend and append encoding90%Matches: Named special identities and conversation boundaries exist.Differs: No isolated general prepend/append encoding contract.
Raw integer prepend and append IDscatalog:F03→ Raw integer prepend and append IDs90%Matches: Raw IDs are used internally for boundaries and stopping.Differs: No isolated caller-owned prepend/append ID contract.
Named and raw boundaries on batch inputcatalog:F04→ Named and raw boundaries on batch input90%Matches: Batch generation and boundary-aware conversation paths exist.Differs: No isolated ordered batch-boundary API.
Raw single-token bytescatalog:F05→ Raw single-token bytes90%Matches: Portable ranks and exact byte metrics exist.Differs: No pinned public single-token byte contract.
Supplementary-plane Unicode encodingcatalog:F06→ Supplementary-plane Unicode encoding90%Matches: UTF-8/tiktoken Unicode paths are tested.Differs: No shared supplementary-plane trainer/import/bytes fixture.
Tokenizer training and official data acquisitioncatalog:F07→ Tokenizer training and official data acquisition90%Matches: Bounded BPE training and local parquet reading exist.Differs: No ClimbMix acquisition/cache/canonical shard proof.
Best-fit packing and resumable distributed loadercatalog:F08→ Best-fit packing and resumable distributed loader90%Matches: Rank partitioning, best-fit packing, epoch cycling, and resume-state emission exist.Differs: No checkpoint or process-group integration.
Upstream SmolTalk and task mixture orderingcatalog:F14→ Upstream SmolTalk and task mixture ordering90%Matches: Exact local split-bound task objectives exist.Differs: Upstream HubDataset mixture membership, weights, and order are absent.
Ordered task dataset sequencescatalog:F16→ Ordered task dataset sequences90%Matches: Ordered local task pipelines exist.Differs: Upstream TaskSequence/HubDataset curriculum semantics are absent.
Immutable HubDataset row viewscatalog:F17→ Immutable HubDataset row views90%Matches: Immutable content-preserving local task rows exist.Differs: Official Arrow-backed HubDataset acquisition is absent.
Real upstream checkpoint and tokenizer interchangecatalog:F21→ Real upstream checkpoint and tokenizer interchange90%Matches: Pinned non-pickle tensor/tokenizer interchange is tested with synthetic fixtures.Differs: No direct .pt reader or authorized real-artifact round trip.
Token-native chatcatalog:F23→ Token-native chat90%Matches: Real token-native resident chat and safe structured parts exist.Differs: No authorized upstream artifact token/logit agreement.
Safe calculator evaluatorcatalog:F25→ Safe calculator evaluator90%Matches: Bounded calculator and token state machine exist.Differs: No resident generation/cache/token-budget integration.
Inference benchmark matrixcatalog:F31→ Inference benchmark matrix90%Matches: Planning, accounting, summaries, and CPU decode timing exist.Differs: No CUDA timing, real VRAM, or full CLI matrix.
Parquet row-group partition and resume planningcatalog:F33→ Parquet row-group partition and resume planning90%Matches: Row-group planning/reads, rank partitioning, epochs, and resume-state emission exist.Differs: No training-checkpoint/process-group/device integration.
Flash Attention 3 and SDPA fallbackcatalog:F34→ Flash Attention 3 and SDPA fallback90%Matches: PyTorch SDPA and CPU semantic masks are tested.Differs: No FA3 integration or upstream fallback matrix.
Canonical headless test baselinecatalog:F36→ Canonical headless test baseline90%Matches: Engine passes 413/414; only the pre-existing Forward Pass visualization contract fails. Prior AI baseline was 157/157 and was not refreshed.Differs: Synthetic fixtures and one unrelated failure leave the canonical baseline partial.
Only in NanoChat Web 0
Not defects. Parity is directional, so what the target adds beyond the baseline
shows up here rather than counting against anyone.
36 runtime capabilities: 12 aligned, 18 partial, 4 missing in Web, 2 intentional differences. UI excluded. Target is codex/foundation at c950592 with recorded dirty runtime manifest; no commit made.