An institution downloads a model checkpoint.

It now has the weights. It may be able to run them inside its own boundary, pin a version, adapt the model, and continue operating if a hosted API changes. Those are meaningful freedoms.

It still does not know enough to deploy the model.

Which languages were evaluated? Which licence conditions apply? What changed between the base and post-trained checkpoint? Do output tests cover the intended workflow? Did an adaptation alter internal signals in a way the final-answer benchmark missed? Which claim came from the publisher, and which one has been independently reproduced?

Open weights create the possibility of inspection. They do not perform it for us.

Inventory before endorsement

The Limes Open Weight Tracker starts at the simplest layer: release metadata. It records whether weights are actually downloadable, licence and deployment constraints, claimed language coverage, evaluation links, known limitations, sources, and the date an entry was checked.

The repository is early. Its schema and initial sourced entries exist, with unknowns left explicit. It is an inventory, not a leaderboard.

That distinction protects institutional buyers from a common collapse of categories. “Open weight” does not mean safe. A publisher's multilingual benchmark does not mean the model works on a municipality's documents. Inclusion in a tracker is not a Limes endorsement. The tracker separates publisher claims from Limes evaluation status and asks contributors to use unknown when a field is not sourced.

This is basic work. It is also the first inspection layer. A checkpoint with an unclear licence, missing revision, or unverifiable source should not acquire credibility because its download button works.

Documentation is an evidence map

The Limes Model Card Template asks release teams to document training-data boundaries, intended use, limitations, safety considerations, language coverage, and deployment guidance. It can be used for base models, instruction-tuned checkpoints, public-sector fine-tunes, and small experiments.

A model card cannot establish AI Act, GDPR, security, procurement, or sectoral compliance. The repository says so. Its value is different: it shows the reader which evidence exists, which assumptions were made, and where a qualified review is still required.

For an institutional deployment, that map should link to actual result artifacts. A language claim should point to tasks and examples. A safety statement should name the tested boundary. A fine-tune should identify the base revision and data path. A limitation should be specific enough to affect the deployment decision.

Unknowns are acceptable. Unmarked assumptions are not.

Outputs are necessary and incomplete

Behavioural evaluation remains the centre of deployment evidence. If a model will summarize public records, classify support tickets, or draft law-adjacent explanations, it should be tested on those outputs with human review where needed.

But output-only evaluation has blind spots. Two checkpoints can produce similar aggregate scores while relying on different internal representations. A post-training run can improve one visible behaviour and damage another that the task suite does not cover. A merge can introduce drift that appears only under a particular prompt or token position.

Internal readouts do not solve this. They add another instrument.

Limes Workspace Lens is a public workbench for producing audit cards around Jacobian-lens and J-space-style readouts in open-weight language models. Its question is narrow: after pretraining, fine-tuning, merging, or post-training, can a team produce a reproducible record of token-aligned internal signals that changed in ways an output suite may miss?

The current v0.1 slice is an audit-card and workflow layer. It validates audit specifications and readout artifacts, summarizes top-k readouts into JSON and Markdown, compares compatible checkpoint reports, and validates evidence bundles that bind readouts to behaviour artifacts, controls, command logs, compute manifests, and status gates. It also includes optional paths for real Hugging Face models and gradient-attribution artifacts.

The checked-in readout fixture is synthetic. It exists for tests and onboarding. There is no basis to present it as evidence about a real model.

An audit card is not a mind reader

Interpretability language becomes dangerous when a diagnostic token is turned into a story about a model's inner life.

Workspace Lens refuses that move. A readout hit on a word such as “fraud” or “fake” is a hypothesis-generation signal, not proof of deception. Jacobian-lens readouts are not ground-truth thoughts. Scores cannot be compared across models unless the lens fit, tokenizer, layer, position, prompt suite, and controls are compatible. The repository makes no claim about consciousness, alignment, or model improvement.

Those non-claims are part of the tool's usefulness. An institutional audit should be able to say: this checkpoint changed on this declared internal measure, under this lens and prompt suite, alongside these observed behaviours. It should not say the instrument revealed a hidden intention.

A serious evidence bundle preserves the fitted lens, model and tokenizer revisions, readout JSON, audit card, exact command log, compute manifest, behaviour and control artifacts, and validation status. Another team can then replay the path or challenge the interpretation.

Inspect the learning loop too

Checkpoint inspection should extend backwards to how the model changed.

The Learning Signal Density workstream studies how much useful learning signal a system extracts from each external observation after selection, transformation, replay, feedback, and internal compute are charged. Its current evidence is a controlled causal-domain audit, real-text selection probes, and tiny neural replications—not a language-model result. The next stated step is a tiny transformer or nanoGPT-scale replication.

The connection is practical. If a team fine-tunes an open model, the final weights do not explain which examples were selected, which generated labels were trusted, which search was charged, or which held-out data remained untouched. Open inspection needs artifacts from the learning process as well as the checkpoint it produced.

The audit stack

For an institutional model decision, the minimum useful chain is becoming clear:

1. sourced release and licence metadata;
2. a model card that marks known and unknown boundaries;
3. behaviour evaluations tied to the intended workflow;
4. checkpoint and internal-readout audits where they add information;
5. provenance for the training or adaptation loop;
6. a replayable bundle with compatible revisions and controls.

No single layer certifies the model. Together they make disagreement more productive. A buyer can reject a deployment for a specific missing result instead of relying on brand. A model developer can show exactly which claim an update supports. An interpretability researcher can criticize the instrument without losing the surrounding behaviour evidence.

If you develop or evaluate open-weight models, produce one audit card another team can replay. Choose a base and changed checkpoint, pin the revisions, pair internal readouts with behaviour and control artifacts, and write the limit beside the result. Possession is useful. Understanding has to be built.