A 125-million-parameter Llama-style language model, pretrained from random weights on a legal and financial corpus. Give it the start of a sentence and it continues in the legal register.
This is a base model, not a chatbot. It was trained on next-token prediction only, so it continues text rather than answering questions. Prompt it with the opening of a sentence and watch it complete the thought.
The honest quality metric is held-out validation perplexity: 8.36 (lower is better), reached over 10 epochs of pretraining. It speaks the legal register fluently (case-citation phrasing, procedural language) but it does not know facts, at 125M parameters a model holds only about 31MB of usable knowledge. Grounded facts would need retrieval (RAG).
Corpus: US case law (~40%), SEC filings (~40%), educational web text (~20%). First call may take ~15–30s while the model wakes from idle.
| Stage | Cost | Share |
|---|---|---|
| Data pipelinecorpus clean + dedup + 16K tokenizer + tokenize (CPU) | $2.15 | 1.8% |
| Pretraining10 epochs on 8x H100, 21.9B tokens seen, incl. smoke runs | $118.64 | 98.2% |
| Total to assemble | $120.79 | 100% |
Almost all of it is one line: 10 epochs of pretraining on 8x H100. Building the corpus and the tokenizer that fed it cost $2.15; the GPUs that learned from it cost $118.64.
Figures are actual invoiced Modal usage (GPU + CPU + memory) for this model’s lineage, not estimates. Serving is billed separately and scales to zero.