Small legal reasoning models are back
Scroll to the end to try them yourself!
The biggest AI story in legal tech now isn’t “bigger models.”
It’s smaller, sharper ones.
Compact “Small Language Models” like DeepSeek R1 now run offline yet reason at near-GPT-4 level.
Legal teams are working with them on narrow domains - immigration, indirect tax, securities regulation, specific court rules.
Retrieval-Augmented Generation (RAG) is also moving past simple text search. New stacks pull live statutes, documents, even images in real time through secure APIs.
Who wins?
Solo lawyers, boutiques, midsize firms, and lean in-house teams. Also the vendors that serve them.
For example, a tax-litigation boutique pointed its RAG pipeline at 20 years of opinions, orders, notifications, judgements, rules and statutes. The system now drafts counter-arguments in minutes, citing precedents that junior lawyers used to dig for over days.
Before you jump in:
• Garbage in, garbage out, curate and tag your data.
• RAG narrows hallucinations; it doesn’t erase them. Keep human review.
• Budget time and cost for tech/model updates as laws change.
The takeaway:
The next competitive edge isn’t owning the largest model. It’s owning the right data and pairing it with a model small enough to run where your clients’ secrets or legal expertise lives.
Shrink a model and grow your impact? 😄
You can try Magistral small and medium by Mistral here. We have also written in instructions and to-dos based on our experience using the models.