Research & field notes
Useful findings from real work.
We publish evidence-backed findings when our engineering work produces something that may help other people: tool evaluations, architecture lessons, benchmarks, failure analysis and reusable methods. Reports keep the detail needed to understand the result while protecting private systems and sensitive workloads.
AI journalism
What's Happening in AI
Checked weekly editions plus occasional long-form features and research essays. Journalism sits apart from the field evaluations below.
Latest feature (advertisement): Shoot PlanWhere fast semantic decisions actually help
30 labelled cases, 29 correct; strongest when a deterministic system has already narrowed the problem to a bounded semantic choice.
Read the field report Needle 3.0.3A tiny model is not a tiny general-purpose agent
Routing, extraction, retrieval and depth tests show a sharper sweet spot: narrow specialist work behind deterministic gates.
Read the field reportPublished records
| Report | Exact version tested | Test date | Report version | Data |
|---|---|---|---|---|
| Jev field evaluation | jev-1.13.0 | 19 Sep 2026 | v1.0 · 23 Sep 2026 | JSON |
| Needle field evaluation | cactus-needle 3.0.3 | 20 Sep 2026 | v1.0 · 23 Sep 2026 | JSON |
All records: research data index (JSON) · llms.txt · sitemap
What belongs here
This is not a model-review site. We publish when there is a substantive result worth preserving and sharing: an empirical tool evaluation, a useful system or architecture pattern, a benchmark with a meaningful baseline, a failure whose cause teaches something reusable, or a research method that improved real work.
How to read the research
Claims are scoped to the evidence that produced them. For software and models we publish exact versions and dates where they matter; for methods and architecture we state the conditions, comparison baseline and known limits. We avoid turning a local result into a universal claim.
Workloads are described at the level needed to understand the result while keeping private application details private. Measurements and aggregate outcomes are real; identifying payloads and internal project names are not published.
How to cite
Cite the exact versioned report, not this index. Each record below is the citation: publisher, tested version, test date, report version and URL.
- Report
- Jev 1.13.0 field evaluation
- Tested version
jev-1.13.0- Test date
- 19 Sep 2026
- Report version
- v1.0 · 23 Sep 2026
- Full citation
The Amateur Limited (2026). Jev 1.13.0 field evaluation, tested 19 Sep 2026, report v1.0. https://theamateur.co.uk/research/jev/1-13-0/
- Report
- Needle 3.0.3 field evaluation
- Tested version
cactus-needle 3.0.3- Test date
- 20 Sep 2026
- Report version
- v1.0 · 23 Sep 2026
- Full citation
The Amateur Limited (2026). Needle 3.0.3 field evaluation, tested 20 Sep 2026, report v1.0. https://theamateur.co.uk/research/needle/3-0-3/
Measurements are independent observations from our test environment, not vendor claims or universal rankings. Found an error, or a newer release that behaves differently? Send a correction.
/research/data/, canonical URLs, sitemap entries and a root /llms.txt. Prefer the versioned report over an unversioned summary.