# LatamBench audits LLMs on Latin American culture

> An independent leaderboard for how well language models know Latin American culture, with a calibrated judge and open transcripts.

**Published:** March 6, 2026 | **Area:** Research | **Kind:** Ship | **Crafters:** Railly Hugo

![](https://crafter.run/changelog/latambench/cover.webp)

---

LatamBench is an independent leaderboard for how well language models know Latin American culture. It reruns the Trueque and CHOCLO benchmarks with a judge anchored to the reference answers, publishes every transcript and audits the benchmarks along the way. It is built by Crafter Research and lives at [latambench.org](https://latambench.org).

<Clip src="/changelog/latambench/leaderboard.mp4" poster="/changelog/latambench/leaderboard.webp" size="1280x800" alt="Switching the LatamBench leaderboard between Trueque and CHOCLO and sorting by hallucination" caption="Sort by hallucination and the ranking changes." />

## Three numbers per model

Raw accuracy counts an honest "I don't know" the same as an invented answer. LatamBench splits every model's result into correct, abstention and hallucination, so you can see which models make things up when they fail. On CHOCLO, the regional LatamGPT model hallucinates on 53.2% of items, the highest on the board.

## Two benchmarks, audited

Trueque has 500 curated questions on culture, history, geography and food from 20 countries. CHOCLO covers long-tail cultural entities by country, category and difficulty, and its results are marked preliminary because the judge calibration covers Trueque only.

![LatamBench cards for the Trueque, CHOCLO and planned in-house datasets, and the public harness section](https://crafter.run/changelog/latambench/benchmarks.webp)

*The harness, raw responses, judge transcripts and calibration are all public.*

## Try it

Read the leaderboard at [latambench.org](https://latambench.org). To audit a verdict or run a new model, the harness, raw responses and judge transcripts are in the [GitHub repository](https://github.com/crafter-research/latambench), with seed 42 and temperature 0 so the table reproduces.

---

**Repositories**

- [crafter-research/latambench](https://github.com/crafter-research/latambench)

**Links**

- [Open the leaderboard](https://latambench.org)

**Sources**

- https://github.com/crafter-research/latambench
- https://latambench.org

**All changes:** [Markdown index](https://crafter.run/en/changelog/sitemap.md) | [Crafter Station](https://crafter.run)
