The Black Gate Benchmark

“What is the exact opening scene of Game of Thrones?”

Leaderboard

Weighted score out of 100, averaged over five independent runs per model.

bar ends at the mean score lowest to highest run

Accuracy against cost

Cost of the five generation runs against mean score. Up and left is better.

Cost breakdown

Dimension scores

Judge scores out of 10, averaged over runs. Cell shading follows the score; the best score in each dimension is set in bold.

Model detail

Method

Ground truth given to the judge