A product shot looks perfect until you notice the label changed. A generated film scene has gorgeous lighting, but the camera drifts the wrong way. These are the details that decide whether an AI output survives to the final cut — and they are exactly what a single overall ranking hides.
A new wave of creative-model leaderboards launched this month with a different premise: rank models by the job, not in general. One of them, an arena-style board covering image and video generation, published its first results this month after blind judging by 28 expert reviewers and 1,003 community tastemakers. Whatever board you eventually trust, the design choices behind it are worth understanding — because they change how you should read every number on it.
Why one ranking can't answer a production question
Consider a skincare campaign: stills for the product page, a short video ad, a talking presenter for social. One campaign, three genuinely different demands. The stills need the bottle, label and color to stay accurate. The ad needs the product convincing in motion. The presenter needs mouth shapes matched to audio. Being strong at one tells you almost nothing about the others.
That is why task-specific boards are splitting from overall rankings. A typical launch lineup now runs two views — image and video — each with an overall board plus specialized ones. On the video side: ads, film, animation, motion design, video editing and lip sync. On the image side: product/e-commerce, film, graphic design and image editing. Film appears on both sides because composing a still frame and sustaining a moving sequence are different skills, and editing gets its own board because making a new image well says nothing about making one precise change to an existing image.
The evaluation criteria fork the same way. Advertising boards score product and brand consistency; film boards score camera and lighting control; graphic design boards score layout and typography. A distorted logo that ruins an ad is not the same failure as a face that breaks continuity in a scene — and a leaderboard that averages them tells you neither.
How the numbers are made
The judging method behind these boards borrows from competitive ranking systems, and three design choices matter:
- Blind pairwise comparison. Judges see two unlabeled outputs and vote on one named criterion — not "which do you like" but "which better preserves the label." Model identity is hidden and left/right position randomized to kill brand recognition and position bias.
- Fixed inputs. Every model gets the same brief, drawn from real creative use cases in both professional and amateur styles, and the output entering judging is selected by a fixed rule rather than hand-picked. No cherry-picked best runs.
- Weighted consensus. Votes convert to relative scores through a Bradley-Terry estimator — the Elo-like math used to rank game players — with expert-council votes carrying three times the weight of the broader community pool.
Two reading rules fall out of the method. Every score ships with a 95% confidence interval, and when intervals overlap, a small gap is not an advantage — it is noise wearing a ranking. And scores belong to their own board: the same number on the film board and the ads board does not mean the same capability, because the criteria behind each are different.
What the launch lineup covers
The first published wave evaluates seven image models — Seedream 5.0 Pro, GPT Image 2, Nano Banana Pro, Grok Imagine 2.0, Nano Banana 2, Qwen Image 3.0 and Flux.2 Pro — and eleven video models including Seedance 2.5, Wan 3.0, Kling 3.0 Omni, Google Omni Flash, MiniMax H3, Flux 3 Video, PixVerse V6 and Grok Imagine 1.5. Individual boards carry smaller subsets depending on eligibility and evaluation coverage.
These are launch rosters, not permanent ones. Model versions matter more than names: a board's number for "Kling 3.0 Omni" says nothing about its successor, and a model that sits third today can lead next quarter. Check the version string before quoting any ranking.
From a ranking to a workflow decision
A leaderboard narrows candidates; it never selects your model. The sequence that works:
- Find the board matching your deliverable, then read the specific criteria that could make or break it — brand consistency for packaging, text accuracy for headlines, reference adherence for recurring characters.
- Inspect the sample prompts and generations behind the scores. A number is easier to interpret when you have seen the output shape producing it.
- Shortlist two or three models and run them on your own brief with your own assets — actual packaging, the reference image you will really use. Benchmark strengths do not always transfer to your subject.
- Check the production specs separately: resolution, reference limits, clip length, generation time and price. Quality scores deliberately exclude them, which is why a top-ranked model can still be the wrong one for a 500-image batch.
Where these boards sit next to the others
General AI rankings — preference arenas, technical benchmark suites, price-and-speed trackers — answer different questions. A chat-preference leaderboard measures what large crowds like in text; a technical suite measures capability ceilings on fixed tests; a creative production board measures judged output quality on media tasks. Different evaluations produce different rankings without ever answering the same question, so the useful habit is checking what each site measures, which model versions it includes, and who does the judging.
One structural caveat deserves naming: creative boards are typically operated by platforms that also sell access to the ranked models. The disclosed position is that no provider paid for inclusion or placement, and blind judging with published methodology is a real safeguard — but a platform ranking its own inventory is still a ranking shaped by commercial context. Read the disclosures, then weight accordingly.
The honest takeaway
Task-specific leaderboards are the most useful thing to happen to model selection in a while, because they finally rank the thing creators actually buy: not "best model," but best-at-my-job. Use the boards to shortlist, the confidence intervals to discount ties, the sample galleries to build intuition — and then let a same-brief test on your own packaging, your own face, your own scene make the call. The ranking is the starting gun, not the finish line.
