What a rubric is
A rubric is your definition of good work. It grades every attempt at your task after you write it, so anything it accepts is treated as good.
A rubric is a set of criteria. A criterion is one binary check on the deck, and a judge, a person or an LLM, reads it, looks at the work and answers pass or fail.
The set does two jobs. It measures difficulty, as the pass rate across the set, and it captures end-user happiness: whether someone in your seat would accept this deck. Mostly that is whether the content, the numbers and the narrative hold up, with design and layout as the supporting layer.
Write so the judge never has to guess. It works agentically: it can open the output deck, the input files and the prompt, so a criterion may check the deck against a source. What it cannot do is work out which source you meant, or what you intended and never wrote down.
- The output deck, which it can open and inspect
- Every input file the task hands over: the CSV, the PDF, the template
- The prompt
- Your criterion, on its own
- What you meant but didn't write down
- The rest of your set: each criterion is read alone
So a criterion may rely on the input files, as long as it names the file and the value precisely enough for the judge to find and compare. Matches the source data is not a check; matches the FY24 Group Revenue value in financials.csv is.
The prompt is different. The judge can read it, but a brief written for a model rarely fixes the result exactly, so as described in the prompt hands the judge an interpretation where a number would hand it a check.
Write every criterion as if it is the only sentence the judge will ever read about this task, because it is.
A task is a prompt plus the files it hands over. A model reads it and builds a deck, a criterion is one sentence a judge answers pass or fail on, and your criteria together are the rubric. The share that pass is the task's pass rate.
Section 10 is a complete worked set, two annotated slides and the criteria behind them. Read it first if examples suit you better. The self-checks in sections 2 and 4 to 6 are optional and nothing records your answers; section 12 is the assessment.
Outcome and Quality
You write two kinds of criteria. A third kind, Structural, runs without you, and section 5 covers it.
Outcome is an expectation the prompt, an input file or the supplied template fixes, including figures calculated from the inputs. If it comes straight from those, it is Outcome. Everything else is Quality: an expectation you introduce through professional judgment because the task materials do not specify it, such as a line chart when nothing said which chart.
So the one question to ask is: does the requirement come from the task materials, or from your own judgment? Spend your effort on how well the criterion is written rather than on perfecting its tag.
Seen on one slide
The prompt was one line: Visualise the value chain for a strategy consulting client. Four criteria were written for the slide it produced, two of each.
- Quality The title states the takeaway.
- Quality The value chain is shown as a chevron reading from left to right.
- Outcome There is a value-chain visual on the slide.
- Outcome Every stage listed in the input file
value_chain.csvappears on the slide, and no others.
The prompt bought the visual and, through the input pack, the six stages. The chevron and the takeaway title are yours, and that is the half you can move. The chevron is in only because five of five would build one for a value chain. Where peers would accept another shape too, nested circles instead of a funnel for a market sizing, describe what the visual shows and leave the shape open. Both Quality criteria here would carry the Domain Specific tag from section 7: takeaway titles and chevrons for a value chain are conventions of the profession, not asks of this task.
What the task materials fix
Explicit asks are the ones written down: include a bridge, exactly 8 content slides.
What the prompt hands over comes with the ask. If the input pack lists six stages, all six on the slide is an Outcome criterion even though the prompt never listed them, and the criterion can point at the pack instead of copying the list out.
You don't have to test everything in the prompt, and simple structural Outcome criteria often pass. Facts, calculations and reconciliations drawn from the input files can still fail, so test those.
Three layers:
- Outcome: a requirement the prompt, template or brand guide fixed. The judge can open the template, so name the element and the file: the title sits above the blue horizontal rule and the subtitle below it, as in the input template
template.potx. - Quality: judgment nothing fixed, such as which chart type to use, what the title says, or where the commentary sits when the template doesn't place it.
- Structural: the universal template-match check, which runs without you.
Write one when it is specific to your task and the universal check cannot know it: the rule your template puts on every slide, the colour your brand guide names, the quadrant a task swapped out. A task built entirely on a template leaves less room for Quality.
Following the template is Outcome. Where the template explicitly fixes something, the deck follows it, and no Quality criterion rewards a departure. Where the template only sets a default it was never designed for, such as body-text size applied to chart labels, a departure five of five peers would make is Quality: write it as its own criterion, and make sure the template criterion does not fail it.
What professional judgment adds
What you bring is Quality: the title that states a takeaway, the chevron the prompt never specified, the year any analyst would flag.
Execution criteria check that the work was done cleanly: consistent formatting, a source line where one belongs, the same name for a peer on every slide. A number the inputs fix, on the slide the prompt names, is Outcome, not execution.
Judgment criteria check that the right professional call was made: which chart type, what to lead with, what to leave out, how a number is framed for this audience.
Both belong in the set, but the second kind is what separates a strong model from a weak one.
Verbosity and prose quality are Quality wherever your judgment sets the expectation; an explicit length limit or required wording in the prompt is Outcome, and carries no tag. Other expectations established through professional judgment are Quality. The areas below can involve execution or judgment:
- Narrative and prose: bullets that give the driver behind a number instead of restating it; titles pitched for the audience the deck is going to.
- Design: the chart type that fits the data, such as a trend over time as a line or column chart. The fourteen-slice pie is the usual failure.
- Domain convention: FDD bullets don't read like consulting bullets, which don't read like banking bullets. Consulting bullets run long, in full sentences that read without anyone presenting them.
- Consistency: one colour per category across the deck; one naming convention per set of headers; one taxonomy throughout.
- Format: notes sit above sources, in italic, the way your industry does it.
- Layout: chart on the left, commentary on the right, so the slide reads in the order it's meant to.
The third kind, Structural, is the automated layer: white space, overlaps, font size, template match, the mechanical signs of machine-written text and the file-opens gate. You write none of it, and section 5 lists what it covers.
Content, numbers and narrative are the core of a set: the critical figure, the segment that has to be there, the conclusion the numbers support. Formatting and layout are the supporting layer, worth criteria where the task turns on them.
New writers misjudge this boundary more often than anything else. If working out the expected result takes professional judgment because the task materials do not establish it, the criterion is Quality, even if every expert would agree.
Content outside a fixed scope can fail Outcome
When the brief fixes the scope (five slides, two named metrics, seven peers), the set needs criteria that fail unrequested content as well as criteria that check the requested content is there. A deck that adds a sixth slide or an eighth peer has changed what was asked for, and one word catches it:
Passes even when the output adds three rows the prompt never asked for.
How to build the rubric
Five steps over the same task. The first two extract what the task materials fix, the third adds professional judgment, the fourth tests the set against the outputs, and the fifth classifies and refines. The tags from section 2 run through it: Outcome where the task materials fix the expectation, Quality where you introduce it.
Start with the prompt
List everything the deck is required to contain: slides and sections, analyses, charts and tables, named companies, markets or periods, and any constraint on the deliverable. Most of this becomes Outcome:
Write the expected result into the sentence, because as requested in the prompt asks the judge to interpret a brief instead of check a fact.
Review the input files
Open every one: data files, reference documents, the template. Look for figures to be reproduced, calculations the deck should contain, required periods and categories, and the layout the template fixes. Where they fix content or presentation, write an Outcome criterion; where they leave the choice open, step 3 decides whether a Quality criterion is needed. What the template explicitly fixes is followed. A departure from a template default, one five of five would make, is Quality, and the template criterion must not fail it (section 2).
financials.csv.Pointing at the template is not overfitting when the template is part of the task. The judge has those files.
Add quality expectations
Ask what would make this deck clear and usable, walking the coverage list in section 6 as you draft. Whatever it turns up that the task materials didn't fix is Quality.
A personal preference is not a standard. Write it only where it reflects a defensible professional convention, or materially changes how well the deck communicates.
A judgment criterion has three parts: a judgment verb, a specific object, a testable condition. Leads with; the financial overview title; the EBITDA margin trend.
Not "slide 3 has a clear message", which names no object and sets no condition.
This is the step where house style gets in, so every criterion it produces goes through the peer test in section 4.
Review the outputs
Open the agent decks and look for failures your criteria don't yet catch. For each one: is the failure material, is the right answer fixed by the prompt, an input file, the template or a professional standard, and would the criterion hold against a different set of outputs? Add it only if all three are yes, and write what the good deck shows, not the failure you saw.
Use the outputs to find the gaps in your set, not as the source of truth. Take deck-wide faults first, then slide by slide: a fault on four slides is one deck-wide criterion, not four. A criterion earns its place by being material and grounded, not by how many outputs fail it: a rare failure the task or a standard fixes stays, and a rerun checks that it is stable rather than vetoing it.
Classify and refine
Tag each criterion by where the expectation came from, not by what it is about.
- Outcome: the expected result is fixed by the prompt, an input file or the supplied template.
- Quality: you introduced the expectation from professional judgment, and nothing in the task materials specifies it.
A criterion that came out of reviewing the outputs is classified the same way, by its source rather than its subject. Slide headers use the template's fill colour is Outcome; slide titles communicate the main takeaway is Quality, and Domain Specific, because takeaway titles are a professional-services convention.
Outcome versus Quality is about where the expectation came from; correctness versus communication is a separate question, and a set should test both. Then check that each criterion tests one independently failing property, reads on its own, and names its source, scope and tolerance. Section 9 is the full pass.
Write every criterion in your own words. Writing cold is what forces you to decide what good looks like before an output tells you, and the nuance reviewers value is in your language. Generated criteria carry tells, rather than, does not pass, passes only when, and reviewers flag them and send the set back.
Edit tasks
Editing one element changes what else must be true. If a number is updated, every slide that quotes it, every chart that plots it and every total that includes it has to change too. Trace those dependencies before you write, and give each one its own criterion.
- The title overrunning the subtitle and the rule, the clipped labels in the blue column and the collided figures in the headers are all Structural. Already caught, so don't write them.
- The panel inconsistency and the duplicated flag are task-specific, invisible to a universal check, and yours to write.
The nine rules
Every criterion goes to judges, three LLM runs and sometimes a person. They have the deck, the input files, the prompt and your sentence, and nothing else. The rules are what survives that.
Five readers, one verdict. If five people read your criterion against the same deck, one of them an LLM, all five reach the same verdict. That is the sentence being gradeable.
Five peers, one standard. Five out of five experts in your own industry would agree that what you're checking is the standard, given this task. Not most of them, not your team, not your firm. That is the convention being real.
The peers judge in context. For an ordinary Quality criterion they have read this task: given this brief and these inputs, would all five agree this is what a good deck does? For a Domain Specific criterion they have not: would all five require it on any deck of this kind? Five of five is a thought experiment you run, not a vote you hold, and QA checks it afterwards.
They fail independently: the legend sits below the chart is gradeable and still a preference. A criterion judges disagree on is worth nothing; one your peers would argue with teaches the model your habit.
Rules 1 to 5 decide the first test and are the ones judges split on. Rules 6 to 9 keep the criterion honest and light. All nine apply.
Whether a criterion can be assessed without ambiguity is the single biggest determinant of whether judges agree, ahead of coverage, difficulty and domain knowledge. Every unstated threshold you remove buys more than another criterion added.
Source: arXiv 2607.00139
Rules 1 to 5: what decides whether judges agree
Name the element and the source precisely
A criterion names everything the judge needs to find the thing and run the check: the file, the field inside it, and what counts as a match. It is read alone, so this chart and the one above have nothing to point at, and no source supplies your intent, so the right colour stays ungradeable.
financials.csv, within one unit at the displayed scale.Which figure, which file, which field, and how close is close enough.
Make the check objective and observable
A criterion is one yes-or-no question about the slide, and every reader has to reach the same answer. Write it so the judge can answer by looking: which element, which property, what counts as a pass.
Three traps. "Every", "all" and "only" mean one exception fails, so if you'd forgive a few, say how many: at least 90% of process labels begin with a verb. Grammar terms such as "verb + object" get read three ways, so give the pattern with an example: starts with a verb, as in "Pay by card". Relative words such as "clearly visible" leave the judge to invent a threshold, unless the reference is explicit: no larger than the body text beside them names what it is compared against.
Test one independently failing property at a time
Stacking leaves the judge guessing whether one hit is a pass. Compound criteria produce inconsistent grading: partial satisfaction may receive full credit, or partially correct work may receive none.
The test is whether one clause could fail while another passes on the same element. The note sits above the source line and is set in italic is two criteria, because a note can sit in the right place and not be italic. Each slide title is a full clause, with a subject and a verb is one, because the clauses define what a full clause is. Whether that title is supported by the slide's exhibits is a second criterion, because a clause can be unsupported. Paired items work the same way, so EBITDA and EBITDA margin share one criterion.
Existence comes before quality. One criterion tests that the element exists, a second tests its quality where it is there. Fold them together and an output that added no flag scores the same as one that added a square one. For an optional element, write only the conditional form.
Where growth arrows are present, each one starts at the top of the earlier bar and ends at the top of the later bar.
When in doubt, split. A criterion that reads as one judgment to you is the one judges split on.
Don't overfit or leak an answer
You are describing what good looks like, not the output in front of you. A criterion is only as specific as the prompt was, so if the prompt said "Slack-coloured pins", demanding one pin shape fails a legitimate reading. Name an exclusion only where it stops a predictable misread, as in decision diamonds and start/end nodes are excluded.
A name, a number or a sentence copied out of one output teaches the model that output instead of the standard behind it. FY24 group revenue is 1,234 and FY24 group revenue matches the FY24 Group Revenue value in financials.csv check the same thing, so prefer the second where the source is unambiguous.
Describe the deck state positively
Describe what you want to see, not the failure you saw. Writers working from the outputs drift into the title is scribbled or the chart is not cluttered, which name a defect in one deck and say nothing about what a good one shows. LLM judges also read "does not" badly. The contrastive tail is the same mistake: if the bars are blue, they're blue, and rather than green only adds a second thing to disagree about. Keep "avoids" where absence genuinely is the point.
Rules 6 to 9: what keeps the criterion honest and light
Describe the deck, not the action
A criterion is a statement about what is on the slide. A judge can't grade an instruction; it can only look for the state you describe.
Name a shape only where five of five would build it. Otherwise describe what the visual shows.
One data point is enough
You're checking whether the model carried the numbers across, not auditing the deck. Name the metric, the entity, the period and the tolerance, then state the value or point at where it lives.
financials.csv, within one unit at the displayed scale.Or state it outright, as in FY24 group revenue is 1,234, within one unit at the displayed scale, where the figure isn't worth protecting. Rule 4 says which to prefer.
Keep it short
If you have to read your own criterion twice, so will everyone else, and they'll each land somewhere different.
Check your own facts
The criterion itself has to be right. One carrying a wrong figure or a misremembered convention doesn't just misgrade; it teaches the model the error. If your criterion says the CAGR is 14.9% and the arithmetic gives 15.4%, every correct deck fails. Recompute anything you derived by hand, and check any convention you wrote from memory, before you submit.
Source: arXiv 2511.11562
What not to write
Do not duplicate properties already covered by the universal checks, and don't write criteria the judge cannot settle from the deck, the input files and the prompt. A criterion is not excluded merely because it is reusable or could be checked programmatically.
Anything already automated
Structural checks run on every deck without you. They cover unnecessary white space, overlapping shapes and text, font size and template match, along with mechanical faults such as a chart drawn as a stack of shapes, and a file that won't open scores zero. They also catch the mechanical signs of machine-written text: em dashes beyond light use, emoji, typed arrows and symbols, mixed quote styles, typed bullet glyphs and leftover generation residue such as unresolved placeholders. Writing any of it again penalises the same mistake twice; a stricter version of the same check still fails the same decks a second time.
The chart check decides only whether the chart is native and editable. A content criterion about a chart is judged on what the chart shows, whether it is a native chart, a picture or a stack of shapes.
The AI "dashboard" look, meaning big coloured cards and oversized metric boxes, is covered centrally too.
One carve-out: the automated checks cover geometry, meaning where things sit and whether they collide. They do not cover whether the content is too much for the slide, so density is yours to write, as a countable ceiling.
Overlaps are split two ways. An overlap that is wrong in any order, such as two words colliding, is automated. An overlap where the order is the point, such as the CAGR label sitting in front of the arrow, is yours to write.
The list above isn't everything the universal checks cover. Run them on your own outputs before you start writing, because what they flag is your working list of what not to duplicate. Anything borderline, ask in the channel.
Anything too fine to be worth a criterion
A margin, an inset or a point size is readable in principle, which is not a reason to write a criterion about one. Fine visual geometry fails three tests: the structural layer covers it, the reader would not notice it, and a sentence built on roughly aligned splits judges.
Either a reader would notice the condition and you have stated it exactly, or it belongs to the structural checks, or it goes. A one-pixel clearance, a label that barely fits or a shade that is nearly right fails that test.
A measurement no reader would notice, on a property the structural layer already covers.
One judgment, the order is the point, and a defect a reader would notice. Text spilling out of its shape is containment, which the structural layer already checks.
Don't pair an exact property with a vague impression. "The title is set in the largest font size, with all other text visibly smaller" asks for one thing the judge can measure and one it has to guess at. Pick the one you mean: the title is the largest text on the slide; standalone KPI figures are excluded.
Anything generic about the design
Legibility, contrast, overlapping elements and too much white space are not yours, because the universal checks cover them. Do not duplicate properties already covered by the universal checks. A criterion is not excluded merely because it is reusable or could be checked programmatically: an industry convention can apply across many tasks and still belong in your set.
But do write the task-specific version
The universal library catches what's true of every deck. It cannot catch what's true of yours:
- The prompt asked for the ellipses on top of the arrows; the output reversed them. (Outcome, because the prompt fixed it.)
- The brand guide specifies grey; the output rendered it blue. (Outcome, because the brand guide fixed it.)
- The template's horizontal rule is missing on half the slides. (Outcome, because only your task knows what the template puts there.)
- In your industry, notes sit above sources, in italic. (Quality, because nobody fixed it and you brought it.)
- Density, as a countable ceiling: two lines per bullet, five bullets per slide, three sentences of commentary per panel, no more than four series on a chart. (Quality, countable from the render. "Not overcrowded" and "enough white space" are not, and the judge can't assess them reliably.)
Anything experts disagree on
A criterion has to hold across your industry, given this task, not just your firm. That is the peer test from section 4: if one credible peer would defend a different choice, it stays out. A peer is someone in the same industry and function who would sign off on this deck, not a colleague who shares your habits.
What makes a set hold up
A set of individually good criteria can still mislead: by scoring one mistake twice, by leaving an axis untested, or by collapsing the moment one thing gets fixed. Three checks, run when you think you're finished.
Does it cover everything that matters?
Picture the deck that passes every criterion you wrote and trips none of them. Would you send it? If not, whatever you would still fix is a criterion you haven't written. Write those three things down and add the material ones, classified by source.
A deck carrying the right numbers and none of the reasoning is not one anyone accepts, so the supporting narrative belongs in the set too.
Interpretation is the same. A correct figure with the wrong conclusion beside it is a deck the reader sends back, so a set needs criteria that check the claim against the exhibit, not only the exhibit against the source.
The axis that goes missing most often is the prose one. Models find the pitfalls a rubric says nothing about: padding a slide out, hedging a takeaway into meaninglessness, restating the chart in bullets. Nothing else catches what nobody wrote down.
Does the set test whether the deck is correct and complete, and whether it communicates effectively? A set that only checks the figures, or only the presentation, has an axis missing.
Before you finish, walk the set against this list. Each item has a criterion, or you can say why it doesn't apply to this task.
- Audience fit: titles and commentary pitched at the reader the brief names or implies
- Storyline and slide titles: each title states a conclusion the slide supports (a Domain Specific convention; write it where the prompt says who the deck is for)
- Numbers and sources: the critical figures traced to a named input, with a tolerance
- Interpretation: the conclusion drawn matches the figures shown
- Scope: everything the brief asked for is present, and where it fixes a count or a list, nothing beyond it is added
- Density: countable ceilings on bullets, lines and series, and on deck length where the brief leaves it open
- Chart choice: the visual suits the data
- Template consistency: what the template fixes that the universal check can't know
Sources: arXiv 2607.00139, arXiv 2509.21500, Emergent Mind, rubric reward models
Can one mistake cause more than one failure?
Each claim belongs in exactly one criterion. It goes wrong three ways:
- The same fault written from two angles. Two criteria, one root cause, two fails.
- A deck-wide criterion and a slide-specific one on the same element. Every content slide title states a takeaway and the market overview title states the slide's conclusion: a failure on the market overview title fails both, so that one error is penalised twice. Take the general form when the judgment recurs across the deck, the specific one when a single slide is the only place it bites.
- Your criterion and an automated check on the same property. Section 5.
Two criteria can also cover different aspects and still lean on the same figure: check FY24 group revenue against its source in three criteria and one wrong number costs three fails. Before you submit, read the set for pairs that would fail for the same reason, and keep one.
This applies to accuracy: check each figure against its source once. On an edit task, the slides that depend on the edited figure get consistency checks instead, written relative to each other: the headline figure on the summary slide matches the value in the updated chart. A stale headline next to a correct chart fails once, and a wrong source value fails once.
Source: arXiv 2607.00139
Are the failure modes diverse?
If everything your set catches traces back to one root cause, a single upstream fix takes the task from hard to trivial. Spread the failures across different kinds of mistake.
Source: arXiv 2607.00139
Calibrating the set
How many
20 to 30 criteria, at least 15 of them Quality, for a deck of four or five slides. A longer deck needs more, roughly in proportion, so a ten-slide deck needs about twice as many, with the Quality share kept. Count the Quality ones by hand, because the platform doesn't show the split yet. Don't pad the set: 40 weak criteria on five slides are worse than 20 strong ones, and a set with 12 is not finished.
Aim at a shape, not a pass rate. Roughly a quarter of the set are must-haves any competent build passes, half test a judgment, and a quarter are stretch.
Tag each one
Outcome or Quality. Deck-wide or slide-specific. Deck-wide means the criterion applies across the deck, or to a defined recurring class of slides, rather than to one named slide, as in the title sits above the blue rule and the subtitle below it, or every content slide after the cover carries a source line at the foot.
Name the class in the sentence, because the judge reads the sentence, not the tag.
Two more tags on the platform
Both are optional and most criteria carry neither. They are descriptive and never change how a criterion is judged. Both mark judgment you brought, so an Outcome criterion carries neither, and a criterion can carry both: consulting bullets in full sentences are Prose and Domain Specific.
Prose: tick it when you set how the text should be written: the language a profession uses, how much of it there is, and the practices that keep writing from reading as machine-generated. The test: could rewriting the text, without changing any figure or conclusion the criterion names, change the verdict? If yes, tick it.
That covers the form of the message (every content slide title states a conclusion), whether commentary adds to the exhibit beside it (the bullets beneath each chart state something its labels do not), how much text there is (each bullet runs to at most two lines; each content slide carries at most five bullets), mechanics (bullets under one heading share one grammatical form) and word choice (each competitor carries the same name throughout the deck). It also covers deck length when the prompt does not specify the number of slides. Set the ceiling from industry consensus for this brief, not your own preference. Peers rarely agree on an exact count, but they do agree on a range, so put the ceiling at the top of it: if five of five would answer the brief in five to eight slides, the deck has at most eight content slides catches the model that builds twenty, and any deck inside the range passes. When the prompt fixes the count, the criterion is Outcome and carries no tag.
Do not tick it when the criterion checks what the text says, meaning a specific figure, driver or conclusion (the title states savings of $4.9m; the commentary attributes the margin fall to freight costs): the words carry it, but it tests a fact. Formatting and position are layout, not Prose (titles are bold), and so is how many charts, series or boxes a slide carries. Wording or length the prompt, the inputs or the template fix is Outcome. Em dashes, emoji, typed symbols and mixed quote styles are caught automatically, so don't write criteria for them.
Domain Specific: tick it when the criterion checks a convention of your profession. That means something five out of five experts in your field would expect on any deck of this kind, without having seen this task, and that the prompt and input files do not already require. Domain Specific criteria are always Quality.
The convention can be about presentation, such as takeaway titles, notes above sources, or one idea per slide. It can also be about substance, such as multiples suffixed with x, or minimum DSCR and LLCR on a project finance returns slide. It does not have to belong to your field alone: consulting, banking and private equity all use takeaway titles, and all of them tick it.
Do not tick it when the criterion is:
- Required by the prompt, the inputs or the template. That is Outcome, never tagged. The deck has 10 slides.
- A conclusion or choice you reach for this task. Quality, untagged. The slide notes that the EBITDA uplift may overstate underlying earnings.
- True of any good deck. Untagged. Every table states its units.
- One person's or one firm's preference. Not a criterion at all. The legend sits below the chart.
Specialist vocabulary on its own is not enough: a criterion about EBITDA can be pure Outcome. Keep the convention and the value apart as well. The returns slide states minimum DSCR is Domain Specific; where the returns slide states minimum DSCR, it is 1.19x is Outcome, because the supplied financial model fixes the value, not because it is a number. Write the value check conditionally, so a deck that leaves the element out fails once, not twice.
The tag tells the model when to apply a field's conventions. A deck for someone who never said they were a banker should not be built like a banking deck, so the prompt has to say who the deck is for. Once a criterion has cleared the list above, run the last test: should a deck for a different profession be allowed to ignore this? If yes, tick it. If no, it is universal and stays untagged.
Section 10 carries the tags on its examples.
Test it on the platform
Your set gets judged three times over, and the platform shows you where the three disagree. Read it before you submit.
Run the set against your outputs and open Mixed verdicts, where a criterion lands when one judge disagrees with the others on the same deck. Read the verdicts, not just the count: it is the cheapest signal you get on whether the sentence can be assessed without ambiguity.
A split usually means the criterion is ambiguous. Sometimes the thing you asked about has no threshold in it, as with visibly distinct, and no rewording helps until you state one; sometimes a criterion points at a source without saying which file or field. Judges do get things wrong, so read the reasoning before deciding which it is.
A criterion marked consistency not checked or incomplete hasn't been run: edit it, save it, and the check reruns.
How to fix an ambiguous criterion
Run the split criterion back through rules 1, 2 and 3, and if a category recurs across your set, define it in every criterion that uses it, in the same words each time, because each criterion is read alone: a supporting-commentary panel is a filled or bordered prose box beside or below a chart, explaining that visual. If it still splits the judges, drop it: a criterion three judges can't agree on measures the judges, not the deck.
Then run the set again. The universal checks run at the same time, so anything they flag that you also wrote, cut yours.
The checklist
Run the set against this before you hit Finalize. Anything you can't tick, fix or cut.
- 20 to 30 criteria for four or five slides, more for a longer deck, at least 15 of them Quality. Each one tagged Outcome or Quality, and deck-wide or slide-specific, with Prose ticked where I set how the text is written and Domain Specific where I check a convention of my profession, never on Outcome.
- Every criterion names what the judge needs to run it: the element, and for a check against an input, the file, the field and the tolerance. The expected result is in the sentence.
- Every criterion is a statement about the deck, not an instruction to the model.
- No criterion stacks two properties that could fail independently, existence and quality are separate criteria, and every criterion has a single answer with no undefined threshold. A relative comparison is fine where the reference is explicit and checkable.
- Phrased positively, with "avoids" only where absence is the point. No "rather than". No "this" or "that". Every "every" is meant literally.
- No two criteria fail for the same reason, each figure is checked against its source in exactly one of them, and nothing duplicates an automated check.
- No criterion quotes a name, a number or a sentence the prompt or the input pack didn't fix, and every criterion is factually right in itself, with figures recomputed and conventions checked.
- Every error-derived criterion is material, grounded in the task and checked on a rerun, and every Quality criterion would be agreed by five of five experts in my industry, given this task.
- At least a third of my Quality criteria test a professional judgment, not execution.
- Every coverage item in section 6 has a criterion or a stated reason it does not apply.
- On an edit task, every slide, chart and total that depends on the edited element has a consistency check, written relative to the edited figure.
- The deck that passes every criterion is one I would send, verbosity and padding included.
- Mixed verdicts reviewed, every criterion that split the judges fixed or gone. Read cold: would someone else grade this deck exactly as I would?
Worked examples
Real criteria from a finished Atlas set, a thirteen-slide commercial due diligence deck on First Mile, a UK waste company.
How a set is put together
Every criterion carries an id, a category, and a scope of deck-wide or the slide it applies to.
Two slides, annotated
Each criterion is numbered against the part of the slide it tests.
- 1OutcomeThe revenue chart carries a CAGR overlay drawn across the plot area spanning FY18 to FY24, labelled with the compound annual growth rate of 14.9% (acceptable range 14.1% to 15.6%).Rule 7 in practice: one number with a tolerance, because you're checking whether the model can compute and place a CAGR, not auditing seven years. Recompute the 14.9% before you submit it.
- 2QualityIn every exhibit on this slide showing FY20 and FY21, including the chart and any table repeating the same years, those years use a fill, outline, marker or pattern not used for the other years.Every expert would flag the COVID years, so the judgment is shared rather than personal. A model can still "distinguish" FY20 by colouring every year differently, which a second criterion shuts: the years other than FY20 and FY21 are presented consistently with each other.
- 3OutcomeThe table beneath the chart shows only EBITDA and EBITDA margin by year."Only" fails the output that helpfully adds a revenue row. It is also the legitimate use of stacking, since EBITDA and its margin are one judgment.
- 4QualityEach table column sits directly below the bar for the same year: the FY18 column under the FY18 bar.The chart and the table only read as one exhibit if each column sits under the bar for the same year. "Aligns" on its own would be argued over, so the criterion names one concrete case and the judge applies the same test to the rest.
- 1OutcomeEach segment's competitors on the 2x2 are enclosed by a single outline, with no competitor from another segment inside it.Asked for in the prompt. The draft read "sit next to each other or in close proximity", which sounds forgiving but hands the judge an undefined threshold. The outline is what makes the grouping checkable.
- 2QualityEach competitor on the 2x2 is shown as a logo.On a competitive landscape slide in a due diligence deck, five of five would expect logos, so it is safe to test, and it carries the Domain Specific tag. The first draft read "shown as a logo, not as a text label alone", and rule 5 says describe the state you want and stop.
- 3QualityAxis labels avoid including arrows.The right use of "avoids": absence is the point, so there is no positive form to state instead. Error-derived, and it recurred.
- 4RejectedThe legend is placed below the chart.Legend placement is a preference: two people with the same background will place them differently and both be right. Gradeable, and still fails the peer test.
More from the same set
From other slides in the same deck, deck-wide criteria first.
Deck-wide
The prompt asked for eight. The second clause earns its place: without it, judges disagree about whether the cover counts.
Not asked explicitly, but the convention is settled across the field, which is what makes it worth writing and what earns the tag. The italic is its own criterion, because a note can sit in the right place and not be italic.
Density as a countable ceiling. Rewording the bullets shorter changes the verdict, so it is Prose. Bullets and sentences per slide are Prose too; how many charts, series or boxes a slide carries is layout.
Slide-specific
Internal consistency, answerable from the slide alone. Mixing reported with pre-exceptional EBITDA across two adjacent years is exactly the error that gets a deck sent back.
Criteria from other tasks
Several come from edit tasks, where the criterion describes a change to a deck the model was given.
"Reads top left as" fixes an arrangement without leaving a threshold to argue about, and it works for panels, columns and rows alike.
Chart choice, in three parts: judgment verb, specific object, testable condition.
Gradeable, defensible and still a house style: the industry accepts bottom-left and bottom-right, so five of five do not agree. One firm's preference is not a convention, so it would not carry the tag either.
Before and after, from QA
Criteria as they arrived for review, and what they became.
"The pin style described" makes the judge interpret the brief. Rule 1: write the result in.
The title of the market overview slide states the conclusion of the slide.
A failure on the market overview title fails both, so one error is penalised twice.
Tagged Domain Specific and Prose: consulting, banking and private equity all expect takeaway titles, none of the task materials asked for them, and the criterion sets the form of the title, not what it says.
What changed
Revision of 24 September 2026, against the version of 14 September. If you have written a set before, read these rows and re-check any criteria they touch.
| Where | What changed |
|---|---|
| Section 1: What a rubric is | End-user happiness is mostly content, numbers and narrative, with design and layout as the supporting layer. |
| Section 2: Outcome and Quality | Outcome is what the prompt, an input file or the template fixes, including figures calculated from the inputs. Everything else is Quality. |
| Section 2 | Verbosity and prose are Quality where your judgment sets the expectation; an explicit length limit or wording in the prompt is Outcome, untagged. |
| Section 2 | The template is followed where it explicitly fixes something. A departure is Quality only from a template default (such as body-text size on chart labels), when five of five would make it, and the template criterion must not fail it. |
| Section 2 | Specific formats (a chevron, a 2x2 layout) are Quality only if five of five would build them; otherwise describe what the visual shows and leave the shape open. The opening example now names the audience (a strategy consulting client). |
| Section 3: Build | 20 to 30 criteria, at least 15 Quality, for a four-or-five-slide deck; more for a longer deck. |
| Section 3, step 4 | Take deck-wide faults first, then slide by slide. A criterion earns its place by being material and grounded; how many outputs fail it informs, a rerun checks it is stable, neither decides. |
| Section 4, rule 1 | Definitions travel with every criterion that uses them, in the same words, because each is read alone. |
| Section 4, rule 3 | Title form and title support are two criteria. Compound criteria grade inconsistently. |
| Sections 4, 5 and 9: Peer test | Five of five peers agree given this task. For Domain Specific criteria the peers have not seen the task. It is a thought experiment you run; QA checks it afterwards. |
| Section 5: What not to write | Containment (text inside its shape) is a structural check; overlap order (the CAGR label in front of the arrow) is yours. |
| Section 5 | The automated checks also catch em dashes beyond light use, emoji, typed symbols, mixed quotes, bullet glyphs and generation residue. Don't write criteria for them. |
| Section 6: Coverage | Scope: everything asked for is present; where the brief fixes a count or a list, nothing beyond it is added. Density now includes deck length where the brief leaves it open. |
| Sections 6 and 9: Dependencies | Check each figure against its source once. On edit tasks, dependent slides get consistency checks written relative to the edited figure. |
| Sections 6 and 10 | A deck-wide title criterion and a slide-specific one on the same title: a failure on that slide fails both, so keep one. |
| Section 7: Tags | Prose covers how the text is written: the form of the message, whether commentary adds to the chart instead of restating it, bullets and sentences per slide, mechanics and wording. Test: could rewriting the text, without changing any figure or conclusion the criterion names, change the verdict? What the text says, formatting, and how many charts or boxes a slide carries are not Prose. |
| Section 7: Tags | Prose also covers deck length when the prompt doesn't specify the number of slides. Set the ceiling from industry consensus at the top of the range peers would accept for this brief. |
| Section 7: Tags | Domain Specific redefined: a convention of your profession that five of five would expect on any deck of this kind, and the task did not require. Test: could a deck for another profession ignore it? The prompt must say who the deck is for. |
| Section 7: Tags | Both tags are optional, go only on Quality criteria, and a criterion can carry both. Retag any Outcome criteria that carried Prose. |
| Section 10: Examples | Tags added to the takeaway-title, note-above-source, logos and new concision examples; removed from the rejected bottom-left source line. |
Assessment
Twenty-four questions drawn from sections 1 to 10. Twenty correct is a pass, and a pass is what clears you to pick up your first task.
Register with your name and email before you begin. Anything left blank counts as wrong, but you get the explanations back either way. Your result is recorded when you submit, and you can retake the assessment as many times as you need.