I have a side project that rewrites short news stories at several reading levels for language learners. One story goes in, and four versions come out, from “just starting” up to “fairly comfortable.” A language model writes each version, and a set of checks in code decides whether each draft is good enough to publish.

For a long time those checks looked at each level on its own. Is the beginner version short enough? Does the upper level stay inside its word range? Are the facts the same as the source? Every level could pass, and the set could still be wrong in a way no single check could see: on some stories, the “harder” version had shorter sentences than the “easier” one.

This post is about the fix, which is simple and applies well beyond language learning: when you generate a ladder of anything, check the rungs against each other, not just against a target.

Why per-level targets are not enough

A per-level rule says something like “upper intermediate should average 20 to 28 words per sentence.” That is a fine rule, but it is a band, and the bands for neighboring levels are close. If the model lands near the bottom of the upper band and near the top of the lower band, both pass and the difference is a word or two. Readers moving up a level feel nothing change.

Worse, each level was written separately from the same thin source. Nothing tied the drafts together, so random variation alone was enough to flip the order now and then. When I measured a recent batch, a bit under half the stories had a step of less than 2.5 words per sentence between the top two levels, and a handful had the order fully inverted.

You will see the same shape in other places:

• Pricing tiers where a cheaper plan accidentally includes more than the next one up.
• Summaries at “short,” “medium,” and “long” that come out within a few words of each other.
• Test fixtures at “small,” “medium,” and “large” where medium is the slowest.
• Thumbnails at several sizes where one size is heavier in bytes than the next.

In every case each item can be valid while the relationship between them is broken.

Measure the step with the same ruler everywhere

The first job is to make the measurement boring and shared. I already had a reporting script that computed mean sentence length from the published HTML. The generator needed the exact same number, not a close cousin, or the generator would think it had fixed a story that the report still flagged.

So I pulled the formula into one small, pure function and used it in both places:

const strip = (html) =>
	String(html)
		.replace(/<[^>]*>/g, ' ')
		.replace(/&[a-z]+;/gi, ' ')
		.replace(/\s+/g, ' ')
		.trim();

export function meanSentenceWords(html) {
	const text = strip(html);
	const words = text.split(/\s+/).filter(Boolean).length;
	const sentences = text
		.split(/[.!?]+/)
		.map((s) => s.trim())
		.filter((s) => s.length > 1).length;
	return sentences ? words / sentences : null;
}

It is crude. It does not know about abbreviations or quotes that end in a question mark. That is fine, because the point is not linguistic accuracy, it is that every part of the system agrees on one number. Returning null when there is nothing to measure matters too: an empty draft should not silently compare as zero and “pass.”

The generator writes Markdown and the report reads HTML, so the generator converts its draft to HTML before measuring. Same input format, same function, same answer.

Write the levels in order and compare to the one above

The second change is about order. If you write the hardest level first, then each easier level has a finished neighbor to compare against. My pipeline already wrote from the top down, so the check for a level only needs the text of the level just above it:

export function sentenceStepProblem(level, text, higherText, minStep = 3) {
	if (level !== 'B1') return null;
	const mine = meanSentenceWords(text);
	const above = meanSentenceWords(higherText);
	if (mine == null || above == null) return null;
	if (mine <= above - minStep) return null;
	const target = Math.floor(above - minStep);
	return `sentences too long for this level: mean ${mine.toFixed(1)} words, the level above has ${above.toFixed(1)}; aim for at most ${target} words per sentence on average, without dropping facts`;
}

A few choices are worth calling out.

It returns a sentence, not a boolean. The output goes straight into the list of problems that feeds the model’s rewrite. “Too long” is useless to a model. “Your mean is 20, the level above is 21, aim for 18 or less, keep the facts” is something it can act on. The last clause matters, because the easiest way to shorten sentences is to delete content.

It is scoped to the rung that was actually broken. I only had data showing the top step was too small, so the check only runs there. The lower steps already had a healthy gap. Adding the same check everywhere would have spent revision budget on stories that were fine.

It is a hint, not a gate. This was the most important decision. The check adds a line to the revision request, uses the existing retry budget, and never blocks a story from publishing. If the model cannot get the step right in the allowed attempts, the story still goes out, and the report counts it. A hard failure would have traded an occasional flat step for more stories stuck as drafts, which is a worse outcome for readers.

Fix the prompt too

The check catches drift, but the cheapest fix is to not drift in the first place. The two upper levels had no numeric sentence guidance in the prompt at all, while the two lower levels did. So the prompt now gives each upper level its own non-overlapping average, with room between them. The check in code is still the authority; the prompt just makes the first draft pass more often.

Decide how you will know it worked

Before merging, I wrote down the metric and the bar: the share of new stories whose top step is under 2.5 words, measured on the first 20 stories after deploy, with a target of 3 or fewer, zero inversions, no rise in the draft rate, and the upper level not drifting past its band to win the gap. That last condition keeps the fix honest. You can widen a gap by making the top level bloated, and that is not the goal.

Tests cover the edges you would expect: exactly three words below passes, two and a half fails, other levels never trigger, an empty neighbor returns nothing, and the hint only shows up when the higher text is passed in.

The short version

• When you produce a ladder of variants, check the relationship between neighbors, not only each variant against its own band.
• Use one shared measurement function everywhere, so the generator and the report agree.
• Generate from the top down so each step has a finished neighbor to compare against.
• Return an actionable sentence the model can use, and protect the content while it revises.
• Make it a revision hint inside your existing budget, not a new way to fail.
• Write down the metric and the guardrails before you ship, including one that stops you from gaming the gap.