I already wrote about checking generated text in code and retrying exactly once. That post was about limits the prompt claims to enforce: title length, word ranges, that kind of thing. Two quieter failure modes kept showing up in the same pipeline, and neither is a length problem.

The first is garbage in. A feed item whose “description” is only a publisher disclaimer or a paywall notice gives the model nothing real to rewrite. It will still produce an article. It will just be about investing, cookies, or “subscribe to continue,” because that was the only prose in the prompt.

The second is leftover source language. The articles are supposed to be in the target language. Most of the time they are. Every so often a common English news word slips through unchanged: “hurricanes” instead of the French equivalent, “lawsuit,” “lawmakers,” “meanwhile.” A reader notices. A search snippet notices. The prompt already said not to do that; saying it again did not help.

The fix is the same shape as the length check, applied in two places: reject empty sources before generation, and measure leftover source words after.

Skip sources that are not sources

Before ranking or clustering feed items, drop anything whose text matches a short list of boilerplate patterns. The list is boring on purpose: real articles almost never consist of only these sentences, and the false-positive cost of skipping one item is low.

const BOILERPLATE_PATTERNS = [
	/for informational and educational purposes only/i,
	/should not be construed as an offer/i,
	/subscribe (now )?to (continue|read)/i,
	/this (content|article) is (only )?available to subscribers/i,
	/please enable (javascript|cookies)/i,
	/we use cookies/i,
];

function isBoilerplateSource(text = '') {
	return BOILERPLATE_PATTERNS.some((re) => re.test(String(text)));
}

function keepUsableItems(items, log = () => {}) {
	return items.filter((item) => {
		if (isBoilerplateSource(item.description)) {
			log(`Skip boilerplate source text: ${item.title.slice(0, 80)}`);
			return false;
		}
		return true;
	});
}

A few choices matter:

• Match the description, not the title. Titles on paywalled items are often fine. The body (or the RSS summary) is where the footer or wall copy lives.
• Keep the list short and specific. Broad patterns like /subscribe/i will eat real news. Prefer full phrases you have actually seen in the wild.
• Log the skip. When a story you expected never appears, you want to know it was filtered here, not lost later in ranking.

I unit-tested this against the exact disclaimer that produced a bad article once, and against a normal launch blurb that must not be filtered. Regressions like that are cheap insurance.

Flag leftover source words in code

After generation, the same verify step that checks claims and word ranges also scans for a small dictionary of source-language words that should not appear in the target text. Each entry points at a suggested replacement so the rewrite prompt can be concrete.

const ENGLISH_LEFTOVERS = {
	hurricane: 'ouragan',
	hurricanes: 'ouragans',
	lawsuit: 'procès / plainte',
	lawmakers: 'élus',
	meanwhile: 'pendant ce temps',
	however: 'cependant',
	// …grow this from real misses, not from a dictionary dump
};

function englishLeftovers(html) {
	const text = toPlainText(html).toLowerCase();
	const found = [];
	for (const word of Object.keys(ENGLISH_LEFTOVERS)) {
		const re = new RegExp(`(^|[^a-zà-ÿ'-])${word}(?![a-zà-ÿ-])`);
		if (re.test(text)) found.push(word);
	}
	return found;
}

function leftoverProblems(html, label) {
	return englishLeftovers(html).map((word) => ({
		where: label,
		type: 'English word left in French text',
		message: `Replace "${word}" with ${ENGLISH_LEFTOVERS[word]}. Keep the same facts.`,
	}));
}

Word boundaries matter. A naive includes('and') will match half of French. The regex above requires a non-letter on each side, and it knows accented letters count as letters so you do not split inside a real word.

What you leave off the list matters as much as what you put on. Sports loanwords that readers expect in the target language (touchdown, walk-off) stay allowed. So do words that are also normal French. Grow the list from live misses, one regression test at a time, instead of importing a bilingual dictionary and drowning in false positives.

Wire both into the same retry loop

Neither check needs its own control flow. Boilerplate filtering runs on the candidate list before any model call. Leftover words return the same { where, type, message } shape as title-length failures, so they share the one-rewrite path:

async function generateArticle(candidate, generate, verify) {
	if (isBoilerplateSource(candidate.description)) {
		return { skipped: 'boilerplate source' };
	}

	let output = await generate(candidate);
	let problems = verify(output);

	if (problems.length) {
		output = await generate(candidate, problems);
		problems = verify(output);
	}

	return { output, problems };
}

If leftovers survive the rewrite, treat them like any other hard failure: keep the story as a draft, or drop it, depending on how strict you want the publish path. Do not invent a second retry. The earlier post covers why once is enough.

Tradeoffs

• A static word list will never be complete. That is fine. Completeness is not the goal; stopping the words you have already shipped by mistake is. Add entries when readers or spot checks find them.
• Boilerplate patterns are publisher-specific. When a new feed starts delivering walls of legal copy, you will add a line. Prefer that over asking the model to “ignore disclaimers,” which it will sometimes obey and sometimes turn into the topic.
• False positives on leftovers cost a rewrite, not a lost story. False negatives cost a published embarrassment. Bias the list toward words with a clear, unique translation.

The prompt still says “write in French” and “do not leave English words.” I leave those lines in. I just no longer trust them alone. Filter what the model is allowed to see, measure what it writes, and keep both checks in the same boring verify step that already owns length and grounding.