Improving Example Sentences with Claude Skills, Part 1: The Skill
Every vocabulary card in AI Japanese Tutor has an example sentence. Our first batch was AI-generated. That was fast to produce but uneven in quality. Some sentences are too long for beginners, some drift off-topic, and some use words above the card's JLPT level. We have an evaluator that scores each sentence 1–10, and we started with 647 sentences out of 14,378 scoring 5 or below.
That's a lot of sentences to fix, so we've been chipping away at them one PR at a time.
One sentence, one PR
We built a Claude skill: improve-example-sentence. It handles one sentence at a time. A single invocation produces a single PR that:
- Picks the next unprocessed sentence at or below a threshold you give it
- Writes a replacement Japanese sentence and English translation
- Updates the relevant vocabulary data
- Opens the PR
That's it. The diff is tiny. Reviewing it takes a minute. We've merged 23 of these so far, each with its own focused conversation in the PR.
Why a skill instead of a prompt
We could have just told Claude "rewrite this sentence." We tried that. The results were inconsistent: sometimes too literary, sometimes drifting above the JLPT level, sometimes not actually focusing on the target word.
A skill is a packaged process. It includes:
- A rubric for what a good sentence looks like at each JLPT level (length, register, grammar)
- Explicit rules about which fields to edit and which to leave alone
- The exact filename convention for the new SQL block
- The git/PR sequence, with the repo and PR number pinned
What changed in the sentences
A typical improvement looks like this. The card is for 秘密 (himitsu, "secret", N3). The old sentence scored a 2. The Japanese was missing punctuation, and the English translation silently dropped the subject entirely:
友達には秘密です It's a secret
The replacement is complete, natural, and makes the target word do real work:
これは私たちだけの秘密だ。 This is a secret just between us.
Both the Japanese and English now feel like something a person would actually say. That's the kind of judgment the skill encodes in its rubric so we don't have to re-explain it every time.
What this pattern is good for
Claude is just doing the writing here. The useful part is the process: a batch operation becomes a sequence of small units, with the conventions and guardrails written down so each one comes out consistent.
That helps anywhere you have a long tail of similar fixes, like copy edits, lint cleanups, or data migrations. Splitting the work into pieces small enough to actually read is what makes each one mergeable.
In Part 2 we look at running this skill unsupervised inside a Docker sandbox, so Claude can open PRs in the background without you watching, and how automated review catches anything that slips through.