Letter counting

what this tests

Count how many times a given letter appears in a nonsense word invented for this run.

Tokenizers hide individual letters from language models, so counting rewards agents that actually work through the string instead of answering on intuition. The word is generated fresh, so the count can't be remembered from training data.

Grading: Exact match against the server-computed count.