Writing · Rust · TypeScript · Testing
Keeping a TypeScript validator and its Rust port in agreement
The same validation rules live in a TypeScript web app and a Rust server. A generated test corpus keeps them identical, and UTF-16 length is the first trap.
DWEEB, my Discord message builder, validates a message before it is sent: character limits, how many components a message may hold, which component is allowed inside which, required fields. Those rules were written once, in TypeScript, for the browser editor.
Then I added a server feature that needed the same rules. The server is written in Rust. It exposes the builder to AI clients through the Model Context Protocol, and a model that submits a message needs the same list of problems a person would see in the editor, before anything reaches Discord. Running the TypeScript on the server would have meant adding a JavaScript runtime as a second service. I chose to port the rules to Rust and accept the cost that comes with that choice: two implementations of one set of rules will drift apart unless something forces them not to.
This is what does the forcing, and the bugs it caught on the first day.
Split the problem: data is generated, rules are pinned
A validator is partly data and partly logic. I treat the two differently.
Data is never written twice. Limits (how long a button label may be, how many options a select menu holds), the template catalogue and the list of placeholder tokens all live in TypeScript source files. A script reads them and writes a JSON file into the Rust project, which the server embeds at compile time with include_str!. The Rust code contains no limit as a number. If I change a cap in one place, the other side picks it up on the next build.
Logic is pinned by a shared corpus. The rules themselves can't be generated, so they are tested against the same expectations on both sides. The script builds 119 small messages, most of them broken in one specific way, runs the TypeScript validator over them, and records what it reports:
{
"name": "button-at-top-level",
"message": {
"flags": 32768,
"components": [
{ "type": 10, "content": "hi" },
{ "type": 2, "style": 1, "label": "Go", "custom_id": "go" }
]
},
"errors": [{ "code": "BUTTON_OUTSIDE_ROW", "path": "components[1]" }],
"warnings": []
}
That file is committed. The TypeScript test suite checks that the validator still produces exactly these results. The Rust test suite loads the same file and checks that the port produces exactly these results. Neither side can change behaviour without the other side's tests failing.
Four rules that make the corpus trustworthy
1. Pin codes and paths, not wording. Each expected issue is a (code, path) pair. The human-readable sentence is left out on purpose. I rewrite error copy often, and a corpus that included it would fail on every wording change and teach me to regenerate it without reading the diff.
2. The generator refuses an incomplete corpus. The script knows every issue code the validator can produce. If any code is not triggered by at least one case, it exits with an error and writes nothing. So adding a new rule without adding a case for it fails the build, and the Rust port can't silently lack a rule.
3. The committed file must match a fresh run. CI regenerates the files and fails if git diff shows any change. Nobody can commit a corpus that is out of date, or edit it by hand to make a test pass.
4. The output is deterministic everywhere. Anything sorted in the file is sorted by plain code-point comparison, never with JavaScript's localeCompare. That function's ordering depends on the machine's locale and on the ICU data the runtime ships, so the file could differ between my laptop and CI, and Rust could not reproduce it. Code-point order is the same on every machine and is what Rust's string comparison does by default.
Trap one: what "length" means
The first trap is string length. The editor counts characters with JavaScript's .length, the same unit an HTML input's maxLength uses. That unit is the UTF-16 code unit. Most characters are one unit. Characters outside the Basic Multilingual Plane, which includes most emoji, are two.
Rust strings are UTF-8, and Rust offers two obvious ways to measure one. Both are wrong here:
let s = "🎉🎉🎉";
s.len(); // 12 bytes of UTF-8
s.chars().count(); // 3 Unicode scalar values
s.encode_utf16().count(); // 6 UTF-16 code units, what .length returns
With chars().count(), a message made of emoji passes the server's check at twice the length the editor allows. With len(), the same message is rejected at half the allowed length, and any text with accents or non-Latin script is cut short too. Only encode_utf16().count() agrees with the browser. The corpus has cases built from emoji that sit exactly on a limit and one unit over it, so a port that counts in the wrong unit fails immediately in both directions.
Trap two: an algorithm defined over UTF-16
The editor can turn a message into a share link by compressing its JSON with LZ-String and putting the result in the URL. The server needed to produce the same links. LZ-String is a JavaScript library, and its algorithm is specified in terms of JavaScript string indices: it walks the input one UTF-16 code unit at a time and builds its dictionary from those units.
A Rust port that iterates over chars() looks correct and works for every ASCII message. Give it one emoji and it produces a link the browser can't decode, because the two sides disagree about how many symbols the input has. The fix is to convert the input to a Vec<u16> first and run the algorithm over that. This is pinned the same way as the validator: a generated file of input strings and the exact output the JavaScript library gives for each, including strings with emoji and surrogate pairs.
One deliberate difference
The two sides differ in one place, and the corpus records it instead of hiding it. In the editor, a certain kind of malformed input (a section missing a required part) is refused at the import boundary with an exception, because there is no sensible way to repair it and the editor must never hold a tree it can't render. On the server the caller is a model that can fix its payload if told what is wrong, so the Rust side reports it as an ordinary issue. The corpus stores the Rust behaviour for that case and the TypeScript test knows about the exception. A written-down difference is fine. An accidental one is what the whole setup exists to prevent.
Was porting worth it?
For this project, yes. The server stays a single binary with no second runtime beside it, and a rule changed on one side only now fails the other side's tests at build time. The work also improved the original. Feeding model-written payloads through the new server turned up two inputs that made the TypeScript validator throw instead of report: a text block with no content field, and a link button with no URL. Both are things a person could paste into the editor's JSON import, and both now produce an ordinary validation message.
The cost is real, though. Every new rule is written twice, and the generator is one more thing to keep working. If the second implementation could have called the first one cheaply, I would have done that instead.
Summary
- Generate shared data from one source. Never copy a limit by hand.
- Pin shared logic with a corpus produced by the reference implementation and checked by both.
- Make the generator fail when a rule has no case, and make CI fail when the committed corpus is stale.
- Sort by code point, never by locale.
- JavaScript's
.lengthis UTF-16 code units. In Rust that isencode_utf16().count(), notlen()orchars().count(). - Algorithms written for JavaScript strings must run over UTF-16 units when ported.
