nitai/projects
noxstrap / src / README.md
240 lines · 12.8 KB Raw
1# Noxstrap
2
3A Roblox bootstrapper written from zero in Rust — no .NET, no Bloxstrap fork, and no GUI
4framework. The entire interface is drawn by a renderer that lives in this repo.
5
6## Rules
7
8- No `egui`, `winit`, `skia`, `wgpu`, or any other GUI/graphics crate.
9- No .NET, no WPF, no WebView, no HTML/CSS.
10- Everything down to glyph outlines and anti-aliasing is our code. The operating system
11 is asked for exactly three things: a window, input events, and "put these pixels on screen".
12
13## Crates
14
15| crate | what it is | status |
16|---|---|---|
17| `nox-gfx` | 2D engine — paths, analytic AA rasterizer, gradients, blur, shadows, compositing | **working** |
18| `nox-text` | TrueType parser + glyph rasterizer + cache + layout | **working** |
19| `nox-ui` | layout, input routing, animation, widgets | not started |
20| `nox-win` | Win32 window, DIB present, input; offscreen backend for headless dev | **working** |
21| `nox-zip` | DEFLATE decompression + zip reading | **working** |
22| `nox-json` | JSON reader and writer | **working** |
23| `nox-net` | HTTPS via WinHTTP (curl backend for headless dev) | **working** |
24| `nox-roblox` | version resolve, manifest, MD5, download, extract, launch | **working** |
25| `noxstrap` | the app — installs Roblox; launching into a game still to come | **working** |
26
27## nox-gfx
28
29No dependencies, not even for PNG output.
30
31- **Rasterizer** — signed-area coverage accumulation (one f32 buffer, per-row prefix sum).
32 Analytic anti-aliasing, no supersampling. `abs().min(1)` approximates nonzero winding.
33- **Paths** — lines, quadratics, cubics, adaptive flattening; rects, per-corner rounded
34 rects, ellipses, arcs; affine transforms.
35- **Strokes** — emitted as a union of per-segment quads plus joint discs. All contours share
36 a winding sign, so overlaps union for free and self-intersecting strokes just work.
37- **Paint** — solid, linear gradient, radial gradient, multi-stop, 256-entry LUTs.
38- **Blur / glass / shadows** — 3 box passes at reduced resolution (k = σ/3, up to 1/8),
39 bilinearly upsampled. `blurred_layer` gives a backdrop for frosted panels.
40- **Pixels** — premultiplied BGRA `u32`, which is exactly a Win32 DIB section, so presenting
41 is a single blit with no conversion. Blending is SWAR: four channels per `u32` op.
42- **Clipping** — rect clips, snapped to integers so the inner loops carry no clip math.
43
44### Dev loop
45
46The renderer is developed headless. `examples/preview.rs` draws a mock launcher and writes
47a PNG; `examples/bench.rs` times each primitive.
48
49```sh
50cargo run --release -p nox-gfx --example preview -- /tmp/preview.png
51cargo run --release -p nox-gfx --example bench
52```
53
54### Correctness
55
56`cargo test -p nox-gfx` — 31 tests, and they are the point, not decoration:
57
58- Every blend function is checked against an `f64` reference compositor, exhaustively over
59 a grid of alpha/colour combinations, to within 1/255.
60- The rasterizer is checked against a brute-force 16x supersampled reference on rects,
61 circles, rounded rects, triangles and a self-intersecting star.
62- **The SSE2 paths are checked against their own scalar twins for bit-exact equality** on
63 random rows at every length mod 4. A vector path that disagrees with the scalar path is
64 a bug even when it looks fine.
65- Plus clipping, stroke geometry, shadow symmetry and falloff, blur energy conservation,
66 transform algebra, curve flattening tolerance, and PNG chunk/CRC structure.
67
68Two real bugs came out of writing these, both invisible to the eye:
69
701. The box-blur size formula was missing its pass-count terms, so **every blur in the engine
71 was about 30% too wide**.
722. The "coverage is full, so skip blending and just store" shortcut used a threshold that
73 didn't match what coverage actually quantizes to, so the scalar and vector paths
74 disagreed by 1/255 on some pixels. Both are now pinned to `simd::ZERO` / `simd::FULL`,
75 derived from the quantizer itself, which makes the decision grouping-independent.
76
77### Performance
78
79Measured on a loaded 2-core VM, so treat these as a pessimistic floor — several times
80slower than the machines this will actually run on.
81
82| operation | before | now |
83|---|---|---|
84| full 1100x680 scene, 5 soft shadows | 44.9 ms | **14.0 ms** |
85| fullscreen rounded-rect gradient | 3.97 ms | 1.30 ms |
86| large soft shadow (sigma 26) | 8.08 ms | 1.42 ms |
87| frosted-glass backdrop (sigma 18) | 7.13 ms | 0.28 ms |
88| radial glow, r=320 | 2.43 ms | 0.78 ms |
89| card fill | 0.85 ms | 0.32 ms |
90
91What got it there, in order of payoff:
92
931. **Blur at reduced resolution** (k = sigma/3, up to 1/8) then bilinear upsample — 25x on
94 backdrops. Blurred output cannot resolve detail the blur is about to destroy.
952. **SSE2 composite** — four pixels per instruction group, for solid fills, both gradient
96 types, and shadow masks. Roughly 3x. SSE2 is baseline on x86_64, so there is no runtime
97 detection and no fallback to maintain (a scalar twin exists purely to test against).
983. **SWAR blending** for the scalar path — all four channels in one `u32` operation.
994. **Per-row span bounds** in the rasterizer instead of one bounding box for the whole shape.
1005. **Fixed-point incremental bilinear** in the shadow compositor, replacing per-pixel
101 float floor/convert/clamp.
1026. **Occlusion skipping** — `shadow_occluded` drops the shadow pixels that the opaque shape
103 casting it will cover anyway, typically half of a card shadow.
104
105The cost is now almost entirely the per-pixel composite: a large shadow spends 0.04 ms
106building and blurring its mask and 1.4 ms compositing it. Remaining wins, in order:
107
1081. Damage-rect redraw — only repaint what changed. Belongs in `nox-ui` and will dwarf
109 everything else, since a real UI changes a fraction of the screen per frame.
1102. AVX2 (8 pixels wide) behind runtime detection.
1113. Caching composited shadows for shapes that did not move.
112
113## nox-text
114
115Also dependency-free. Parses TrueType directly and feeds glyph outlines through
116`nox-gfx`'s rasterizer, so text and shapes share one anti-aliasing path.
117
118- **Tables**: `head`, `hhea`, `maxp`, `hmtx`, `loca`, `glyf`, `cmap`, `kern`, plus TrueType
119 Collections. Every read is bounds-checked; a malformed file returns an error, never a panic.
120- **cmap** formats 0, 4, 6 and 12, chosen by a platform/encoding preference score.
121- **glyf** simple glyphs (flag run-length decoding, short/long coordinate deltas) and
122 composites (offsets, scale, 2x2 transforms, nested up to 6 deep).
123- **Quadratic outlines** with implied on-curve midpoints reconstructed, emitted straight
124 into a `Path`.
125- **Glyph cache** keyed by face, glyph, size and horizontal subpixel position, with 4
126 subpixel phases and a gamma curve for stem weight. Per-glyph alpha bitmaps rather than a
127 packed atlas — an atlas exists to reduce GPU texture binds, which a CPU renderer does not have.
128- **Layout** with kerning, letter spacing, and measurement without rasterizing.
129
130Rendering is ~160 ns per glyph at 14px with a warm cache, so a screenful of UI text costs
131well under a millisecond. `cargo run -p nox-text --example specimen` writes a proof sheet.
132
133### Correctness
134
135The risk with a from-scratch font parser is quietly producing *plausible* garbage, so
136correctness here is measured against an independent implementation:
137`tests/gen_fixture.py` is a second TrueType reader, written separately from the spec using
138only Python's `struct`, that emits expected values for every font on the machine.
139
140Against **294 installed fonts**, the Rust parser must agree on:
141
142- units per em, glyph count, vertical metrics, font bounding box
143- **7,350 character-to-glyph lookups** across Latin, accented, Greek, CJK and symbols
144- **924 exact decoded point bounding boxes** for simple glyphs — this is what proves the
145 flag-repeat and short/long delta decoding is right, since any slip moves a point
146- **420 composite glyph contour counts**, so nested component transforms are exercised
147- **1,538 kerning lookups**, 656 of them non-zero, hitting both ends of the binary search
148
149Plus 2,112 truncated and byte-corrupted font cases that must be rejected or survive without
150panicking, and a test that the glyph cache never serves one face's glyph for another's.
151
152Bugs this caught:
153
1541. **The glyph cache keyed on (glyph, size, subpixel) but not the face**, so glyph 39 of
155 DejaVu Bold was served for glyph 39 of DejaVu Sans. Visible in the specimen as random
156 bold letters mid-word. Faces now carry a content fingerprint.
1572. My first kerning check silently passed while testing nothing: the pair list referenced
158 characters that were not in the probe set, so every lookup resolved to "skip". The
159 fixture now samples pairs out of each font's own kern table.
160
161### Known gap
162
163**GPOS kerning is not implemented yet** — only the legacy `kern` table. Of the 294 fonts
164here, 16 have `kern` and 243 have `GPOS`, so most modern faces currently render unkerned.
165This matters for whichever UI font gets bundled and is the next thing to build in this crate.
166
167## nox-win
168
169Raw win32 through hand-written `extern "system"` declarations — no `windows-rs`.
170`RegisterClassExW`, a window procedure, a message pump, and `StretchDIBits` with a negative
171bitmap height to present the top-down frame buffer directly. Mouse move/buttons/wheel/leave,
172keys, and character input with surrogate pairs joined. DPI awareness, minimum window size,
173and a panic hook that turns a crash into a message box, since the windows subsystem has no
174console to print to.
175
176It also ships an **offscreen backend** for non-Windows targets. That is not a stub: it runs
177the real drawing code for a given number of frames, accepts scripted pointer input so hover
178and click states are reachable, and writes a PNG. Every screenshot of the app was produced
179on a headless Linux box that way:
180
181```sh
182NOX_FRAMES=150 NOX_DUMP=out.png NOX_POINTER=342,224 NOX_PRESS=1 cargo run -p noxstrap
183```
184
185## The deploy pipeline
186
187`nox-roblox` resolves the live client version, reads `<version>-rbxPkgManifest.txt`,
188downloads all twenty-one packages, checks each against the MD5 in the manifest, inflates
189them with `nox-zip` and lays them out in the directory structure the client expects, then
190writes `AppSettings.xml` and `ClientSettings/ClientAppSettings.json`.
191
192On the build machine that is **326 MB on disk in about 25 seconds**, with
193`RobloxPlayerBeta.exe`, `RobloxCrashHandler.exe` and every content folder in place. The
194download runs on its own thread and reports package name, byte counts and throughput back
195to the interface, so the window never blocks.
196
197Everything under it is also from scratch: DEFLATE decompression (checked against python's
198zlib at every level), MD5 (checked against hashlib across 200 input lengths), JSON, and the
199zip reader. The one place the no-dependencies rule bends is TLS — implementing it is not a
200reasonable thing to ship, so on Windows this calls **WinHTTP**, a system library in the same
201sense as USER32. Off Windows there is a development backend that drives curl, used only so
202the pipeline can be tested here.
203
204```sh
205cargo run -p nox-roblox --example livecheck # resolve, manifest, download, verify, extract
206```
207
208### The bug a synthetic test could not have found
209
210The zip extractor refused every real Roblox package. Its zip-slip guard normalised
211backslashes to forward slashes, then rejected anything starting with a slash as an absolute
212path — and Roblox's packages contain an entry named exactly `\`, the archive root marker,
213which normalises to a bare slash. Every hand-written test passed, because the hostile paths
214were invented rather than observed. The fix separates three cases that were conflated: a
215path that escapes is refused, a leading slash is stripped (it cannot escape anyway), and an
216empty result is a directory marker to skip. It only surfaced by pointing the thing at the
217real servers.
218
219## Published
220
221Source, screenshots and the Windows binary: <https://projects.nitaimaarek.com/noxstrap>
222
223**The .exe has never been run on Windows.** It cross-compiles from Linux with MinGW, links
224cleanly, is a valid PE32+ GUI binary, and every import resolves against USER32/GDI32/
225KERNEL32 — but that is compile-verification, not testing. The graphics and font layers are
226platform-independent and heavily tested; the win32 layer is the part taken on faith.
227
228## Roadmap
229
2301. **`nox-gfx`** — done for now.
2312. **`nox-text`** — done except GPOS kerning, line breaking, and bidi/complex scripts
232 (none of which a launcher needs on day one).
2333. **`nox-win`** — done, pending a real run on Windows. Custom title bar still to come,
234 since we want to draw our own chrome.
2354. **`nox-ui`** — retained tree, flex-ish layout, hit testing, focus, animation clock,
236 damage tracking; buttons, toggles, lists, text fields, scroll views.
2375. **`noxstrap`** — the bootstrapper itself: channel/version resolve, `rbxPkgManifest`,
238 parallel download + extract, mods and FastFlags, protocol registration, launch.
239 Roblox-side details are already known from the previous attempt and should be ported,
240 not rediscovered.