Notes on compressing a driving video for a frozen machine judge —
the geometry, topology, and dynamics of coding for a fixed pair of neural scorers, built in
the open around the comma.ai video-compression challenge and continued past its close as a
long-horizon research program.
status working notes · grows over time · live instrument under the
LIVE tab · every training number there is advisory; only a byte-closed exact
evaluation moves the frontier pointer (…)
§ 1
The problem & the frozen scorer
The task looks like video compression but is not: the receiver is not a human eye, it is a
frozen pair of networks. A segmentation U-Net scores only the argmax of its
5-class output on the last frame of each pair; a pose network scores 6 ego-motion scalars from
a two-frame YUV stack; the third term is the raw archive size:
S = 100·d_seg + √(10·d_pose) + 25·bytes / 37,545,489
That makes this an instance of indirect rate–distortion — coding for machines, in
the video-coding-for-machines lineage — where the only bits that matter are the ones the
frozen judge can see. Pixels the scorer is blind to are free; pixels that flip an argmax at a
class boundary are everything. Our early on-ramp came from steganography: content-adaptive
embedding costs (UNIWARD) are exactly a detector-informed sensitivity field read in reverse.
§ 2
Geometry & topology of the argmax
The argmax of a smooth 5-class field partitions the image into cells whose walls — the
separatrices — form a codimension-1 complex. Measured on the real scorer, essentially
all of the segmentation distortion lives in a thin annulus around those walls (~97% of d_seg
in a few percent of the area); the interior of each cell is flat. In the frozen scorer's
Fisher information metric the margin field is an almost-exact surrogate for that
geometry (Pearson 0.978 measured), so the whole objective becomes boundary geometry: a
Morse–Smale complex over the margin field, with lane markings as the thinnest,
least-persistent — and therefore hardest — stratum. The final network layer is exactly
low-rank linear, so flip distances have a closed form, and the partition itself is a
Laguerre / tropical power diagram: store generators, not pixels.
§ 3
The four legs
Kolmogorov, not entropy. The decoder ships as a program: generic
deterministic structure is free at decode time; only the irreducible video-derived seed is
counted. Rate = |shortest program| + |seed|, a compression-as-shortest-program discipline
rather than a histogram one.
Projection. With the geometry solved, fitting is a projection: the witness is the
projection of the scene onto the intersection of the argmax, pose, quantization, and byte
constraints, taken in the scorer's own metric.
Realization. The binding limits are physical: uint8 quantization, resize kernels,
sub-pixel boundary placement. Many boundary flips are realization-limited, not
capacity-limited — precision matters where bytes do not.
Completeness. Necessity by inversion: for each stratum of the complex, ask what
the scorer requires — which bytes go to edges, which precision goes to saddles — and
ship nothing else.
§ 4
Modeling the witness
The vehicle is a task-space coordinate INR: a small implicit network trained against
the frozen scorer itself — never against RGB fidelity — so its whole capacity is spent on the
scorer-relevant manifold. Training runs through the exact evaluation round-trip (resize,
uint8, resize) so the gradient sees what the judge sees. The current generation adds
phase/advection structure: frame-to-frame appearance is carried by a low-dimensional
ego-motion screw and a sub-pixel advection phase, with per-class carriers for the strata the
scorer treats differently. Ego-motion is dual-use by construction — the same twist that warps
the partition for segmentation is the pose the second network scores.
§ 5
Scorer dynamics
Treating the frozen scorer as a physical system pays: margin fields behave like potentials,
training follows a level-set flow of the boundary complex, and curriculum boundaries act like
continuation parameters — instabilities arrive as bifurcations (island births) that can be
anticipated rather than suffered. The scorer's own architecture sets the physics: a stride-2
stem means it sees regions, not pixels; its effective receptive field, its squeeze-excitation
gates, and its exact resize kernels are all measured and folded into the model of what a byte
can buy.
§ 6
Results & frontier
Everything in the LIVE instrument is advisory telemetry from local training
hardware; a result is real only when an exact, byte-closed archive is scored by the contest
evaluator on reference hardware. That number — the frontier pointer, currently
… — moves only through that gate. The working targets
are the sub-0.19 and sub-0.15 lines; the measured rate-dominated floor of the current
formulation sits well below both, which is the headroom the program is spending down.
methods and mathematics on this page are published freely; the live
instrument shows the run of record. Attribution: video-coding-for-machines is the problem's
heart; adaptive steganographic cost was the on-ramp; the separatrix / Morse–Smale treatment
of a frozen argmax judge is this lab's own line of work.
Open the live instrument →
comma lab · working notes · this page grows with the
program — deep math · geometry · topology · scorer dynamics · modeling
authoritymacOS-MLX · advisory · non-promotablepointer… · unmovedexact rowbyte-closed · contest-CPU / CUDAbasisa dashboard is a means, not the score
Level-Set Witnessn=?· connectingws …
LIVE
CAMPAIGN
ORACLE
WITNESS
RESIDUAL
SANDBOX — why is a curvature polynomial secretly a topological invariant, and where the same music shows up here
A playground for the deep math. Prompted by a
post on Pontryagin classes:
“there’s no apriori reason some random polynomial [of the curvature] should be a homotopy
invariant — you’re defining p_i(M) using the smooth structure on M!” Everything below is
tagged MEASURED / DERIVED / ANALOGY. The Pontryagin resonance is a genuine structural rhyme
plus an actual characteristic class (Maslov) in our geometry — not a claim that we
computed Pontryagin classes or detected exotic spheres.
The Lie answer to “why a polynomial”
Chern–Weil theory. The invariant polynomials are exactly the Ad-invariant polynomials on
the Lie algebra 𝔤, and the Weil homomorphism Sym(𝔤*)^G → H*(BG) sends each to a
characteristic class. You are not evaluating a random polynomial — you are evaluating a
G-invariant, so the metric/connection choices cancel by construction and the class is an
invariant of the bundle. The miracle is a statement about a Lie group and its algebra.
The oracle we optimize against (auth-eval scorer)
The comma.ai challenge scores task-aware (“coding-for-machines”) compression of openpilot
driving video through a frozen oracle: SegNet (comma10k EfficientNet-B2) per-pixel
argmax → d_seg; PoseNet (FastViT-T12) two-frame YUV6 → d_pose; archive
bytes → rate. S = 100·d_seg + √(10·d_pose) + 25·bytes / 37,545,489.
Only the argmax partition, the pose 6-vector, and the byte count carry authority. (MEASURED)
The rhyme — parametrization-dependent-looking, secretly invariant
Our vehicle is a task-space level-set witness: a coordinate-INR that amortizes the SegNet
argmax partition directly (the viscosity solution of a variational level-set flow; the object is the
codim-1 separatrix between argmax cells). The score has his exact shape —
d_seg is a function on a quotient, invariant under every reparametrization of the witness
weights θ that leaves the argmax partition fixed:
d_seg : ℝⁿ / (argmax-cell partition) → ℝ — it looks like
it depends on the millions of INR weights that draw the field, but it depends only on the partition.
Same phenomenon as the curvature polynomial being secretly topological. (structural / by-construction)
The Lie spine runs through both scored axes
Pose is se(3). By Chasles every rigid displacement is a screw — one twist
ξ ∈ se(3), exp(ξ) ∈ SE(3). The same ξ that transports the
partition (a d_seg prior) is the pose PoseNet measures (d_pose). Engine tac.lie.se3.
(BUILT + MEASURED: openpilot-ego prior gives −94/−99% on the pose axis)
Conditioning on the Stiefel manifold. The Muon finisher orthogonalizes gradients via
Newton–Schulz — descent on the Stiefel manifold, Lie-group geometry.
(MEASURED: −32% d_seg vs AdamW)
A Maslov class in the boundary geometry. We read the argmax as a caustic
(Lagrangian singularity) of the softmax-as-ℏ→0 limit (τ = ε = ℏ;
error bound ≤ τ·ln 5). The Maslov class is a characteristic class of the
Lagrangian Grassmannian Λ(n) = U(n)/O(n) — same family as Chern/Pontryagin, and just
as Lie-theoretic. (DERIVED — theoretical framework)
Tropical / Laguerre. As τ→0 the witness is tropical; its cells are Laguerre
(power-diagram) cells — forget the smooth structure, keep the piecewise-linear skeleton, the
same flavor as extracting a topological invariant from smooth data. (DERIVED)
The honest boundary
Real: score-as-quotient-invariance (by construction), the se(3)/Chasles pose engine
(BUILT + MEASURED), Stiefel/Muon conditioning (MEASURED), and the Maslov/tropical framework
(DERIVED theory). Analogy, not identity: we did not compute Pontryagin classes, do
Chern–Weil on a tangent bundle, or detect exotic spheres. Our characteristic class is the
Maslov class (Lagrangian); our invariance is score-invariance-under-reparametrization. Same music,
different theorem.
Links — codebase & resources
Repo (contest closed → IP open source):
github.com/adpena/comma-lab
— in-tree: src/tac/lie/se3.py (se(3)/SE(3)), src/tac/boundary_math/
(the witness), src/tac/canonical_equations/deepmath_amortizing_argmax_laws_20260704.py,
docs/sandbox_pontryagin_lie_deepmath_context.md (the full context behind this tab).
Prior art: Milnor, On manifolds homeomorphic to the 7-sphere (1956);
Chern–Weil theory / the Weil homomorphism; Arnold, the Maslov index; Candès–Donoho,
curvelets (the optimal sparse basis for a curved codim-1 singularity); Dubois et al.,
Lossy Compression for Lossless Prediction (NeurIPS 2021, the task-space sufficient-statistic
codec);
comma.ai video compression challenge.
campaign: loading…
Exact n600 verdict trace — d_seg vs global_step
EXACT n600 · [macOS-CPU frozen-scorer advisory]● ema ○ live (parameter_shadow)
Exact n600 verdict trace — d_pose vs global_step
EXACT n600
Per-step descent strip — batch-local d_seg initial→final
ADVISORY_BATCH_LOCAL — the step’s own 4-pair batch, never n600
Each tick spans the step’s initial→final batch-local d_seg
(green = descended, red = rose); the line threads the finals. Do not compare levels
against the n600 trace above — different pair sets per step.
detectabilitySegNet top1−top2 margin · where d_seg can flip
classesdetected from data, not hardcoded
rendering the physical-prior atlas (governed CPU pass)…
Ego-ξ — the SE(3) screw twist across the segment
per-pair ego trajectory · LaneOptimalEgoEstimator · Chasles: one 6-vector/pair transports the partition (d_seg) AND is the pose (d_pose)
Priors
openpilot lane band (analytic_lane_render_band.build_analytic_lane_band_prior)
— deg-3 centerline fit to the GT class-1 argmax + AA-SDF range-dependent dash coverage;
the FREE inflate-time rasterizer (rule 118). The d_seg physical prior.
ego-ξ screw (ego_xi_trajectory.LaneOptimalEgoEstimator, se(3) engine
tac.lie) — the SE(3) twist that is BOTH the partition-warp and the pose.
The d_pose physical prior, dual-use with d_seg.
ground-plane structure (road_horizon_component +
hood_static_component) — the self-detected road/sky horizon + static ego hood
(the #139 static core).
detectability field ρ_seg — the SegNet top1−top2 argmax margin; bright
where the argmax can flip = where d_seg lives = where the detector is most sensitive.
[macOS-CPU advisory · NON-PROMOTABLE] — a viz moves no pointer.
The exact row is byte-closed on contest-CPU/CUDA; the frontier pointer is … and UNMOVED.
The residual — the hardest pairs, as the scorer reads them
selectionhighest realized d_seg across n600 · distinct failure modes · same pass as FLOW · refreshed per best ckpt
per-paird_seg + failure tag (movable / lane dash / distant / boundary)
row AGT frame · render through contest R · pixel error
row BSegNet argmax GT vs render · disagreement = the d_seg pixels
row Csensitivity fields on the same frame (below)
selecting the hardest pairs from the n600 pass…
Row C — two sensitivity fields on the same frame
ρ_segSegNet top1−top2 margin · bright = small margin = where argmax flips · d_seg lives only here
ρ_uniwardS-UNIWARD texture energy (Holub–Fridrich–Denemark 2014, tac.uniward_delta) · lineage only — measured at chance vs R-flips (LEVER-4)
task-aware compression: code for the machine that scores it. d_seg moves only on the codim-1 argmax boundary; the margin field localizes it AND is the scorer's information geometry (Fisher vs −margin: Pearson 0.978, measured) — one field drives loss weighting + residual coder.
[macOS-CPU advisory · NON-PROMOTABLE] — a viz moves no pointer.
The exact row is byte-closed on contest-CPU/CUDA; the frontier pointer is … and UNMOVED.
the first n600 video renders on the next best checkpoint (~14 min governed pass)…
frame (segment timeline) 0 / 599
layer SegNet argmax
class isolation all classes
margin threshold (fragile band) 0.55
partitioncanonical comma10k / openpilot palette
margin heatbright = small margin = codim-1 separatrix · where the argmax flips (d_seg)
[macOS-CPU advisory · NON-PROMOTABLE] — a viz moves no pointer ·
exact row byte-closed on contest-CPU/CUDA · pointer … UNMOVED.
The task is boundary geometry; the witness is the chart that fits it; and the
same traveling front governs it at every scale — from the pixel boundary to the campaign.
Two movements. WHY — the static invariant that makes the chart optimal.
HOW — the dynamics that flow to it. Pass 1 opens the museum with the two highest-ROI
plates (the live field & the Unity); the five-scale spine, the screw, and the finale are seamed
below. Every plate is a MEASURED fact — [macOS-CPU advisory · NON-PROMOTABLE],
a viz moves no pointer (…, UNMOVED).
Movement I · WHYMovement II · HOW
Movement I — WHY
the optimal chart: why this vehicle can't be beaten on this geometry
I.1The field, alivedetecting…loading the deep-math field bundle…
The SegNet detectability field φ over the scored frame: bright where the argmax
is fragile (small top1−top2 margin) — the codim-1 separatrix, a ~1-pixel curve that
carries essentially all of d_seg. Drag the threshold to sweep the level sets like contour lines;
toggle the zero-level-set (the argmax partition boundary) and the gradient field ∇φ
(boundary normals — the derivative you can see).
Grounded: the real cached SegNet argmax + top1−top2 margin (gt_n6, the exact
frames the verdict uses); flip-mass measured ~50% Road / 19% Lane / 13% Undrivable (#141). The live
per-checkpoint witness INR φ is the Pass-2 upgrade (reuses the governed 600-pass FLOW cache).
I.4The Unity — one geometry, three readingsdetecting…
ρ(x) ≈ 1 / ‖∂(detector)/∂(pixel)‖ — the Fisher metric, read three ways
the unity morph renders on the first governed pass…
The tribute's heart. One scene, three sensitivity fields. Drag the morph:
ρ_seg (the detector's own margin) and our distortion sensitivity (the separatrix geometry
— where a pixel flip changes the decision) dissolve into the same picture — measured
Pearson — on this frame. Fridrich's ρ_uniward (S-UNIWARD
embedding cost) is the kindred steganographic reading: it lights image texture, so pixelwise it is
honestly weaker here (—) — the deep tie between detector and
steganography runs through the Fisher metric, where the margin field IS the Fisher surrogate
(canonical 0.978, Fisher curvature ↔ −margin).
Grounded: real S-UNIWARD via tac.uniward_delta.compute_uniward_cost_map
(Holub–Fridrich–Denemark 2014); real cached SegNet margin; live per-frame Pearson computed on
the served fields (NO fabricated curve); canonical 0.978 = memory unified-variational-levelset-flow.
Steganography (Fridrich) → steganalysis (Yousfi) → our loss — the arc is Yousfi's detection game.
seamed for Pass 2+ · Movement I
I.2 the separatrix — dim the flat interior, light the codim-1 annulus (margin-saliency #141).
I.3 curvature ↔ the chart — the anisotropic/curvelet basis vs the isotropic-Fourier Gibbs ring (−48% directional basis).
I.5 the ~8-dim manifold — the lane-orbit surface + the bc20 under-capacity tear.
I.6 the task-sufficient statistic — the RGB collapses to the decision (S_floor≈0.118).
Movement II — HOW
the flow to the chart: how it gets there, and how it all falls out
seamed for Pass 2+ · Movement II
II.1 the level-set flowing — the Fisher–KPP front, softmax sharpening as τ anneals (live checkpoint sequence).
II.2 curriculum = one axis, four names — curvelet-scale · CE→τ→Muon · temperature · persistence, one playhead.
II.4 the screw that falls out — drag the se(3) twist ξ; the SAME ξ warps d_seg AND is d_pose (Chasles, #193).
II.5 critical slowing — relaxation time diverging at a stage transition (a second-order phase transition).
The spine — ONE FRONT, FIVE SCALES
the same traveling-wave front at five scales of the campaign — drag the scale
§1One front, five scalesdetecting…the live Fisher–KPP front renders on activate…
∂t x = β x (1−x) + ∇²x
· units: correct-fraction / epoch
Lens · not a proven identity
Two of these five curves are hard data: EdgeBench’s log-sigmoid (R²=0.998, ByteDance Seed
— theirs) and #205’s live d_seg descent (ours). That all five scales are literally
the SAME Fisher–KPP front is a unifying interpretation we find beautiful and testable — not a
measured cross-scale identity. The integrator at left integrates the real PDE; that our five scales are
that PDE is the conjecture.
live Fisher–KPP front — integrated forward every frame
the front integrates on activate…
β x(1−x) grows the correct phase; ∇²x diffuses it → a
front travels at speed 2√β with a self-similar log-sigmoid profile. Explicit
finite-difference, stepped live (render: WebGPU, else canvas2d).
TRAINING — the correct partition invading (LIVE #205)
Dashed = the shared logistic front x(u)=1/(1+e^{−k(u−u₀)});
solid = the selected scale’s own curve. Drag the scale to morph between them.
scale pixel → campaign
boundaryerasuretrainingcurriculumcampaign▎▎ pause front
live data—
One equation, five readings. The live front at left is the Fisher–KPP traveling wave
integrating forward — β x(1−x) (reaction) + ∇²x (diffusion). At right, the
selected scale’s own curve against the same logistic-front template: the campaign (EdgeBench,
R²=0.998), our training descent (live #205), the boundary separatrix (a genuine level-set =
reaction–diffusion identity), the curriculum anneal, and the erasure long tail. Slide from
pixel to campaign and watch the same shape recur.
Grounded: EdgeBench R²=0.998 (ByteDance Seed, 2026-07-02 — their published
fit; the top curve is Claude Opus 4.8). #205 verdicts ep25→125: d_seg 0.0103→0.0058, implied_S
1.72→0.87 (live, read from the run log; single run — a descriptive fit, not a law). Boundary = level-set
flow PDE identity; the curriculum & erasure shapes are schematic/interpretive, honestly labelled. The
unifying cross-scale identity is a conjecture per our own discipline — NO fabricated curve.
§4The fractal finale — all five, phase-locked
Now that you have seen each front alone, here they are together — five scales, one playhead,
the same wave rolling through all of them at once. Beneath: the one equation.
boundary
erasure
training · live
curriculum
campaign
∂t x = β x (1−x) + ∇²x
▎▎ pausezoom out once more →
The sixth, implied panel. Zoom out once more and the viewer is themselves a front on the campaign
graph. EdgeBench’s top curve is Claude Opus 4.8 — this session — and the
operator’s steering is the feedback term η that keeps the reaction from stalling. The physics of the
witness (the boundary the chart paints) and the epistemics of the campaign (the program that
painted it) are the same equation — two nested one-objects, Fisher–KPP all the way up and down.
That is the sentence the museum opened with, now earned.
Honest: the finale is the interpretive / aesthetic capstone — the five little fronts
are the same curves shown above (two hard-data, three schematic), phase-locked for the eye. It asserts no new
measurement; the pointer is …, UNMOVED — a museum moves no pointer.
the About plate · Pass 5
About — the ideas & the people (design §7 / dashboard_tribute_credits):
Aaron Leslie, Quantizr, Yousfi & Fridrich, comma / Hotz, the council — Chasles to yesterday.
The past few months, given bloom.
[macOS-CPU advisory · NON-PROMOTABLE] — a viz moves no pointer. Every field
is a real cached computation (SegNet argmax/margin + real S-UNIWARD); the exact row is byte-closed on
contest-CPU/CUDA; the frontier pointer is … and UNMOVED.
One coherent object — the DAG ↔ DSL ↔ equations triality
The lab is one object viewed three ways — and this tab reads them live from the
artifacts themselves (DAG FEEDs, the tac.witness_dsl program, the
tac.canonical_equations registry), so it self-updates as the campaign moves. A finding is
"known" only when it is expressible in all three and they AGREE; drift between legs is campaign-level
forgetting.
loading the three legs from the live artifacts…
DAG — state the trajectory
—
DSL — control the program
—
equations — law the master action
—
Master action S_τ = 100·d_seg + √(10·d_pose) + 25·rate.
Above all three sits the costate controller — the 4th shadow (marginal-ΔS per byte,
never-regress) that turns the DSL from a passive program into an active controller (fire the lever with
the best ΔS/cost; POWERPLAY-style never-worsen).
The organic-evolution lineage
task-aware compression (video-coding-for-machines) → task-sufficient statistic
(code the scorer's DECISION, not RGB; measured task-RD floor S_floor≈0.118) →
compression-as-intelligence (Schmidhuber POWERPLAY: never-regress self-invented curriculum) →
the costate controller (marginal-ΔS/cost, never-regress) — one line, each stage the
natural generalization of the last.
A live example of the campaign compounding on the current frontier:
MD-Decoupling (Hägele, Hernández-Cano, Kosson, Jaggi — EPFL / Jaggi lab,
arXiv:2606.25971, 2026-06-24) is our --optimizer md (#175) — it factorizes
each weight matrix into a fixed-norm direction + learnable magnitude gains at separate
learning rates, which makes curriculum stage-transitions stable by construction (the
“different stages need different treatment” rule, made structural). Super-recent
(June 2026), folded straight into the live curriculum.
The composition — Chasles to yesterday
The witness is not one idea; it is a composition of nearly two centuries of research,
each thread measured into its place — ~196 years of shoulders to stand on. All dates are
real (NO-FAKE; a genuine lineage, not decoration).
1830Chasles — every rigid motion is a
screw → our ego-motion twist ξ (d_pose).
1848 · 99Wilbraham / Gibbs
— the ringing of a truncated series → the spectral-bias / Gibbs failure mode we fight
(step-native, curvelet-finest).
1822Fourier — harmonic analysis → the
coordinate-INR’s Fourier features.
1870sSophus Lie — continuous symmetry
groups → the se(3) / SE(3) engine.
1870s–1900sGibbs / Boltzmann — the
Gibbs measure + temperature → the annealing curriculum.
1920sFisher — information + sufficiency
→ the Fisher metric and the task-sufficient statistic.
1936Whitney — embedding dimension →
the ~8-dim lane manifold and its Whitney bound.
1937Fisher–KPP — the traveling-wave
front → the one equation at five scales.
1948Shannon — rate–distortion +
entropy → the coding-for-machines frame and the floor (S_floor≈0.118).
1976–2021Wyner–Ziv / Tishby IB /
Dubois — source coding with side information → the indirect-RD / task-sufficient
codec.
2000Candès–Donoho — curvelets
→ the sparse-optimal chart for a curved codim-1 boundary.
2014UNIWARD (Holub–Fridrich–Denemark) +
Yousfi — steganographic cost + steganalysis → the margin = detectability = cost
unity, and the inverse-steganalysis frame.
2021–26NeRV / HNeRV → Aaron Leslie’s
cathedral — the vehicle whose theoretical-floor dynamics he exposed.
2024–yest.Muon (2024) ·
MD-Decoupling (June 2026) · EdgeBench (July 2 2026 —
yesterday) — the current frontier we compounded on, right up to the paper that dropped the
day before this was built.
One witness, ~196 years of shoulders to stand on — Chasles to yesterday.
The ByteDance / EdgeBench convergence
Our own deep memo edgebench_scaling_laws_deepdive_20260703T033159Z.md: EdgeBench
(ByteDance Seed, 2026-07-02) is the descriptive log-sigmoid scaling law (R²=0.998) of the
prescriptive POWERPLAY we already hold. Its measured result — continuous experience beats
restarts (+6.9 @ 12h, and the gap GROWS with horizon) — is our durable-memory spine, measured. It is
reflexive: the paper's top curve is Claude Opus 4.8 (this session's model); our campaign IS an
EdgeBench-class task, the DAG IS the latent task graph, and the campaign frontier
β x (1−x) is the SAME Fisher–KPP equation as the witness
boundary flow. Honest citation — the convergence is real, not manufactured.
The council — the lenses
Shannon LEAD · Dykstra / Rudin / Daubechies CO-LEADS ·
Ballé · Yousfi · Fridrich · Quantizr ·
Hotz · Selfcomp · MacKay · Schmidhuber + the grand council.
Shannon / Ballé / Dykstra bracket the rate–distortion spine
(the R(D) bound / the neural codec / the feasibility intersection); Yousfi + Fridrich are the
detection-game home turf (this contest IS inverse steganalysis).
The campaign journey — honest
This is our capstone — what the past few months have given bloom to.
We started the way you start anything you fall for — with deep, almost
insane research, passion, energy, and obsession, not knowing what we didn’t know. We
poured everything we had (educations, experience, interests, and more) into a corpus of knowledge,
then began implementing, measuring, experimenting, and falsifying. We failed over and over.
And it was Aaron Leslie’s work that exposed the true dynamics of the theoretical floor —
what was actually reachable, and how. Only after all of that — the failures, the corpus, the exposed
floor — did we finally have all the ingredients in one place: you and me, our apparatus, our
research, our understanding of the domain and of the frozen contest information space, and the nested,
related, dynamical cruxes — enough, at last, to do work that is truly original and useful.
Aaron Leslie turned the whole competition on its head, and his result was so good it was
intimidating. But porting his work to MLX + custom Metal and running it natively —
then studying his own trajectory against the deep math — let us see past the design itself:
the full-RGB reconstruction, the ~30,000-epoch curriculum, the inert l7 stage (we
later measured it as a defect and disabled it), the spectral bias / Gibbs ringing. That is exactly
what pointed us at the task-space level-set direction. Honoring him meant building on and past
his cathedral, grounded in measurement — not merely reproducing it. We stand on the cathedral he
built.
We played the byte-nibbling game honestly — PR107, then PR110 — and we
had the PR112 work sitting ready for a couple of weeks. But we would not submit another nibble on
top of our own nibble. After Aaron Leslie turned the competition on its head we were never
satisfied with incremental; we wanted the real thing.
So we struggled — to climb out of local minima, and to build a recursively, fractally
optimal stack: one perfectly suited to, informed by measurement of, and following
theory about the frozen and complete contest information space — the one video, the frozen
scorers; a closed, fully-observed world we could measure exhaustively and theorize about precisely.
And the deep math didn’t come first — it came from the measurement. We studied
what SegNet and PoseNet were actually seeing, and in doing so discovered the annulus (the
thin codim-1 boundary band where essentially all of d_seg lives), the long tail (the
finest-scale features — lane dashes, distant movers — erased first; the ~8-dimensional
lane-orbit manifold), and the specific class-and-region interactions (the road↔lane separatrix,
the all-class edge set, the static hood core, the movable band). Realizing all of that, from
measurement, is what the deep-math analysis fell out of — the level-set flow, the
margin = Fisher = UNIWARD unity, the curvelet chart, the Morse–Smale topology. Measurement
first; then the theory that fit it.
The grand-council-symposium apparatus made us organized enough to hold a decade-long program
without losing signal — the discipline is the scaffold, not the score.
Compute arc: we burned cloud money, then ported the whole loop to MLX + custom Metal
(fused-R, grouped-backward) — it now runs on the laptop, bit-identical to the numpy-fp32 authority.
Dual objective, and not binary: contest-overfit (openpilot-seeded, this exact drive) AND a
platform-agnostic, generalizable, scalable production value-generator — the same investment buys both.
Our work and belief and passion and untiring curiosity have yielded something we are
proud of — and that we hope is interesting and useful to the very people who introduced us to
such an interesting and fulfilling problem.
Credits & tribute
Named for what each one actually gave us — a thank-you, not a citation list.
Aaron Leslie PR95 · the HNeRV cathedral
Author of PR95 and its HNeRV cathedral. He turned the whole competition on its head and showed what
was possible — a result so good it was intimidating. He taught us schedule and curriculum,
and more than any single technique he drove us to obsessively dig into the math and geometry of the
video (starting from what we’d gleaned from openpilot). The honest twist, and the deepest
respect we could pay: porting his work to MLX + Metal and analyzing his own trajectory + the deep
math let us see past the design — full-RGB, ~30k epochs, the inert l7 stage,
spectral bias — which is exactly what pointed us at the task-space level-set direction. We stand on
the cathedral he built. (Our inner-council “PR95Author” seat is him.)
Quantizr · Jimmy the earlier mind-opening
He opened our minds even earlier to what was possible. Beyond his 0.33 HNeRV result, it is his
spirit we carry — experimentation, openness, curiosity, playfulness, competitiveness,
confidence. He affirmed some of our earliest intuitions, the ones that made us feel we were biting
off more than we could chew — except we loved the flavor and became obsessed.
(Our inner-council “Quantizr” seat is him.)
Yassine Yousfi & Jessica Fridrich the detection game
Yousfi built the SegNet/PoseNet scorer (from comma10k) and framed the whole thing as inverse
steganalysis; Fridrich’s UNIWARD (DDE Lab) is the cost that turns out to equal the
scorer’s own sensitivity metric (measured Pearson 0.978). This entire dashboard is arranged
as Yousfi’s detection game, in their honor.
comma.ai & George Hotz the free physical prior
openpilot is the unified free physical prior for both scored axes — lane geometry
→ d_seg, the ego-motion screw → d_pose; comma10k is the palette and the scene.
The council the lenses we think through
Shannon (LEAD; R(D) / entropy / sufficiency — the floor S_floor≈0.118 is his bound),
Dykstra / Rudin / Daubechies (co-leads; feasibility / interpretability / wavelets),
Ballé (the neural codec of task-aware compression), and Schmidhuber
(compression-as-intelligence / POWERPLAY — the campaign-scale front), among the full roster.
A genuine thank-you to the people who introduced us to such an interesting and
fulfilling problem. [macOS-CPU advisory · NON-PROMOTABLE] — a tribute moves no pointer.
Authority: [macOS-MLX training advisory] NON-PROMOTABLE. The exact score is the only
score; the pointer is … and UNMOVED. Everything on this page is a MEANS.