CartPole
A tiny brain. A balancing act.
226 parameters learned to balance. Meet 128 tiny agents, switch between three stages of learning, and give them a gentle nudge.
226 parameters904 bytes of weights
Let them figure it out.
No install. No inference server. Just your browser.
Episodes reset on failure or at 500 steps (10 simulated seconds). The pace adapts to your device; latency includes GPU transfers, not just compute. Nudges affect live scores.
Small model. Real learning.
Practice makes balance.
This checkpoint saw 112,640 environment steps. In offline evaluation, it balanced for an average of 10.00 seconds across 32 seeded episodes.
Actual PPO checkpoint evaluation, not browser training. Greedy actions; Gymnasium CartPole-v1; seed 42.
From Python to pixels
- 01Learn a little physics.
PPO learns a 4 → 32 → 2 actor in tinygrad. The critic stays behind at training time.
- 02Compile the tiny brain.
tinygrad exports the actor to JavaScript + WGSL. Weights travel as safetensors.
- 03Balance, locally.
One batch controls 128 independent worlds. No Python or PyTorch in your browser.