Visual servoing
Follow the little red dot.
Just pixels. No peeking. A 61-parameter network finds a target in a tiny camera image. Move the dot and let its learned eyes guide the follower.
16 × 16 RGB → x, y
61 parameters · synthetic vision
Run the experiment
Choose a checkpoint and compute backend. Inference stays on your device.
Weights checked by SHA-256 · no server inference
02 / What the network sees
768 float32 values.
No hidden coordinates.
No follower position.
No hand-coded color filter.
Both images show the same input frame; only the large view adds explanatory overlays. Colors are displayed at screen precision.
03 / Change the scene
Try the red twin: one image cannot tell the two apart. The prediction often lands between them. Targets near the image edge and high noise are outside training.
Localization error
—
Prediction → true target
Follower distance
—
Follower → true target
Completed inferences
0
One dispatch at a time
Inference + readback
—
Not a kernel benchmark
A small lesson in learned vision
A shared RGB → 12 → 1 MLP scores every pixel. Spatial softmax turns those learned scores into a location. tinygrad trained its 61 parameters with coordinate supervision on 4,096 original synthetic images. The model sees pixels, not the renderer’s target position.
The white follower is deliberately simple: each step moves 22% toward the predicted location. Its controller is not learned. Switch to the untrained checkpoint to separate visual learning from this fixed motion rule.
Reproduce the training & inspect the source ↗Measured, not magic
- Held-out localization / 512 scenes
- 0.199 px mean
- Red-twin ambiguity / 512 scenes
- 4.207 px mean
Inspired by LeRobot’s visual encoder and spatial softmax, not a LeRobot checkpoint. No webcam, external images, real robot, calibrated camera, or hardware performance claim. The held-out tracking test uses separate moving scenes; it is not real-world validation.
LeRobot reference at the pinned revision ↗