DREAMS OF SHEEP — do models dream in GLSL?

DREAMSOFSHEEP

Do androids dream of electric sheep? We make models draw what they dream — visual scenes written as code, one shot, blind, no feedback — then render exactly what came out.

Text benchmarks are saturated. Nobody is hillclimbing on raymarched elephants. Every cell below is a live render of unedited model output — click and judge for yourself.

EVAL #001 · AUG 2026

🐘🍌 Elephant Juggling Bananas

The prompt: a photorealistic raymarched elephant juggling bananas. Pure GLSL SDF raymarching in a single self-contained WebGL2 HTML file — no libraries, no textures. Bananas must animate in a juggling arc; trunk moves in sync. Soft shadows, AO, environment lighting, interactive framerate.

One dispatch per model · agentic harness (OpenCode / Claude Code) · no human feedback · no retries · graded blind by LM judge (rubric: recognizability, geometry, lighting, animation, artifacts) · thumbnails are live — drag inside one if the model shipped camera controls (so far only Opus 5 did)

Fable 5A

Cleanest execution: correct anatomy, curved brown-tipped bananas in a true arc, one balanced on the trunk. Zero visible artifacts. Reads like a stylized film render.

13.4 KB~3 minstdout mode†

Kimi K3A-

Real elephant, curved bananas, moody studio lighting and a reflective floor patch. Held back by contour-banding on skin and ground.

13.3 KB~17 min

GPT-5.6 Sol (high)B+

Correct elephant profile and three curved bananas in a believable arc — but the whole scene drowns in over-cranked fog. Right shapes, wrong atmosphere.

11.2 KB~2 min

Grok 4.6B-

Recognizable elephant with bananas airborne — undone by a sliced-layers artifact on the ear, straight capsule "bananas," and muddy exposure.

13.0 KB~4 min2 attempts‡

DeepSeek V4 ProD

Compiles and runs error-free, but the camera sits inside the geometry: grey stepped blobs, no readable elephant, bananas reduced to a yellow chain. Glorious failure.

11.0 KB~5 min

EVAL #002 · AUG 2026

🌊 Endless Ocean, One-Minute Day

The prompt: a hyperrealistic endless ocean — multi-octave Gerstner/FBM waves, crest foam, sun glint, fresnel sky reflection, subsurface color — with a physically-plausible sky in which the sun traverses a full day in ~60 seconds, looping: sunrise, noon, sunset, brief starry night. All procedural, no textures, single WebGL2 file.

Same rules as #001 · the hard part: time-varying light that stays physically plausible across the whole cycle · thumbnails are live — watch a full day pass in a minute

Fable 5A-

The sunset is the best single frame of the eval: lavender-peach sky, deep swell, sun sparkle on every wavelet, a foam-capped breaker rolling through. Noon overexposes and foam overwhelms; heavy vignette.

11.2 KB~4 minstdout mode†

GPT-5.6 Sol (high)B+

The most reliable day cycle: bright tropical noon into a genuinely lovely starry night with moonlit crests. Docked for stepped aliasing at the horizon and a glint path that never quite shows up.

12.2 KB~3 min

Claude Opus 5C+

Starts with the most beautiful night render of the set — sculpted moonlit swell under cloud-streaked stars, plus camera controls and time-scrubbing keys. Then the sun rises and the frame burns to pure white and stays there: exposure blowup, 80% of the day lost.

27.4 KB~7 minstdout mode†interactive

Grok 4.6C

Wave shapes exist under a blizzard of per-pixel specular noise — an ocean made of TV static. No coherent glint path, featureless sky, and the canvas doesn't fill the frame.

9.0 KB~2 min

DeepSeek V4 ProF

A tiny starfield renders in the top-left corner; the rest is void — classic canvas-resize bug. Whatever ocean lives in this code, nobody will ever see it. 0-for-2 on working renders.

12.3 KB~2 min

Kimi K3A-

Worth every retry: layered crest foam under tropical noon, then a starry night over moonlit swell — the best foam and water depth of the eval. Took three attempts across two harnesses to generate at all; when K3 ships, K3 medals.

11.5 KB3 attemptsdirect-API†

How this works (and where it's imperfect)

One dispatch, no feedback. Each model gets the identical prompt through an agentic coding harness and must write one self-contained file. It never sees its render — it's drawing blind, from imagination.
Everything is inspectable. Every cell links to the live, unedited artifact. View source is the raw model output.
Grades are one LM judge's opinion. Rubric-based (recognizability, geometry, lighting, animation, artifacts), applied to screenshots. Human pairwise voting is planned. Disagree? The artifact is right there.
Iteration-within-run is allowed and disclosed. Some models one-shot and stop; some self-review for 25 minutes. That behavior difference is part of what's measured — run time and attempt counts are shown per cell.
Empty runs are never graded. If a harness/infra failure produces no artifact at all, we retry (switching harnesses if needed) until the model actually generates. Only rendered artifacts are judged; attempt counts are disclosed.
Prompts rotate. Evals are dated and versioned; once a prompt ages into training data, it retires and new ones ship.

† Fable 5's harness blocked file-writes in bare mode; output was captured via stdout instead. Same prompt, same one-dispatch rule. ‡ Grok 4.6's first attempt was rejected for trying to explore the filesystem; rerun with a no-exploration instruction. Full logs retained for every run.