Text → 3D geometry · open-source research project

Let the LLM write
the shape as code.

Vibe-CADing turns natural-language prompts into 3D geometry. An LLM writes a program for the shape (CAD code for B-rep solids, or GLSL for implicit surfaces), the program runs, and errors go back to the model until the result is valid.

Masoud Jafaripour

GitHub repository Project report (PDF)Try the live 3D viewer →

Turntable of a dragon written by GPT-4o as a GLSL signed distance function — “A stylized dragon…”, written by GPT-4o as 1.8 KB of GLSLSDF

Mounting plate generated with CadQuery by Qwen3-8B — Plate · Qwen3-8BB-rep

Rubber duck signed distance function turntable — Duck · smooth unionsSDF

Method

Geometry as code: generate, run, check, repair

Every harness shares one loop. The model never outputs geometry directly; it outputs a program, and the program is treated as untrusted code: run in isolation, checked, and sent back with the error if anything fails.

1Prompt
Natural-language description, or an edit instruction for an existing shape.
2LLM
Any text-in/text-out model: local Qwen via transformers, or an OpenAI-compatible API.
3Program
CadQuery / build123d Python, or a GLSL function float sdf(vec3 p).
4Sandboxed run
Executed in a subprocess with a timeout: OpenCascade kernel or headless GPU.
5Validate
Tracebacks, compile logs with the offending lines, render checks.
6Artifact
STEP solid for CAD code; PNG + turntable GIF for SDFs.

↺

Repair loop (5 → 2). On failure, the model gets back its own code plus the error, and tries again.

✎

Refine loop (6 → 1). A new instruction (“make the wings larger”) edits the last valid program.

CAD-code harness

B-rep

The LLM writes CadQuery or build123d code that assigns the part to result.
Runs in a subprocess and exports a STEP file through the OpenCascade kernel.
On failure the traceback goes back into the prompt (CADCodeVerify-style), up to max_retries.
Two kernel variants in separate environments, since they pull in conflicting OCP builds.

src/CAD_Harness →

SDF harness (GLSL)

SDF

The LLM writes float sdf(vec3 p), and optionally sdf_color, using a shared primitive library: sdSphere, sdCapsule, opSmoothUnion, rotX/Y/Z…
An OpenGL 3.3 sphere tracer (moderngl + EGL, no display) renders it with soft shadows and AO.
Rejects compile errors, a missing sdf(), empty or inside-out renders and off-frame shapes, then asks for a repair (0–3 rounds).
Refine an existing shape with a text instruction; outputs .glsl, .png, .gif and run metadata.

src/CAD_Harness/sdf →

LLM → JSON → pythonOCC

B-rep

Qwen2.5-3B-Instruct turns a prompt into a structured spec such as {"type": "cube", "size": 5}.
pythonOCC builds sketch → wire → face → extruded solid, exportable to STEP / IGES.
Simple and robust for primitives, but limited to what the JSON schema can express.

src/B-rep →

Supporting models

Text → image → SVG
Stable Diffusion + ControlNet (Canny), then OpenCV contour tracing
PixelCNN 2D CAD grids
Masked convolutions model p(grid) cell by cell over CAD tokens
CLIP retrieval
ViT-B/32 ranks rendered meshes against a text query

Why several representations?

	SDF (GLSL)	Mesh	B-rep (CAD code)
Stores	a function (code)	triangles	exact faces, edges, NURBS
Size	tiny (KB of code)	large	medium
Smooth blends	trivial	hard	hard (fillets)
Exact dimensions / STEP	no	approximate	yes
Best for	organic, sculpted forms	rendering, games, printing	engineering CAD

Gallery

Outputs from the repository

Every image is a file checked into the repo. Captions say who wrote the geometry code: an LLM, or a hand-written reference.

Dragon: GLSL signed distance function — Dragon
SDF
GLSL signed distance function · GPT-4o
“A stylized dragon made of smooth geometric primitives → “Make the wings larger””
Generate and refine each passed on the first attempt. 1.8 KB of GLSL. Crude: floating legs, heart-shaped wings.
src/CAD_Harness/sdf/examples/dragon_gpt4o.glsl

Mounting plate: CadQuery → B-rep → STEP — Mounting plate
B-rep
CadQuery → B-rep → STEP · Qwen3-8B
“a 40x20x5mm mounting plate with two 4mm holes 15mm either side of center and 2mm filleted corners”
Box → face select → hole pattern → edge select → fillet. Also generated through the build123d harness.
src/CAD_Harness/cadquery/plate.step

Snowman: GLSL signed distance function — Snowman
SDF
GLSL signed distance function · Hand-written reference
Three smoothly blended spheres plus coal, carrot, sticks and hat, with per-part colors.
src/CAD_Harness/sdf/examples/snowman.glsl

Rubber duck: GLSL signed distance function — Rubber duck
SDF
GLSL signed distance function · Hand-written reference
An organic shape: ellipsoids and spheres joined with smooth unions.
src/CAD_Harness/sdf/examples/duck.glsl

Cube: CadQuery → B-rep → STEP (SVG projection) — Cube
B-rep
CadQuery → B-rep → STEP (SVG projection) · Qwen3-8B
“a cube of size 5”
Hidden-line SVG export of the generated STEP file.
src/CAD_Harness/cadquery/cube.step

Extruded square: pythonOCC sketch → wire → face → prism — Extruded square
B-rep
pythonOCC sketch → wire → face → prism · Hand-written example
The B-rep building blocks used by the LLM → JSON → pythonOCC pipeline, exported to STEP and IGES.
src/B-rep/example_B_rep.py

Interactive 3D

Drag to orbit. Both are real repository files.

Left: the GLSL shapes compiled live in your browser with a WebGL2 port of the harness's sphere tracer. Right: the CadQuery mounting plate, tessellated from its STEP file, with the solid's B-rep edges drawn on top.

SDFLive GLSL sphere tracing

Auto-rotate Soft shadows

›View the shape's GLSL: this is the whole model

float sdDragonBody(vec3 p) {
    // Main body as a series of capsules
    float body = sdCapsule(p, vec3(-0.6, -0.2, 0.0), vec3(0.6, -0.2, 0.0), 0.15);
    float neck = sdCapsule(p, vec3(-0.6, -0.2, 0.0), vec3(-0.8, 0.3, 0.0), 0.1);
    float tail = sdCapsule(p, vec3(0.6, -0.2, 0.0), vec3(0.9, -0.4, 0.0), 0.08);
    return opSmoothUnion(opSmoothUnion(body, neck, 0.1), tail, 0.1);
}

float sdDragonHead(vec3 p) {
    // Head as a rounded box
    vec3 headPos = vec3(-0.85, 0.35, 0.0);
    float head = sdRoundBox(p - headPos, vec3(0.15, 0.1, 0.1), 0.05);
    // Eyes as small spheres
    float eye1 = sdSphere(p - (headPos + vec3(0.05, 0.05, 0.1)), 0.03);
    float eye2 = sdSphere(p - (headPos + vec3(0.05, 0.05, -0.1)), 0.03);
    return opSmoothUnion(opSmoothUnion(head, eye1, 0.05), eye2, 0.05);
}

float sdDragonWings(vec3 p) {
    // Wings as larger rotated cones
    vec3 wingBase = vec3(-0.2, 0.0, 0.0);
    float wing1 = sdRoundCone(rotZ(p - wingBase, -0.5), 0.0, 0.3, 0.7);
    float wing2 = sdRoundCone(rotZ(p - wingBase, 0.5), 0.0, 0.3, 0.7);
    return opSmoothUnion(wing1, wing2, 0.1);
}

float sdDragonLegs(vec3 p) {
    // Legs as capsules
    float leg1 = sdCapsule(p, vec3(-0.3, -0.8, 0.1), vec3(-0.3, -0.5, 0.1), 0.05);
    float leg2 = sdCapsule(p, vec3(0.3, -0.8, 0.1), vec3(0.3, -0.5, 0.1), 0.05);
    return opSmoothUnion(leg1, leg2, 0.1);
}

float sdf(vec3 p) {
    // Combine all parts of the dragon
    float body = sdDragonBody(p);
    float head = sdDragonHead(p);
    float wings = sdDragonWings(p);
    float legs = sdDragonLegs(p);
    return opSmoothUnion(opSmoothUnion(opSmoothUnion(body, head, 0.1), wings, 0.1), legs, 0.1);
}

vec3 sdf_color(vec3 p) {
    // Color the dragon with a gradient from green to red
    float height = p.y + 1.0;
    return mix(vec3(0.0, 1.0, 0.0), vec3(1.0, 0.0, 0.0), height / 2.0);
}

src/CAD_Harness/sdf/examples/dragon_gpt4o.glsl · 1,851 bytes · 47 lines

// A snowman: three smoothly blended snow spheres, coal eyes and buttons,
// a carrot nose, stick arms and a top hat. Faces +Z, rests on y = -1.

float snow(vec3 p) {
    float base  = sdSphere(p - vec3(0.0, -0.55, 0.0), 0.45);
    float torso = sdSphere(p - vec3(0.0,  0.15, 0.0), 0.33);
    float head  = sdSphere(p - vec3(0.0,  0.62, 0.0), 0.23);
    return opSmoothUnion(opSmoothUnion(base, torso, 0.08), head, 0.06);
}

float coal(vec3 p) {
    vec3 q = vec3(abs(p.x), p.y, p.z);  // mirror: one eye models both
    float eyes = sdSphere(q - vec3(0.08, 0.68, 0.195), 0.03);
    float buttons = min(sdSphere(p - vec3(0.0, 0.25, 0.305), 0.035),
                    min(sdSphere(p - vec3(0.0, 0.12, 0.318), 0.035),
                        sdSphere(p - vec3(0.0, -0.01, 0.28), 0.035)));
    return min(eyes, buttons);
}

float carrot(vec3 p) {
    // the cone's axis is +Y; rotate it to point along +Z
    return sdRoundCone(rotX(p - vec3(0.0, 0.62, 0.2), -1.5708), 0.04, 0.008, 0.2);
}

float sticks(vec3 p) {
    vec3 q = vec3(abs(p.x), p.y, p.z);
    float arm = sdCapsule(q, vec3(0.28, 0.2, 0.0), vec3(0.62, 0.42, 0.0), 0.02);
    float twig = sdCapsule(q, vec3(0.52, 0.36, 0.0), vec3(0.6, 0.5, 0.02), 0.012);
    return min(arm, twig);
}

float hat(vec3 p) {
    float crown = sdCylinder(p - vec3(0.0, 0.92, 0.0), 0.14, 0.09);
    float brim = sdCylinder(p - vec3(0.0, 0.83, 0.0), 0.24, 0.012);
    return min(crown, brim);
}

float sdf(vec3 p) {
    return min(min(snow(p), coal(p)), min(min(carrot(p), sticks(p)), hat(p)));
}

vec3 sdf_color(vec3 p) {
    float d = sdf(p) + 1e-3;
    if (coal(p) <= d) return vec3(0.04);
    if (carrot(p) <= d) return vec3(0.95, 0.4, 0.08);
    if (sticks(p) <= d) return vec3(0.35, 0.2, 0.1);
    if (hat(p) <= d) return vec3(0.08, 0.08, 0.1);
    return vec3(0.95, 0.97, 1.0);
}

src/CAD_Harness/sdf/examples/snowman.glsl · 1,824 bytes · 49 lines

// A rubber duck: an organic shape from ellipsoids and spheres joined with
// smooth unions. Faces +Z, rests on y = -1.

float body(vec3 p) {
    float torso = sdEllipsoid(p - vec3(0.0, -0.6, -0.05), vec3(0.48, 0.36, 0.62));
    float tail = sdEllipsoid(rotX(p - vec3(0.0, -0.4, -0.58), 0.6), vec3(0.16, 0.08, 0.16));
    float neck = sdSphere(p - vec3(0.0, -0.25, 0.25), 0.2);
    float head = sdSphere(p - vec3(0.0, 0.05, 0.3), 0.3);
    vec3 q = vec3(abs(p.x), p.y, p.z);  // mirror: one wing models both
    float wing = sdEllipsoid(rotZ(q - vec3(0.42, -0.52, -0.1), -0.25), vec3(0.08, 0.18, 0.32));

    float d = opSmoothUnion(torso, tail, 0.12);
    d = opSmoothUnion(d, wing, 0.06);
    d = opSmoothUnion(d, neck, 0.15);
    return opSmoothUnion(d, head, 0.12);
}

float beak(vec3 p) {
    float upper = sdEllipsoid(p - vec3(0.0, 0.02, 0.6), vec3(0.14, 0.05, 0.13));
    float lower = sdEllipsoid(p - vec3(0.0, -0.03, 0.57), vec3(0.11, 0.035, 0.1));
    return opSmoothUnion(upper, lower, 0.02);
}

float eyes(vec3 p) {
    vec3 q = vec3(abs(p.x), p.y, p.z);
    return sdSphere(q - vec3(0.12, 0.13, 0.55), 0.045);
}

float sdf(vec3 p) {
    return min(min(body(p), beak(p)), eyes(p));
}

vec3 sdf_color(vec3 p) {
    float d = sdf(p) + 1e-3;
    if (eyes(p) <= d) return vec3(0.03);
    if (beak(p) <= d) return vec3(1.0, 0.45, 0.05);
    return vec3(1.0, 0.82, 0.1);
}

src/CAD_Harness/sdf/examples/duck.glsl · 1,379 bytes · 38 lines

B-repMounting plate from STEP

Loading…

Prompt to Qwen3-8B: “a 40x20x5mm mounting plate with two 4mm holes 15mm either side of center and 2mm filleted corners”. Scroll or pinch to zoom.

src/CAD_Harness/cadquery/plate.step

Results

What has been run so far

These are individual, qualitative runs recorded in the repository: 5 of 8 produced a valid artifact. This is not a benchmark, and the failures are listed on purpose.

Harness	Model	Task	Outcome	Detail
B-repCadQuery	Qwen3-8B	a cube of size 5	✓ valid	STEP exported
B-repCadQuery	Qwen3-8B	mounting plate, 2 holes, filleted corners	✓ valid	STEP exported, render above
B-repbuild123d	Qwen3-8B	same mounting plate (builder API)	✓ valid	STEP exported
SDFSDF	GPT-4o	stylized dragon	✓ valid	1st attempt; crude anatomy
SDFSDF	GPT-4o	refine: make the wings larger	✓ valid	1st attempt; wings visibly larger
SDFSDF	Qwen3-8B	stylized dragon	✗ failed	failed all 3 attempts (last: compile error, missing argument)
SDFSDF	Qwen2.5-Coder-7B	stylized dragon	✗ failed	called undefined functions, omitted sdf()
SDFSDF	Qwen3-8B	snowman of three blended spheres	✗ failed	failed all 3 attempts (last: shape off-frame, caught by the framing check)

What the validators catch

Python exceptions: CAD code that raises, or never produces result, returns its traceback.
GLSL compile errors: The compiler log is returned with the offending source lines quoted.
Missing entry point: Code that never defines float sdf(vec3 p) is rejected before compiling.
Empty or inside-out: Nothing visible from any view, or a surface that fills the whole frame.
Off-frame shapes: Shapes that are not centered or exceed the [-1, 1]³ box get cut off.
Hangs: Every run has a subprocess timeout; SDF code is capped at 64 KB.

Limitations

The checks catch broken code, not wrong shapes: a valid program can still miss the prompt (see the dragon's floating legs).
Small local models (Qwen3-8B, Qwen2.5-Coder-7B) often repeat the same mistake through every repair round on SDF tasks.
SDF output is visual geometry, not dimensioned CAD: no STEP export or exact tolerances.
No quantitative benchmark yet; a vision critic that compares renders with the prompt is the natural next step.

Test suites run without any model: 24 tests for the SDF harness (prompts, compile errors, render checks, timeouts, repair and refine loops, the OpenAI backend against a stub server), plus fake-LLM tests for both CAD-code harnesses.

Implementation status

The bigger pipeline, and how much of it exists

The long-term goal is an agentic loop (Planner → Retriever → Generator → Critic) combining code generation, learned 3D generators, retrieval over CAD libraries and manufacturability feedback. This table mirrors the repository README.

Working demo
CAD-code harness (CadQuery, build123d)
LLM writes CAD code → subprocess → STEP; tracebacks fed back for retry
Working demo
SDF harness (GLSL)
LLM writes sdf(p) → headless GPU sphere tracer → PNG/GIF; repair and refine
Working demo
LLM → JSON → B-rep
Qwen2.5-3B emits a JSON spec; pythonOCC builds the solid
Working demo
Text → image → SVG
Stable Diffusion + ControlNet (Canny), then contour tracing to SVG
Working demo
PixelCNN 2D CAD grids
Masked-convolution AR model over 2D token grids (dummy data)
Working demo
CLIP retrieval
CLIP ViT-B/32 ranks rendered meshes by text similarity
Prototype
Multi-agent loop
LangGraph Planner → Retriever → Generator → Critic (outdated APIs, not runnable)
Prototype
Streamlit demo
Voxel viewer UI; sample data not included
Planned
Voxel diffusion / CVAE
Planned 3D U-Net DDPM and conditional VAE
Planned
B-rep retrieval (RAG)
Planned topology/geometry embeddings for retrieval
Planned
Validator / retrainer, web app
Planned manufacturability checks and UI/API

Let the LLM write
the shape as code.

Geometry as code: generate, run, check, repair

CAD-code harness

SDF harness (GLSL)

LLM → JSON → pythonOCC

Supporting models

Why several representations?

Outputs from the repository

Dragon

Mounting plate

Snowman

Rubber duck

Cube

Extruded square

Drag to orbit. Both are real repository files.

What has been run so far

What the validators catch

Limitations

The bigger pipeline, and how much of it exists

Run it yourself

Cite this project

Let the LLM write the shape as code.

Geometry as code: generate, run, check, repair

CAD-code harness

SDF harness (GLSL)

LLM → JSON → pythonOCC

Supporting models

Why several representations?

Outputs from the repository

Dragon

Mounting plate

Snowman

Rubber duck

Cube

Extruded square

Drag to orbit. Both are real repository files.

What has been run so far

What the validators catch

Limitations

The bigger pipeline, and how much of it exists

Run it yourself

Cite this project

Let the LLM write
the shape as code.