IDEOGRAM 4 · a prompt is a scene graph

Ideogram 4 · released 2026-06-03 · open weights

A prompt is a scene graph, not an incantation.

Older text-to-image models took a bag of keywords or a flat sentence. Ideogram 4 was trained exclusively on structured JSON captions — a typed, declarative description of a scene. The more relationships you pin down, the more grounded the supervision. Learning to prompt it is learning to author that graph.

encoder  Qwen3-VL-8B backbone  9.3B objective  weights  /

01 — the shift

Three eras of the prompt

The same wish — “a neon ramen stall at night” — has been expressed three very different ways as the evolved. Each era expanded what a prompt could say.

CLIP encoder

Keyword soup

neon ramen stall, night, rain, cinematic, (glowing sign:1.3), bokeh, 8k, masterpiece, highly detailed

Token-weighted tags. No grammar, attention bleeds between concepts; layout needs external tooling.

T5 encoder

Flat prose string

A moody cinematic photograph of a neon-lit ramen stall on a rain-slicked street at night, with a glowing red sign and steam rising from the bowls.

Reads full sentences well — but it is still one flat string. No first-class slots for layout, typography, or palette.

VLM encoder · JSON

Structured scene graph

"style_description": {…},
"compositional_deconstruction": {
 "background": "…",
 "elements": [ obj, text … ]
}

Typed fields: explicit layout, literal text to render, and — all in one prompt.

02 — the schema

Three top-level fields

A caption is one JSON object with at most three keys, in this order. is load-bearing — a runs on every prompt and warns on unknown, missing, or out-of-order keys.

high_level_description string optional · recommended

One–two sentence summary of the whole image. Cheap to include and it grounds everything else.

style_description object optional photo XOR art_style

Controls style, lighting, medium, palette. Holds exactly one of photo or art_style.

aesthetics · lighting · medium required when present

Mood, light, and medium (photograph, illustration, 3d_render, painting, graphic_design…).

color_palette ≤16 hex optional · last

Uppercase #RRGGBB only. Must be the final key in the object.

compositional_deconstruction object required

The spatial layout: a background string followed by a list of elements.

background string required · first

The scene/environment behind the elements.

elements list required

One entry per distinct object (type:"obj") and per text string (type:"text"). Each may carry an optional and palette.

03 — style_description

The photo ⊕ art_style fork

Exactly one branch — never both, never neither when the object is present. The branch you pick changes the required key order, because the verifier checks the sequence against a fixed table.

Photographic

Uses a photo string (optics) with medium:"photograph".

#key
1aesthetics
2lighting
3photo
4medium
5color_palette (opt, last)

Everything else

Uses an art_style string with a non-photo medium.

#key
1aesthetics
2lighting
3medium
4art_style
5color_palette (opt, last)

Palette conditioning

isn't decoration — it steers the rendered colors. Name a background tone plus a highlight/shadow contrast pair. Here is the reference caption's five-color palette:

#0B0B14
#1F4068
#E43F5A
#39C5BB
#F5F5F5

04 — compositional_deconstruction

The live caption inspector

This is where the scene graph becomes tangible. On the left is a real, conformant caption. On the right, its elements drawn on the 0–1000 grid. Hover a JSON element to light up its rectangle — or sweep the canvas to find it in the JSON. Note the box convention: is [y_min, x_min, y_max, x_max] — Y-first, origin top-left.


        
x → 0…1000 (left→right)y ↓ 0…1000 (top→bottom)
Hover a JSON element or a rectangle to bind text ↔ layout.

05 — footguns

Four ways to fall off-distribution

Each of these trips the or quietly samples . The bundled validator catches all four.

1 · Key order

Emit via an ordered structure, never an arbitrary dict.

{"medium":…,"aesthetics":…} {"aesthetics":…,"lighting":…,"medium":…}

2 · Hex case & shorthand

Uppercase #RRGGBB only. Regex ^#[0-9A-F]{6}$.

#fff   #e43f5a #FFFFFF   #E43F5A

3 · ensure_ascii=False

Keep in-image text as literal UTF-8 — escaping pushes off-distribution.

"text":"らーめん" "text":"らーめん"

4 · Y-first bbox

Row-major [y_min,x_min,y_max,x_max], 0–1000, top-left.

[x_min, y_min, x_max, y_max] [y_min, x_min, y_max, x_max]

06 — the landscape

Four model families, five axes

Where Ideogram 4 sits relative to the txt2img lineage.

Axis SD 1.5 / SDXL FLUX & newer Ideogram 1–3 Ideogram 4
Prompt shape Weighted keyword tags Flat prose string Plain text Typed JSON scene graph
Text encoder CLIP T5 Proprietary (Qwen3-VL-8B)
Layout control External (ControlNet, regional) Prompt-only, imprecise Prompt-only First-class per element
Text rendering Poor Improving Strong Strong, via literal text elements
Weights Open Mixed Closed Open ( / )

07 — the pipeline

From idea to image

A casual idea becomes a caption, gets checked, and is denoised into pixels. attention over text + patches, steered by .

input
plain text
→
expand
LLM
→
structured
JSON caption
→
check
→
denoise
→
output
image

The IdeogramJSON skill in this project is a self-hosted : Claude becomes the expander, emitting a verified caption with zero dependency on any hosted endpoint. → see The Skill tab.

Built as a teaching tool. Vanilla HTML/CSS/JS, no build step, no tracking. The page models the schema rules it describes.

Deliverable 2 · local Claude skill

The IdeogramJSON skill

A local, self-hosted “magic prompt.” Give it a plain-English image description and a style; it emits a valid, schema-conformant Ideogram 4 JSON caption — then validates it with a deterministic checker that mirrors the . Below is the full text of every file in the skill, exactly as installed. Use copy raw to lift any file verbatim.

ideogram-json/SKILL.md