Why AI video is a slot machine
Some links on this site are affiliate links. If you sign up through them we may earn a commission at no extra cost to you.
A video model fills in whatever you leave out of the prompt. A shot has five things worth being explicit about, and if you specify none of them the model chooses all five. That is what people are describing when they say generation feels like a slot machine.
The five are who is in the shot, what they do, where they are, where they sit in the frame, and how the camera moves. Our production shot specification has carried those five slots for long enough that they are the default. Filling them is the cheapest control available to you, and it costs nothing but a longer sentence.
The five slots
A shot needs all five of these written down somewhere.
Subject. Who or what is on screen. Not “a woman” but enough detail that the model is not choosing hair, age and wardrobe for you.
Action. What the subject does, in order, if there is an order. This is the slot models drop most often, and the one that decides whether your clip has a beginning and an end or just a mood.
Scene. Where this happens, including the light. An office at midday and an office at dusk are different clips.
Spatial framing. Where the subject sits inside the frame. Close, wide, centred, off to one side, how much headroom.
Camera. How the camera behaves: locked off, pushing in, tilting, circling.
The last two get merged constantly, and merging them is a mistake. Spatial framing is about the actor’s position in the picture. Camera is about what the camera does while the actor stays put. A locked-off wide shot of someone walking towards the lens is a different instruction from a slow push-in on a stationary subject, even though both could be described in one rambling sentence. Our shot spec keeps them in two separate fields for that reason, and a prompt that merges them hands the choice to the model.
If what you need is the reference version, with what to put in each slot and a list to run through before you submit, that is our prompt checklist. This article is the reasoning behind it.
Why the same prompt gives a different clip
Here is the arithmetic that makes generation feel random. Every slot you leave open is a slot the model fills. Nothing is being randomised on purpose: the model is answering questions you did not ask, and it answers them the way it answers everything else, by picking a reading that fits.
Suppose each open slot has only two plausible readings. Five open slots is 2⁵, or thirty-two versions of the shot you had in mind. Fill one slot and you are down to sixteen, fill three and you are down to four, and fill all five and there is one clip left for the model to assemble.
That is an illustration of how the count grows, not a measurement. We have not counted the true branches, and the number would not be a constant anyway, because a subject has more plausible readings than a camera move does. The structural point survives the hand-waving: open slots multiply, and a specified slot stops multiplying.
This is also why a vague prompt sometimes produces something better than a careful one. If you leave all five open, the model will occasionally land on a combination you would never have written. It will also, more often, land somewhere you cannot use. That is fine if you are generating for pleasure. It is expensive when every attempt costs money to generate and several minutes of someone’s time to review, which is the situation our team is in.
The gap underneath
None of this is a complaint about the models. It is a property of the language they were trained on, and a 2026 CVPR paper measured it directly.
The team audited eight popular video-captioning datasets published between 2016 and 2025. The same three failures recurred. Captions described what was in the frame and skipped everything else: motion, camera shake, focus changes, shot size, anything temporal. Terminology was imprecise, with captions routinely confusing a dolly-in (the camera physically moves) with a zoom-in (the focal length changes), and describing a fisheye lens as “a round building”. Other captions were subjective in a way nothing can be grounded in, such as “an atmospheric shot full of tension”.
Their response was to write a specification with more than a hundred professional video creators, among them cinematographers, directors of photography and camera operators. It defines five aspects: subject, scene, motion, spatial, and camera dynamics, broken down into hundreds of visual primitives. That is the same five slots we use. We did not arrive at them together by coincidence; that taxonomy is where ours came from, and it is the part of this article we did not discover ourselves.
They then re-captioned professional video with the specification and fine-tuned an existing generator on it. The architecture and the training objective did not change. Only the language describing the training videos changed, and the generator learned to act on camera motion, angle, lens, focus and framing instructions it had previously ignored.
The useful reading for anyone writing prompts is narrower than the paper’s. A model that seems to ignore your camera instruction may not be ignoring it at all. It may simply never have been taught, from the captions it trained on, what those particular words mean.
Closing slots with a start frame
Writing five slots into text is one way to close them. Handing over a first frame is a faster one.
An image-to-video generation takes a still as its starting point. That still already fixes the subject, the scene and the spatial framing, because they are visible in it. Three slots close by construction, and your prompt only has to carry the other two: what happens, and what the camera does.
Five open slots become two. That is the largest single reduction available, and it costs one image. We use it for exactly this reason on the cheap generation route, where a person has to look the same across two stitched segments. Re-applying a reference image between segments is the manual step that makes the inexpensive channel more expensive than it looks, and a locked first frame is the obvious candidate for removing it, though we have not run that test.
When you want variety
Filling every slot is the right default when you need a usable shot. It is the wrong move when the point of the batch is to find something you would not have thought of.
The rule we use is to leave exactly one slot open on purpose. Everything else is specified, and one dimension is left for the model to explore. If you are generating a hundred shots to find three good ones, you are betting on one slot rather than five, and the hundred shots are variations on a theme instead of a hundred unrelated clips. That is also how we get usable variety out of a shot pool rather than a pile of disconnected footage, which matters more than it sounds when the cost per finished item depends on how many you build from one batch.
A test for a prompt, in ten seconds
Cover four of the five slots and read the remaining one on its own. Can you see the picture in your head?
If the answer is no, the model cannot see it either. This catches the failure that produces the strangest outputs, and it catches it before you spend anything. The slot that most often fails this test is spatial framing, which is why malformed compositions and inexplicable subject placement are so common.
What this does not say
It does not say a longer prompt is a better prompt. A prompt with five specified slots is longer than a prompt with none, but length is not the mechanism. A long prompt that repeats itself and leaves the camera unspecified will behave like a short one.
It does not say the five slots are the only correct taxonomy. It is the one our production uses and the one the paper converged on independently. A different team might cut the same problem into four or six parts and do equally well. What matters is that the categories are explicit and that each one gets answered.
It does not claim the paper is a study of prompting. It is a study of video captioning and supervision. We are reading its finding about missing vocabulary across into prompt writing, which is an inference rather than a result.
It does not put a number on how much accuracy improves. We have not run a controlled comparison. We do not have a hit rate to quote, and we would rather say so than estimate one.
Method
- The five-slot specification is our own production convention, documented in our internal shot-script spec, and it is in daily use. It is not a published framework of ours and has not been through external review.
- The convergence claim rests on the taxonomy in Lin, Mitra et al., Building a Precise Video Language with Human-AI Oversight (CVPR 2026 Highlight). Their five aspects are subject, scene, motion, spatial and camera.
- What we read: the paper’s official abstract, and the authors’ own technical write-up on the CMU machine learning blog. We could not open the paper PDF, so the details above come from the authors’ description of their design rather than from the paper’s own text. Their captioning specification was built with more than a hundred professional video creators; they audited eight video-captioning datasets published between 2016 and 2025; they post-trained a vision-language model and fine-tuned an existing video generator on re-captioned professional footage.
- The 2⁵ illustration is arithmetic on an assumption we invented for the purpose. It is not derived from any measurement of model behaviour, and we have not tested whether the branches are real.
- Client work: no client name, brand or commercial figure appears in this article.