Cinematics

A cinematic is a sequence: JSON data (*.sequence.json, schema schemas/sequence.schema.json) with tracks over a duration. It plays, pauses, scrubs and loops inside a Space3D scene, evaluates to the same bits for the same time, and is driven by cinema.* commands (every command is an MCP tool). The format and its evaluation are the crate sparklight-cinema (no scene dependency); the scene side is crates/sparklight-scene3d/src/cinema.rs; the commands are crates/sparklight-control/src/commands/cinema.rs. The 3D scene itself: docs/agent/space3d.md.

Quick start

spark validate samples/space3d/assets/cinema/flythrough.sequence.json          # coded problems with JSON paths
spark schema sequence                                                          # the JSON Schema
spark test samples/space3d/flythrough.test.json                                # the scenario (goldens need a GPU)
spark run samples/space3d/flythrough.space3d.json --assets samples/space3d/assets   # play it in a window (muted with --gw-mute)

{"command": "space3d.load", "params": {"path": "samples/space3d/flythrough.space3d.json", "assets": "samples/space3d/assets"}}
{"command": "space3d.step", "params": {"frames": 90}}
{"command": "cinema.status", "params": {}}
{"command": "cinema.seek", "params": {"time": 5.5}}
{"command": "cinema.evaluate", "params": {"time": 6.0}}
{"command": "cinema.capture", "params": {"time": 5.5, "width": 640, "height": 360, "path": "blend.png"}}

The format

{
  "version": 1, "name": "flythrough", "duration": 8, "loop": false,
  "tracks": [
    {"type": "transform", "target": "FlyCam",
     "position": [{"t": 0, "value": [-9, 2, 9], "interp": "smooth"}, {"t": 2, "value": [-5, 1.2, 5], "interp": "smooth"}],
     "rotation": [{"t": 0, "value": [-8, -45, 0]}, {"t": 2, "value": [-3, -50, 0], "ease": [0.42, 0, 0.58, 1]}]},
    {"type": "camera", "shots": [{"start": 0, "camera": "FlyCam"}, {"start": 5, "camera": "WideCam", "blend": 1.0}],
     "fov": [{"t": 0, "value": 60}], "focus_distance": [{"t": 0, "value": 9}], "aperture": [{"t": 0, "value": 2.0}],
     "shake": {"amplitude": [{"t": 5, "value": 0}, {"t": 6, "value": 0.05}, {"t": 7, "value": 0}], "frequency": 9, "seed": 7}},
    {"type": "cues", "cues": [{"t": 3, "kind": "script", "name": "beat", "target": "Director", "data": {"beat": 2}}]},
    {"type": "audio", "clips": [{"start": 0, "sound": "ambience", "duration": 8, "fade_in": 1, "fade_out": 1}]},
    {"type": "time", "scale": [{"t": 5, "value": 1}, {"t": 5.8, "value": 0.35}, {"t": 7.4, "value": 1}]}
  ]
}

Time is in seconds from the start. Unknown fields are refused. A sequence has at most one camera and one time track.

Keys and curves

Every animated channel is a list of keys {t, value, interp?, in_tangent?, out_tangent?, ease?} in strictly increasing time order inside 0..duration. Before the first key the channel holds the first value, after the last key the last one. interp says how the curve leaves that key toward the next one:

interp	Segment
linear (default)	straight line
step	holds this key’s value until the next key (a jump)
hermite	cubic Hermite with out_tangent here and in_tangent on the next key, in units per second (a missing tangent is 0; a missing in_tangent copies the next key’s out_tangent)
bezier	cubic bezier whose control points are value + out_tangent here and next.value + in_tangent there (offsets in value space, uniform in time)
smooth	Catmull-Rom: tangents come from the neighbouring keys, nothing to author (a fly-through path)

ease is a CSS cubic-bezier(x1, y1, x2, y2) timing curve (x1, x2 in 0..1) applied to the progress of the segment that leaves the key ([0.42, 0, 0.58, 1] is ease-in-out). Rotation channels are Euler degrees [pitch, yaw, roll] (applied yaw, pitch, roll: the order of self.euler in scripts) interpolated per component, so a key may spin past half a turn; the evaluation also reports the quaternion.

Tracks

Track	Fields	What it does
transform	target, position, rotation, scale	writes the local transform of the object target (a unique name or #id); a channel without keys leaves that part alone
camera	shots[{start, camera, blend?, blend_ease?}], fov, focus_distance, aperture, shake	the output camera (below)
cues	cues[{t, kind, name, target?, data?}]	instants: script, sound, vfx, marker
audio	clips[{start, sound, offset?, duration?, volume?, fade_in?, fade_out?}]	sounds on the timeline
time	scale keys (0 to 16)	the speed of the world while the sequence plays

Camera track. A shot shows the pose (world position and rotation, after the transform tracks) of the camera object camera from start until the next shot; the first shot also covers the time before it. blend seconds after a shot starts the view is mixed from the previous shot’s camera to this one (position lerp, rotation slerp, FOV lerp; default easing smoothstep, or blend_ease); 0 is a hard cut. fov (degrees) overrides the cameras’ own FOV, otherwise the object’s Camera component decides. Keys on focus_distance (metres) switch depth of field on (aperture is the f-stop, default 2.8). shake adds smooth deterministic noise to position (metres, scaled by the amplitude envelope) and rotation (rotation_gain degrees per metre), a pure function of time and seed.

Cues fire once each time the playhead passes them going forward (see Playback). script calls on_cue(self, name, data) in the Lua script of target (data is the cue’s JSON as a Lua value; a script without on_cue is fine, the event says delivered: false); sound queues the sound for the player (Space3D::take_sounds, the window plays <assets>/sounds/<name>.*); vfx and marker are reported only (the VFX layer and tools read the event). Audio clips report start and stop edges and, when scrubbed into, the clip that is sounding with its offset into the sound and its gain (fades applied); without duration a clip plays to the sound’s end.

Errors

Every problem is reported at once with a code and a JSON path (/tracks/2/position/3/t): spark validate, cinema.validate, cinema.load.

Code	Meaning
E_CINEMA_FORMAT	not JSON, an unknown field or the wrong type (message has line and column), or a bad name
E_CINEMA_VERSION	version is not 1
E_CINEMA_DURATION	duration outside (0, 3600]
E_CINEMA_KEY_ORDER	keys not in strictly increasing time order
E_CINEMA_KEY_VALUE	a time outside 0..duration, or a value or tangent that is not finite (also FOV, focus, aperture, shake ranges)
E_CINEMA_EASE	ease is not [x1, y1, x2, y2] with x1, x2 in 0..1
E_CINEMA_TRACK	empty target, a second camera or time track, too many tracks
E_CINEMA_SHOT	shots out of order, an empty camera name, a bad blend
E_CINEMA_TIME_SCALE	a time dilation value outside 0..16
E_CINEMA_CUE	an empty name, or a script cue without a target
E_CINEMA_AUDIO	an empty sound name, a negative offset, volume or fade, a zero duration
E_CINEMA_LIMIT	too many keys (10,000 per channel), cues, clips or tracks (256), or text too long
E_CINEMA_TARGET	at run time: a track or shot names an object that is not in the scene, or the output camera has no Camera component (one cinema_error event, the rest still plays)
E_CINEMA_TIME	a time argument that is not finite

Evaluation

sparklight_cinema::evaluate(&sequence, t, &poses) (and cinema.evaluate) returns {time, transforms, camera, missing_cameras, audio, time_scale} for a time (clamped to the sequence). It is a pure function of the sequence, the time and the camera objects’ poses, so scrubbing to the same time twice, in any order, gives the same bits (tested). The pieces are public too: evaluate_transforms, evaluate_camera, evaluate_audio, time_scale_at, fired_in. Standalone (cinema.evaluate {path|json}) camera poses come from the sequence’s own transform tracks.

Playback

Player (crate sparklight-cinema) is the playhead: time, playing, looping, speed (0 to 16), finished. play starts or resumes (after the end it restarts from 0), pause keeps the time, stop goes to 0, seek(t) jumps (clamped; nothing fires for a scrub). advance(dt) moves the head by dt * speed and returns what passed: cues and audio edges with a time in (from, to] (the first advance after a start, a seek to 0 or a wrap also includes from itself, so a cue at 0 fires once). At the end a non-looping sequence stops and holds its last values; a looping one wraps and fires the cues of every lap (at most 64 laps per advance).

In a scene

A sequence lives on an object as the Sequence component (space3d.components lists it; the scene file saves it):

{"name": "Flythrough", "components": {"Sequence": {"asset": "cinema/flythrough.sequence.json", "autoplay": true, "loop": false, "speed": 1, "camera": "CinemaCamera"}}}

asset is a file below the assets folder, read at the first step (so space3d.load {assets} decides where; a missing or invalid file is one cinema_error event), or give the whole sequence inline as sequence. Exactly one of the two. Setting the component validates it first.

How the evaluation reaches the scene, once per Space3D::update / Space3D::step call, before the physics:

1. Transform tracks write the local Transform3D of their targets; bodies follow at the same step (drive kinematic or fixed bodies, or accept a teleport of a dynamic one).
2. The camera track is evaluated from the camera objects’ world poses and written to the output camera: the object named by camera (default CinemaCamera; a missing one is created with the active camera’s settings, an existing one needs a Camera component). It becomes the active camera and gets the evaluated FOV. Do not use a shot camera as the output camera unless the sequence keys its transform. Depth of field is kept in Space3D::cinema.dof; space3d.capture and space3d.golden.* switch the renderer’s dof on with it.
3. Cues become cinema_cue events {sequence, t, kind, name, target, data, delivered}; audio edges become cinema_audio events {action: start|stop, sound, offset, gain}; a scrub emits cinema_seek {time, audio[]} so the player can stop and restart the right clips. Problems are cinema_error events (coded). Scripts: on_cue(self, name, data).
4. Time dilation. update(dt) advances the sequence by dt (real time) and the simulation by dt * scale, where scale is the product of the time tracks of the sequences that are playing: 0.35 is slow motion, 0 freezes the world while the sequence goes on. A paused sequence no longer dilates time. step() is one exact fixed step of the sequence and the world, whatever the dilation (tests and scenarios use it).

Playhead state is in space3d.snapshot.*, in snapshot files (snapshot.save / snapshot.load: save at frame 100, load elsewhere, run to 300: the uninterrupted hash, tested) and in space3d.hash. In a window (spark run) the loop plays cinema_audio clips and sound cues from <assets>/sounds (muted under --gw-mute) and pulls depth of field.

Commands

Command	Class	Does
cinema.validate	read	check a file or text; lists every problem; schema: true also returns the JSON Schema
cinema.load	write	put a sequence on an object (path, json or asset; entity, autoplay, loop, speed, camera); an object named after the sequence is made when entity names none
cinema.play	write	start or resume (from, speed, loop)
cinema.pause / cinema.stop	write	pause at the current time / back to 0
cinema.seek	write	scrub: the scene takes the state of that time at once, nothing fires; returns the evaluation
cinema.status	read	playhead of one or every sequence
cinema.evaluate	read	pure evaluation at a time (entity, or path / json)
cinema.capture	write	scrub and render the frame to a PNG (needs a GPU; the scene stays at that time)

entity defaults to the only sequence of the scene. Scenario files (*.scenario3d.json) may use cinema.* steps by their full name ({"do": "cinema.seek", "time": 4}); state-changing cinema commands are in the 3D command log and replay to the same hash.

Sample

samples/space3d/flythrough.space3d.json is the ragdoll-wall scene with two cameras (WideCam, FlyCam), a Director script and a Flythrough object whose Sequence points at samples/space3d/assets/cinema/flythrough.sequence.json: a Catmull-Rom camera path with eased rotation and a falling FOV, depth of field pulled from 9 m to 5.5 m, a blended cut to the wide camera at 5 s, a shake and slow motion around the ball’s impact, a script cue, a sound cue, a VFX cue and an ambience clip. scenarios/flythrough.scenario3d.json plays it with step, launches the ball, pauses, scrubs, resumes, checks the cues, the events, a snapshot and determinism, and compares three frames with goldens/3d/cinema_flythrough_{fly,blend,shake}.png (recorded on Apple Silicon / Metal, inspected before committing; see goldens/3d/README.md). Without a GPU the golden steps are the only ones that cannot run.

Limits

• Rotation tracks interpolate Euler angles, not quaternions (no shortest-arc surprises, but also no gimbal-free orientation curves).
• The output camera is the only camera the sequence controls; depth of field is the renderer’s dof pass (captures, goldens and the window loop apply it; a sequence with focus keys switches it on in every tier).
• Audio clips start at the clip’s beginning when scrubbed into (the audio engine has no seek); offset is reported.
• Transform tracks write local transforms of objects that may also be moved by scripts or physics; the sequence wins at every step it is playing.