3D audio

Sparklight’s sound has two halves:

The device path (AudioEngine): decodes clips and plays them through the operating system’s output. It is silent under the studio mute rule (GAMEWAY_MUTE=1, --gw-mute, page parameter gw-mute=1): a muted run opens no device and behaves like a real engine otherwise.
The stage (sparklight_audio::Stage): a deterministic software mixer. It does the 3D work (positions, doppler, buses, ducking, layers, music) and renders to a buffer. It opens no device and starts no thread, so an agent verifies sound by rendering it offline and measuring the buffer. The device path plays the very same stage (AudioEngine::attach_stage), so what is measured is what a player would hear.

Everything below renders to memory; audio.render_offline {path} writes a WAV for tools, it never plays it.

Quick start (no scene, no files, no device):

spark call audio.clip.define '{"name": "horn", "synth": {"waveform": "sine", "frequency": 440, "seconds": 1}}'
spark call audio.listener.set '{"position": [0, 0, 0]}'
spark call audio.play '{"sound": "horn", "looping": true, "position": [-200, 0, 0], "velocity": [30, 0, 0], "model": "none"}'
spark call audio.render_offline '{"seconds": 1}' --json    # peak, rms, hash, dominant_hz, envelope

(spark call runs one command per process; use spark serve or spark mcp to keep the state between commands. Every command is an MCP tool named audio_<verb>.)

The stage

use sparklight_audio::{Stage, PlayParams, Emitter, Group, synth, analysis};
let mut stage = Stage::new();                                   // 48 kHz, 64 real voices
let id = stage.play(&synth::sine(440.0, 1.0, 48_000, 0.5), PlayParams {
    looping: true,
    spatial: Some(Emitter { position: glam::Vec3::new(-200.0, 0.0, 0.0),
                            velocity: glam::Vec3::new(30.0, 0.0, 0.0), ..Emitter::default() }),
    ..PlayParams::default() })?;
let out = stage.render_seconds(1.0);                            // interleaved stereo f32
let hz = analysis::zero_crossing_frequency(&analysis::channel(&out, 2, 0), 48_000);

Signal path of a voice: layers (clips read with linear interpolation at a smoothed pitch; loops without a seam) -> occlusion low-pass (two poles, 20 kHz open to 400 Hz blocked) -> distance and cone gain -> equal-power pan with an interaural delay (up to 0.66 ms, the far ear) -> the voice’s bus. Buses add into the master. Parameters change at control blocks of 160 frames (3.3 ms) and are ramped inside a block, so nothing steps.

Determinism. No clock, no thread, no hash-ordered iteration; random numbers come from a seeded generator (StageConfig::seed). The same calls give the same samples, bit for bit (analysis::hash_samples), and splitting render into chunks of any size does not change them.
Placement. Right-handed; the listener looks along forward (a camera looks along -Z) with up +Y. Listener { position, velocity, forward, up }; Emitter { position, velocity, forward, attenuation, cone, doppler }. Flat voices (no spatial) use a balance pan (pan -1..1, centre = unity) and are for UI and stingers; a placed voice at the reference distance straight ahead is 3 dB below its clip (equal-power pan, 0.707 per ear).
Distance models (Attenuation { model, ref_distance, max_distance, rolloff }, distance clamped to the two): inverse ref/(ref + rolloff*(d-ref)), linear 1 - rolloff*(d-ref)/(max-ref), exponential (d/ref)^-rolloff, none. Beyond max_distance the volume stops falling (linear reaches 0).
Cone (Cone { inner_angle, outer_angle, outer_gain }, full opening angles in radians): full volume inside the inner cone, outer_gain outside the outer one, linear in between, measured between the source’s forward and the direction to the listener.
Doppler. OpenAL’s formula on the radial velocities of source and listener, pitch = (c - f*v_listener)/(c - f*v_source), c = 343 m/s, f = Emitter::doppler * StageConfig::doppler_scale. It scales the playback rate of the voice; speeds are limited to just below c/f.
Smoothing. set_volume and set_pitch move a voice toward the target with a time constant (PlayParams::smoothing, default 20 ms): a continuously driven value (engine pitch from RPM every frame) does not click. stop(id, fade) fades out (use at least 5 ms) before removing the voice.
Loops. A clip that holds a whole number of cycles loops with no seam (synth clips do). For a recorded clip that does not, loop_crossfade (seconds) overlaps the end with the start with a linear crossfade; test with analysis::max_sample_delta (the click detector: the largest step between neighbouring samples).
Voice priority, limit and virtualisation. At most max_voices (default 64) voices are rendered per block, chosen by priority (0..255), then audibility (gain times bus gain; a voice that is real now keeps its place against a rival less than 15% louder, so similar voices do not swap every block), then age. The others are virtual: their play position keeps moving (so a looping engine comes back in sync) and nothing is rendered; a voice too quiet to hear (below -80 dB) is virtual too. A voice leaving the real set fades out over a block instead of cutting. max_total_voices (default 512) bounds all voices; a new voice then steals the quietest voice of equal or lower priority, or is refused (E_AUDIO_TOO_MANY_VOICES).
Occlusion hook. set_occlusion(id, 0..1) low-passes and quietens a voice (smoothed). In a scene, AudioSource3D.occlusion = true casts a ray from the listener to the source and counts the physics bodies in between (three quarters of a unit each, so two fully block it; the source’s, the camera’s and their ancestors’ own bodies do not count).

Buses

master, music, sfx, ui, engine, ambience (Group, also called Bus). Each has a volume, a mute and fades; the master scales all of them (voices played straight on master too).

stage.set_bus_volume(Group::Sfx, 0.5);          // ramped over one block
stage.fade_bus(Group::Music, 0.0, 2.0);         // linear fade over 2 s
stage.set_bus_muted(Group::Ui, true);

Ducking. A DuckRule { name, trigger, target, gain_db, threshold, attack_s, hold_s, release_s } lowers the target bus while the trigger bus is sounding above threshold (the peak of the trigger bus after its volume), or while trigger_duck(name, hold) holds it (a crash). The duck reaches gain_db after attack_s, stays hold_s after the trigger stops, and returns over release_s, all linear. Music under the announcer: {"name": "music_under_announcer", "trigger": "ui", "target": "music", "gain_db": -12}.

Blend sounds (layers driven by parameters)

A BlendSound is data: named parameters and looping layers whose volume and pitch follow piecewise curves of those parameters. It is the reusable form of “engine layers crossfaded and pitched by RPM”, and it fits anything driven by a number (wind by speed, tyre squeal by slip).

{
  "name": "car_engine",
  "bus": "engine",
  "smoothing_ms": 30,
  "params": [{"name": "rpm", "min": 800, "max": 8000, "default": 800},
             {"name": "load", "min": 0, "max": 1}],
  "layers": [
    {"clip": "engine_low",
     "gain":  [{"param": "rpm", "points": [[800, 1], [3000, 0]], "shape": "equal_power"}],
     "pitch": [{"param": "rpm", "points": [[800, 0.8], [3000, 1.4]]}]},
    {"clip": "engine_high",
     "gain":  [{"param": "rpm", "points": [[2000, 0], [5000, 1]], "shape": "equal_power"},
               {"param": "load", "points": [[0, 0.4], [1, 1]]}]}
  ]
}

A layer’s volume is volume * blend.volume times the product of its gain curves; its pitch is the product of its pitch curves. Outside the first and last point a curve holds its end value.
shape: "linear" keeps the summed amplitude of two crossfading layers constant (layers in phase); "equal_power" keeps the summed power constant (layers that are not in phase; each is 0.707 at the middle).
Layers follow their curves through the blend’s smoothing_ms, so setting rpm to a new value every frame does not click.
Parameters are clamped to min..=max. Stage::set_param(voice, "rpm", 3200.0); a scene drives them (see below).
Layer clips are looped (looping: true by default; loop_crossfade_ms for clips that do not repeat seamlessly). A blend is one voice: it counts once toward the voice limit and has one position and one bus.
A blend is found by name: registered (audio.blend.define, Stage::register_blend), listed in <assets>/audio.json, or read from <assets>/sounds/<name>.blend.json.

Minimal example: an engine by RPM

Three files and one object. Mono looping recordings of the engine at idle, mid and high revs go in <assets>/sounds/ (engine_idle.wav, engine_mid.wav, engine_high.wav); until they exist, synth:engine:<hz> names stand in for them.

<assets>/sounds/car_engine.blend.json:

{
  "name": "car_engine", "bus": "engine", "smoothing_ms": 40,
  "params": [{"name": "rpm", "min": 800, "max": 8000, "default": 800},
             {"name": "load", "min": 0, "max": 1}],
  "layers": [
    {"clip": "engine_idle",
     "gain":  [{"param": "rpm", "points": [[800, 1], [2500, 0]], "shape": "equal_power"}],
     "pitch": [{"param": "rpm", "points": [[800, 0.9], [2500, 1.3]]}]},
    {"clip": "engine_mid",
     "gain":  [{"param": "rpm", "points": [[1500, 0], [3000, 1], [5000, 1], [6500, 0]], "shape": "equal_power"},
               {"param": "load", "points": [[0, 0.5], [1, 1]]}],
     "pitch": [{"param": "rpm", "points": [[1500, 0.8], [6500, 1.5]]}]},
    {"clip": "engine_high",
     "gain":  [{"param": "rpm", "points": [[5000, 0], [7000, 1]], "shape": "equal_power"},
               {"param": "load", "points": [[0, 0.4], [1, 1]]}],
     "pitch": [{"param": "rpm", "points": [[5000, 0.85], [8000, 1.2]]}]}
  ]
}

The car object (the camera needs nothing: it is the listener):

{"name": "Car", "components": {
  "AudioSource3D": {"blend": "car_engine", "looping": true, "random_start": true,
                    "min_distance": 4, "max_distance": 150,
                    "drive": [{"param": "rpm", "from": "Features/Vehicle/rpm"},
                              {"param": "load", "from": "@speed", "scale": 0.02}]}}}

Features/Vehicle/rpm is the engine RPM of the object’s Vehicle feature (space3d.get on the car’s Features shows what else is there: speed, gear, per-wheel slip_ratio); load here is a speed-based stand-in. Check it without listening:

{"command": "audio.render_offline", "params": {"seconds": 8, "scene": true, "tones": [200, 400], "window": 0.25}}

For a given RPM the level of each layer’s frequency (tone_envelopes) follows the curves above; max_delta stays small (no clicks) when the parameter moves every frame.

One-shot pools

A PoolDef { name, clips, bus, volume: [min, max], pitch_semitones: [min, max], no_repeat, max_instances, priority } picks a clip at random (never the one just played, when it can), a random volume and pitch in the ranges, and fades out the oldest instance beyond max_instances. stage.play_pool("tyre_hit", PlayParams { spatial: Some(..), ..Default::default() }); audio.play {pool, position}. The generator is seeded, so a run repeats.

Music

stage.music_play("menu", &MusicSource::Stream(StreamedSound::open("assets/music/menu.ogg")?), true, 1.0, 0.0)?;
stage.music_play("race", &race_source, true, 1.0, 3.0)?;   // 3 s equal-power crossfade from the menu track
stage.music_stop(2.0);

A track is a MusicSource::Stream (decoded while it plays) or Memory. A new track fades in over seconds while everything playing fades out over the same time (sin/cos gains: constant power); tracks are resampled to the stage rate. The music channel plays on the music bus (set_music_bus changes it). Loops are the file’s own: author music loops to repeat seamlessly.

In a Space3D scene

Three components (space3d.component_schema {name} gives the schema):

Component	Meaning
`AudioSource3D`	`sound` (clip) or `blend`, `bus`, `volume`, `pitch`, `looping`, `play`, `random_start`, `spatial`, `model` / `min_distance` / `max_distance` / `rolloff`, `cone`, `doppler`, `priority`, `occlusion`, `smoothing_ms`, `loop_crossfade_ms`, `params` (fixed blend parameters), `drive` (parameters read from the scene)
`ImpactSound`	a sound where this object hits something: `pool` or `sound`, `bus`, `min_speed`, `full_speed`, `volume`, `cooldown`, `min_distance` / `max_distance`, `priority`, and `duck` / `duck_hold` / `duck_min_speed` to fire a ducking rule on a hard hit
`AudioListener`	`enabled`. The first enabled one is the ears; without one, the active camera listens (a chase camera parented to a car just works)

play = true plays; setting it to false stops. A looping source plays until then; a one-shot plays once per false -> true change. Changing sound, blend, bus, looping, spatial, random_start or loop_crossfade_ms restarts the voice; every other field is applied live.

ImpactSound plays when a contact of the object begins and the two objects meet faster than min_speed (relative physics velocity; a fixed body counts as still). The volume rises from 15% at min_speed to full at full_speed; cooldown stops a rolling contact from machine-gunning. A hit of at least duck_min_speed fires the named ducking rule, so the music dips under a crash.

drive binds a blend parameter to a number read from the scene every update: {"param": "rpm", "from": "Features/vehicle/rpm", "scale": 1, "offset": 0}. from is @speed (m/s), @speed_kmh, @forward_speed, or Component/field/sub into any component of the object as JSON (list it with space3d.get). A path that does not resolve leaves the parameter alone.

AudioWorld::update(&space, dt) (crate sparklight-scene3d, module audio) is the system: call it after each physics step or once per frame. It reads the scene and never changes it, so audio never changes the state hash and is not part of snapshots.

The listener’s and each source’s pose is the world transform. Velocity (for doppler) is the physics velocity of a dynamic body, else how far the transform moved since the last update (a jump over 400 m/s counts as a teleport and gives 0).
Sounds are looked up by name: a registered clip, a file <assets>/sounds/<name> (also with .wav / .ogg), or a name like synth:<waveform>:<hz>[:<seconds>[:<amplitude>]] (waveforms sine, square, saw, triangle, noise, engine) that is generated on the spot. A sound that cannot be found is reported once in the update’s errors and in audio.status notes; it is not retried until audio.clip.define / audio.load changes something.
<assets>/audio.json (an AudioSetup) is applied once per assets folder: buses (volumes), ducks, clips (name to file), blends, pools.

{"buses": {"music": 0.7},
 "ducks": [{"name": "music_under_announcer", "trigger": "ui", "target": "music", "gain_db": -12}],
 "clips": {"horn": "horn.wav"},
 "blends": [ ... ], "pools": [ ... ]}

Commands

spark commands --prefix audio. lists them with schemas (schemas/commands/audio.*).

Area	Commands
State	`audio.status` (buses, voices, ducks, music, listener, registered names, notes), `audio.voices`, `audio.reset`
Mixer	`audio.bus.set {bus, volume, muted, fade}`, `audio.duck.set`, `audio.duck.trigger {name, hold}`
Sounds	`audio.clip.define {name, synth \| path}`, `audio.blend.define {blend}`, `audio.pool.define {pool}`, `audio.load {path}`
Playing	`audio.play {sound \| blend \| pool, position, velocity, ...}`, `audio.voice.set {voice, volume, pitch, position, velocity, occlusion, params}`, `audio.stop {voice, fade}`, `audio.listener.set`, `audio.music.play`, `audio.music.stop`
Measuring	`audio.render_offline {seconds, scene, window, tones, path}`, `audio.analyze {start, end, channel, tone}`

audio.render_offline renders and returns peak, rms (left, right), hash (equal hashes are bit-identical audio), max_delta (click detector), dominant_hz, the RMS envelope per window, and tone_envelopes (the level of each frequency in tones per window: how to see a crossfade or a duck without listening). With "scene": true it steps the 3D scene in lockstep: one fixed step, then the audio is brought in step with the scene, then one step of sound is rendered. A car driving past the listener is heard with its real velocity, and the steps are logged like space3d.step (so snapshot.save still replays). audio.analyze asks more questions of the last buffer without rendering again.

{"command": "space3d.load", "params": {"path": "track.space3d.json"}}
{"command": "audio.blend.define", "params": {"blend": { ... car_engine ... }}}
{"command": "audio.render_offline", "params": {"seconds": 6, "scene": true, "tones": [200, 400], "path": ".gwdev/out/pass.wav"}}
{"command": "audio.analyze", "params": {"start": 2.0, "end": 3.0, "channel": "left"}}

How to prove a sound behaves

The tests in crates/sparklight-audio/tests/stage_offline.rs and crates/sparklight-scene3d/tests/audio.rs are the templates. Measure, do not listen:

Behaviour	Measurement
Doppler	`zero_crossing_frequency` / `dominant_frequency` of a window while approaching and receding: `f * c / (c -+ v)`
Attenuation	RMS ratio of the same tone at two distances against the model’s formula
Ducking envelope	`tone_envelope` of the music’s frequency over 20 ms windows: falls over the attack, holds at `gain_db`, returns over the release
Layer crossfade	`tone_amplitude` of each layer’s frequency at chosen parameter values; with `equal_power`, the squares sum to the full amplitude
Loop clicks	`max_sample_delta` over many loop passes is no larger than the fastest step of a clean sine of that amplitude and frequency
Occlusion	`spectral_centroid` and RMS fall
Determinism	`hash_samples` of two renders is equal; chunking `render` differently does not change it
No sound on a muted run	`AudioEngine::attach_stage` returns `false` and `has_stage_player()` stays `false`

Performance

cargo run --release -p sparklight-audio --example audio_bench [voices] [seconds] renders N spatial voices offline and prints the time. Measured on the development Mac (release build, 64 placed voices with doppler, 5 s of stereo audio) while it was shared with other builds (load average about 130, so the numbers are noisy): 135 to 295 ms over repeated runs, 17 to 37 times real time (best 1.4 microseconds per voice per 160-frame block). The stage is single-threaded; a game thread controls it through Arc<Mutex<Stage>> and the audio thread locks it once per 480 frames.

Limits

The mixer is mono-in-3D: a stereo clip on a placed voice is summed to mono before panning. No HRTF, reverb or room simulation; occlusion is a low-pass and a volume loss.
Linear-interpolation resampling: large upward pitch shifts of bright clips alias.
Audio is derived state: it is not in space3d.snapshot.* or snapshot.save, and snapshot.load does not restore voices.
Music loops are the file’s; there is no beat-synchronised transition.
The device path has not been run on hardware in this task (all runs are muted); StageSource is tested by pulling samples and comparing them with an offline render.