Synthesize game audio with AudioGen and SFX Studio

verifiedAgainst e50826a · verifiedOn 2026-09-10.

What you will build

Voxamine enforces strict technical art provenance (MASTERPLAN.md §14.5 / §14.6): placeholders and technical sound cues are generated procedurally by mathematical synthesis engines rather than scraped, downloaded, or ripped from third-party libraries. This ensures that every audio asset carries zero licensing baggage, reproduces deterministically across clean clones, and remains editable by altering synthesis code.

In this workflow, you will inspect and run the procedural DSP audio generators in Tools/AudioGen, synthesize seamless periodic ambience beds, multi-variant footsteps, block-break cracks, and chemistry/lab sound cues, and audit the entire generated library inside Unity using the SFX Studio Voxel Workshop module (SfxStudioModule.cs).

# Generating the full placeholder audio library headlessly:
cd Tools/AudioGen
./generate-ambience.sh
./generate-footsteps.sh
./generate-break-sfx.sh
./generate-chemistry-sfx.sh

Where this sits

The audio generation pipeline bridges standalone command-line Python/FFmpeg DSP generators, Unity Editor Voxel Workshop tooling, and project-owned audio resource folders:

• Tools/AudioGen holds pure Python scripts (voxel_*_gen.py) and bash wrappers that synthesize audio using numpy, scipy, and ffmpeg.
• Voxel Workshop (SFX Studio) scans disk state via SfxLibraryScanner, presents expected file counts via SfxCategoryCatalog, orchestrates subprocess execution via SfxScriptRunner, and highlights missing files.
• Resources (Assets/_Game/Resources/Audio/) receives the synthesized WAV files, partitioned into Ambience/, Footsteps/, Interaction/, UI/, and Chemistry/.
• Runtime Audio Systems (AmbienceRuntime, FootstepAudioRuntime, InteractionAudioRuntime) load these assets at runtime using Resources.Load<AudioClip>.

flowchart TD
  subgraph GENERATORS["Tools/AudioGen/ (Standalone DSP)"]
    PY_AMB["voxel_ambience_gen.py\n(IFFT random-phase periodic beds)"]
    PY_STEP["voxel_footsteps_gen.py\n(Band-limited resonance pulses)"]
    PY_BRK["voxel_break_sfx_gen.py\n(High-frequency crack transients)"]
    PY_CHEM["voxel_chemistry_sfx_gen.py\n(Glassware, bubbles, effervescence)"]
    SH_WRAP["Shell wrappers\n(*.sh parameter dispatch)"]
  end

  subgraph WORKSHOP["Editor / Voxel Workshop"]
    CATALOG["SfxCategoryCatalog\n(Roster of expected clips & paths)"]
    SCANNER["SfxLibraryScanner\n(Audits present vs missing clips)"]
    RUNNER["SfxScriptRunner\n(Executes generators via Process.Start)"]
    STUDIO["SfxStudioModule\n(Editor UI, category cards, rescan)"]
  end

  subgraph STORAGE["Assets/_Game/ (Project Owned)"]
    RES_AMB["Resources/Audio/Ambience/\n(Day, Night, Cave, Biome beds)"]
    RES_STEP["Resources/Audio/Footsteps/\n(Per-surface Footstep*N.wav)"]
    RES_INT["Resources/Audio/Interaction/\n(Per-surface Break*N.wav)"]
    RES_CHEM["Resources/Audio/Chemistry/\n(GlassClink, Bubble, GasHiss, etc.)"]
  end

  PY_AMB --> SH_WRAP
  PY_STEP --> SH_WRAP
  PY_BRK --> SH_WRAP
  PY_CHEM --> SH_WRAP
  STUDIO --> RUNNER
  RUNNER --> SH_WRAP
  SH_WRAP --> RES_AMB
  SH_WRAP --> RES_STEP
  SH_WRAP --> RES_INT
  SH_WRAP --> RES_CHEM
  SCANNER --> STUDIO
  CATALOG --> SCANNER

Before you start

Review the generation contracts in Tools/AudioGen/README.md and inspect:

• SfxCategoryCatalog.cs, the catalogue of all 7 audio categories and their expected file sets.
• SfxLibraryScanner.cs, checking clip presence on disk.
• SfxScriptRunner.cs, launching bash processes from the Unity Editor.
• SfxStudioModule.cs, the Voxel Workshop UI.

Understand these procedural synthesis rules:

Rule	Implementation	Purpose
Periodic ambience loops	Inverse FFT of random-phase spectra with integer-cycle LFO gusts	Loops tile indefinitely with zero seam clicks and zero fade-to-silence windows.
Multi-variant banking	--variants 3 (e.g. FootstepStone.wav, FootstepStone2.wav, FootstepStone3.wav)	Allows NoRepeatRandom to pick non-identical consecutive one-shots.
Zero external licensing	Generated purely via mathematical functions (numpy.sin, band-pass butterworth filters, white/pink noise)	Completely avoids copyright, royalty, attribution, or licensing entanglements.
Laboratory ahead-of-time banking	Chemistry SFX (GlassClink, Bubble, GasHiss, Alarm) generated before runtime wiring	Decouples audio design from reaction runtime development.

The build, step by step

1. The Periodic Ambience Synthesis Engine

Open Tools/AudioGen/voxel_ambience_gen.py. Rather than recording field audio and crossfading edges, the ambience generator synthesizes periodic loops by construction:

def spectral_noise(duration_sec, sample_rate, low_hz, high_hz, seed):
    np.random.seed(seed)
    n_samples = int(duration_sec * sample_rate)
    freqs = np.fft.rfftfreq(n_samples, d=1.0 / sample_rate)

    # Construct band-limited frequency envelope
    magnitude = np.zeros_like(freqs)
    band = (freqs >= low_hz) & (freqs <= high_hz)
    magnitude[band] = 1.0 / np.sqrt(np.maximum(freqs[band], 1.0))

    # Assign random phase in frequency domain
    phases = np.random.uniform(0, 2 * np.pi, size=len(freqs))
    spectrum = magnitude * np.exp(1j * phases)

    # Inverse FFT guarantees identical values at start and end
    audio = np.fft.irfft(spectrum, n=n_samples)
    return audio / (np.max(np.abs(audio)) + 1e-9)

Because an inverse real FFT produces a periodic signal whose boundary conditions match exactly, looping this audio in Unity’s AudioSource produces no click or pop at the loop boundary.

2. Physical Surface Footstep and Break Synthesis

Inspect Tools/AudioGen/voxel_footsteps_gen.py and voxel_break_sfx_gen.py. Each physical surface has a characteristic acoustic resonance formula:

• Grass: Heavily muffled low-pass filtered noise ($[80, 450]\text{ Hz}$) with rapid exponential decay ($35\text{ ms}$).
• Stone: Sharp high-Q resonant bandpass ($[300, 1800]\text{ Hz}$) with high initial strike transient ($15\text{ ms}$).
• Wood: Hollow resonant cavity thump ($[120, 600]\text{ Hz}$) with slight pitch envelope drop.
• Sand / Snow: Soft broadband white noise burst with slower attack ($60\text{ ms}$) and low peak amplitude.

Running ./generate-footsteps.sh --variants 3 outputs 18 files into Assets/_Game/Resources/Audio/Footsteps/.

3. Chemistry and Lab SFX Synthesis

Inspect Tools/AudioGen/voxel_chemistry_sfx_gen.py. It builds placeholder sound cues for upcoming industry and chemistry apparatus:

• GlassClink: Multi-frequency harmonic sinusoids ($1.8\text{ kHz}, 3.4\text{ kHz}, 5.1\text{ kHz}$) with high-Q ringing decay ($0.4\text{ s}$).
• Bubble: Sine sweep upwards from $150\text{ Hz}$ to $600\text{ Hz}$ with rapid decay, modeling acoustic cavitation.
• GasHiss: High-pass filtered pink noise ($[1.2, 8.0]\text{ kHz}$) with turbulent modulation.
• GlassShatter: Denser burst of high-frequency sinusoids and white noise modeling mechanical fracture.

Running ./generate-chemistry-sfx.sh synthesizes all 22 laboratory cues into Assets/_Game/Resources/Audio/Chemistry/.

4. Cataloguing Audio Categories in C#

In SfxCategoryCatalog.cs, define the categories and expected file rosters:

public static IReadOnlyList<SfxCategory> All { get; } = new[]
{
    new SfxCategory(
        "Ambience",
        "Looping day/night/cave/biome ambience beds.",
        "Tools/AudioGen/generate-ambience.sh",
        "Assets/_Game/Resources/Audio/Ambience",
        new[] { "AmbienceDay.wav", "AmbienceNight.wav", "AmbienceCave.wav", ... }),
    new SfxCategory(
        "Footsteps",
        "Per-surface footstep one-shots (Grass/Stone/Wood/Sand/Snow/Dirt), 3 variants each.",
        "Tools/AudioGen/generate-footsteps.sh",
        "Assets/_Game/Resources/Audio/Footsteps",
        BuildVariantFileNames("Footstep", SurfaceNames, 3)),
    new SfxCategory(
        "Chemistry & Lab",
        "Bubbling, gas hiss, glassware, boiling, alarms, notebook scratch.",
        "Tools/AudioGen/generate-chemistry-sfx.sh",
        "Assets/_Game/Resources/Audio/Chemistry",
        new[] { "GlassClink.wav", "GlassClink2.wav", "Bubble.wav", ... })
};

5. Auditing and Executing via SFX Studio

SfxStudioModule renders a card for each category. Clicking Regenerate executes the tool via SfxScriptRunner.Run:

public static bool Run(string scriptRelativePath, string projectRoot, out string error)
{
    string fullPath = Path.Combine(projectRoot, scriptRelativePath);
    var startInfo = new ProcessStartInfo
    {
        FileName = "/bin/bash",
        Arguments = $"\"{fullPath}\"",
        WorkingDirectory = Path.GetDirectoryName(fullPath),
        RedirectStandardOutput = true,
        RedirectStandardError = true,
        UseShellExecute = false,
        CreateNoWindow = true
    };
    using var process = Process.Start(startInfo);
    process.WaitForExit();
    // AssetDatabase refresh after generation completes...
}

Screenshot pending

Capture sfx-studio-module.png: Voxel Workshop → SFX Studio module window, showing all categories green (“All 7 categories have their full clip set”) with Regenerate All and Rescan buttons. Editor dark theme.

Verify

1. Execute CLI Batch Generation

Run the generators directly from terminal to confirm all Python and FFmpeg dependencies function:

cd /home/soulwax/workspace/engines/unity/minecraft/Minecraft-HD/Tools/AudioGen
./generate-ambience.sh
./generate-footsteps.sh
./generate-break-sfx.sh
./generate-chemistry-sfx.sh

Confirm that WAV files exist in Assets/_Game/Resources/Audio/{Ambience,Footsteps,Interaction,Chemistry}/.

2. Inspect inside Unity SFX Studio

1. Open Tools → Voxel Sandbox → Voxel Workshop → SFX Studio.
2. Click Rescan: verify the summary banner reads: All 7 categories have their full clip set.
3. Click Reveal in Project on the “Chemistry & Lab” category: Unity’s Project window highlights Assets/_Game/Resources/Audio/Chemistry/.
4. Select GlassClink.wav and press Play in the Inspector preview: hear the crisp, high-frequency harmonic glass resonance.

Now do your own

To add a brand-new procedural sound category (e.g. Machinery & Piston SFX):

1. Author the DSP synthesis script: Create Tools/AudioGen/voxel_machinery_gen.py using numpy and scipy.signal to generate metallic strikes, valve releases, and hums.
2. Create the shell wrapper: Add generate-machinery-sfx.sh taking --variants, --peak, and --output flags.
3. Register in SfxCategoryCatalog.cs: Add a new SfxCategory entry with its display name, script path, output folder, and array of expected WAV filenames.
4. Rescan in SFX Studio: Open the module in Unity, click Regenerate, and confirm the card turns green upon completion.

Pitfalls

Symptom	Cause	Fix
Ambience loop exhibits a distinct click or pop every 14 seconds.	Audio was generated in time domain without circular boundary matching.	Use the inverse FFT spectral synthesis method in spectral_noise; do not use time-domain crossfading on periodic noise.
Regenerate fails with ModuleNotFoundError: No module named 'scipy'.	The system Python environment lacks scientific DSP libraries.	Ensure python3-numpy and python3-scipy are installed in the host development environment.
One surface sound fails to load at runtime.	File variant numbering has a gap (e.g. FootstepStone.wav and FootstepStone3.wav without 2).	Follow the strict sequential naming convention (Footstep<Surface>, ...2, ...3). SurfaceClipBank stops scanning at the first missing index.
Generated WAV files do not appear in Unity until Editor restart.	AssetDatabase.Refresh() was not invoked after the external shell process exited.	Call AssetDatabase.Refresh() in SfxScriptRunner after process.WaitForExit().
Unvetted audio samples appear in git diffs.	Someone downloaded third-party audio clips into Resources/Audio/.	Delete scraped files. All audio in AudioGen output directories must be deterministically synthesized by project-owned scripts.