The question
Music generation systems are often designed around prompting and choosing outputs. Interactive Latent Space asks how a generative model can instead become a tool that is continuously operated inside the creative process.
Working system
An end-to-end research instrument
The project connects data preparation, model creation, training, projection, generation, and audio or MIDI output in one graph. Training progress is streamed to the interface, while latent points can be reduced, selected, interpolated, and reused as controls.
- Model layer
- PyTorch implementations of VAE, LSGAN, and autoregressive MIDI models, with reusable training and data pipelines.
- Service layer
- FastAPI endpoints and WebSockets for model lifecycle, training progress, generation, projection, and mapping.
- Interface layer
- A React Flow editor with Three.js views for composing and inspecting interactive machine-learning workflows.
Two working paths
01 · Small-data training
Build a personal latent space
Train compact VAE, GAN, or Transformer models on a creator's own MIDI or audio material, then visualize, interpolate, and generate within that space.
02 · Latent mapping
Connect meaning to generation
Map semantic embeddings such as CLAP to a model's latent representation so that text, examples, or selected points can become learnable controls.
Interaction loop
The interface connects generation, visualization, selection, learning, and re-generation in one node graph. A creator's selections and mappings can feed the next result instead of restarting from another isolated prompt.
This also suggests a different plug-in model: presets become positions in model space, while automation becomes a trajectory through that space rather than a list of conventional DSP parameter changes.