This research extract establishes how temporal mechanisms participate as generative conditions in the constitution of sound–image relations as an operable and examinable problem domain. By doing so, it shifts the analytical focus from interpretations of audiovisual outcomes—such as synchronisation points, mapping strategies, or perceptual effects—towards the constitutive processes at the generative level. Within digital audiovisual systems, sound–image relations do not pre-exist and then become connected; rather, they are organised into forms of relational possibility through system operation, under conditions shaped by temporal structure, control semantics, and execution logic.
This extract can be understood as a methodological slice. Through the construction of a runnable apparatus, two types of temporal mechanisms are distinguished, contrasted, and recorded: one centred on linear playback organised by a global timeline, and the other centred on event-based scheduling organised through triggering and dispatch. Within this framework, the research does not aim to further interpret existing audiovisual results, but instead addresses a constitutive question at the generative level: when time is no longer understood as a single linear timeline but as a structure of triggerable and schedulable events, how are sound–image relations reorganised in terms of rhythm, interruption, repetition, and transition? What tendencies and constraints of digital materiality are revealed through such structural transformations?
Blender is positioned here as a research site rather than a research object. Its animation system provides a representative example of a digital temporal structure in which dynamic organisation is strongly dependent on linear timeline-based playback. Although Blender can be connected to external audio or control systems via OSC, such connections are typically confined to continuous parameter modulation (parameter driving). As a result, event-driven temporal organisation is not directly available at the system level. This limitation forms a clear research entry point: how linear temporal assumptions are stabilised through tool encapsulation, and how these assumptions subsequently shape the organisation of audiovisual events and the interaction semantics that can be realised. This entry point aligns with the research position that software structures are not neutral interfaces, but instead preconfigure and constrain particular forms of generative behaviour.
To transform temporal mechanisms from background conditions into operable and observable research variables, a trigger system based on NLA (Non-Linear Animation) was constructed: AVT (Audiovisual Trigger System). This system extracts animation from continuous playback on the global timeline and reconfigures it as segment-based execution that can be called, scheduled, and recomposed through external events. The key methodological operation of AVT lies in reorganising playback time into scheduled time, thereby establishing, within a single creative environment, a comparative structure between two types of generative conditions: temporal continuity driven by continuous parameter modulation, and temporal discreteness driven by triggering and scheduling. Through this minimal intervention, mechanisms such as trigger granularity, execution order, concurrency coordination, and switching logic emerge as observable system behaviours. These behaviours can be recorded, reproduced, and compared, providing a traceable evidential basis for examining how temporal structures shape the generation of sound–image relations.
AVT System Description: From OSC Control to NLA Event Recording
AVT, the Audio-Visual Trigger System, is designed as an OSC-to-NLA (Non-Linear Animation) control layer for event-scheduled visual animation in Blender. Rather than treating audiovisual interaction only as continuous parameter modulation, AVT enables externally triggered visual actions to be called, recorded and reorganised as editable temporal objects within Blender’s Nonlinear Animation system. Its purpose is therefore not simply to add real-time control to Blender, but to examine how visual animation can adopt temporal operations more commonly associated with sound performance and digital audio workstations, such as triggering, gating, recording, replaying and editing.
The system design can be understood through two related levels.
- Control semantics: from parameter modulation to event triggering
In many audiovisual workflows, the connection between visual software and external audio or control systems is primarily organised through continuous parameter mapping. Incoming values may modify position, scale, rotation, colour or other visual properties frame by frame. While this form of modulation is useful for producing continuous co-variation between sound and image, it does not by itself define visual actions as discrete events with onsets, durations or release states.
AVT introduces an event-trigger layer into Blender’s animation workflow. External OSC messages can call Blender Actions as one-shot triggers, activate gate-based states, switch cameras or control the reuse and transformation of visual instances. In this way, visual behaviour can be organised not only as a changing parameter, but as an event unit that begins, occupies time, repeats, overlaps or ends. This allows audiovisual relation to be studied at the level of temporal grammar: what counts as an event, how it starts, how long it lasts, whether it can be repeated, and whether it remains editable after activation. - Temporal structure: from linear playback to NLA-based event recording
Blender animations are usually authored and played through a timeline-based workflow, where visual sequences are arranged in advance along a global playhead. AVT repurposes the NLA as a DAW-like recording and editing layer for visual events. When an incoming OSC message triggers an assigned Action, the occurrence can be written into the NLA as a strip. The triggered animation therefore does not disappear as a transient runtime response; it becomes a replayable and editable temporal object inside Blender’s own animation infrastructure.
This NLA-based inscription is central to AVT’s research value. It allows triggered visual actions to be compared with sound-oriented structures such as MIDI notes, clips, gates and recorded events. A one-shot visual trigger can function like a punctual onset; a gate state can organise visual duration through press and release; and a recorded NLA strip can be replayed, edited, cleared or recomposed after the triggering moment. AVT therefore creates a shared operational vocabulary through which sound and visual animation can be compared without assuming that they are materially or perceptually identical.
On this basis, AVT currently supports four main categories of operation.
Event Triggering
AVT supports both one-shot triggers and gate triggers. One-shot triggering calls a short visual Action and records it as an NLA strip, allowing discrete animation events to occur at arbitrary moments rather than only through pre-arranged timeline playback. Gate triggering organises visual behaviour through onset, sustained duration and release, making duration part of the audiovisual relation rather than a fixed property of a pre-authored timeline.
Instance Pooling and Transform Control
AVT includes an instance pooling mechanism that allows repeated visual events to reuse available objects rather than manually duplicating them. Triggered instances can be placed, scaled or rotated through OSC-controlled transform values. This supports the study of event density, repetition, spatial distribution and resource thresholds as observable aspects of system behaviour.
Camera Switching
AVT enables OSC-based camera switching, allowing changes of viewpoint to be integrated into the event structure. Unlike marker-based camera changes that are fixed to the timeline, AVT camera switching can occur in response to external triggers. Viewpoint organisation therefore becomes part of the same temporal scheduling logic as actions, gates and instances.
Timeline Management
Because AVT writes triggered events into the NLA, timeline management becomes part of the system design. AVT-generated strips can be replayed, edited, cleared or recomposed during rehearsal and testing. This allows the system to combine live triggering with post-trigger editing, positioning Blender’s NLA as both a visual event recorder and an analytical surface for comparing different temporal grammars.
Implementation Notes: Supporting Reproducibility and Traceability
• OSC-driven one-shot triggering of Blender Actions into generated NLA strips
• Gate-based triggering for sustained animation states with press/release control
• Instance pooling for repeated animated assets and reusable visual events
• OSC-based transform control for location, rotation and scale
• Camera switching through OSC messages or the AVT user interface
• Recording of triggered visual events as editable NLA strips
• Automatic clearing of AVT-generated NLA strips during timeline loop testing
• Configurable OSC prefix for adapting the system to different control setups
• Debug logging for checking incoming OSC messages and triggered Blender operations
AVT v0.2.0 has been released.
Source code, documentation, examples, and version notes are available in a public GitHub repository.
https://github.com/xinyuesun98-max/AVT-audiovisual-trigger-system-.git
Conversation: where critical friends of itinerant space are invited to comment on journal submissions as part of an ongoing research dialogue.