Elektron-Users

jeffe wrote:
With audio (music), you've got the time aspect and the pitch aspect which both have to be handled independently.

actually tou got 3 components that have to be handled independetly for succesful timestretching, being timbre (or formant structure) the third

i think video could also be compared to this.

let's say 25 frames per second, european and australian tv standard frame rate.

to make that twice as slow, that means 25 frames per two seconds.

but there is now only 12.5 frmaes per second.
and that looks jerky.
even if the extra frames are made up by copying each frame and playing it twice, that doesn't achieve any benefit, and the video remains un-smooth.


there is a basic process of "frame-blending", which helps to some degree, but loses detail, as a number of minute crossfades are happening during the space of one second.

there is another basic process of "pixel-blending" which might help very occasionally but usually ends up with a bizarre liquified bendy result. possibly okay for special effects, but not for time-stretching video.

with a Sony EX-1 hd video camera, it is possible to crank the recording frame-rate to 60 frames per second. then, simply bring it in to a video editing app, set the footage to play back at 25 (or 30) frames per second, and hey presto - perfect, slow motion video.

but the audio doesn't naturally timestretch and maintain pitch nor original formant quality this way.


another way of timestretching video is with interpolation. Re:Vision Twixtor Pro is magic for this, and achieves the best software-based results i have ever seen. simply brilliant re-timing. Actually that is the only pro version of video timestretching software i have seen, so there could be many more just as capable.

Anyway, Twixtor works on a number of different levels, to achieve one result: picture-perfect retimed smooth video. It uses tracking (artificial intelligence to recognise objects), warping (the less warping the b

This is a great post/question and I've really enjoyed writing my response in a way to get it straight in my own head if anything at all.

Although audio and video footage from the same source are analogous in terms of motion when syncronised together, then slowed down, the changes that we audibily experience are much more apparent than what we visually experience, even without the additonal artifacts that are produced from timestretching.

Slowing down sound reduces the high frequencies contained within a given sound and in turn elongates the wave cycles - the wavelengths are longer. Slowing down film footage just slows down the action we are looking at and that is where the comparison ends. The equivalent to high frequency sound in film would be light, but as we know this is not compromised when slowing down film.

The only visible change we perceive in film is the speed of the footage, because that is our only end goal. With sound, we want to reduce speed, but speed is frequency, which is also intrinsically linked to pitch.

By trying to maintain the original pitch, we are are adding an extra dimension so to speak, because the frequencies (of the wave cycles, or 'snapshots' or units of Hz) have to be artificially elongated- and the only way to do this is to repeat or duplicate the snapshots/wavecycles - hence the stuttering effect we hear when timestretching audio.

This is not analogous to the jerkiness in films that are recorded at lower rates - because frames are not being added (we aren't trying to stretch film, it is already stretched due to the slower motion), but because the camera shutter couldn't capture enough light frequently enough.

There is no way around this problem - it is physics - higher sample rates can maintain a certain amount of the higher frequencies that get lost and can make sound appear smoother, but the physics cannot be changed - it's kind of like slowing film down and trying detatch part of that footage to maintain its speed independently from another part of the footage - which is impossible, but can be achieved artifically but overlaying or superimposing other footage played at a different rate.