From our sponsor: Leverage AI for dynamic, custom website builds with ease.
We’ve covered in the past how we can read data from Audio, using the p5.sound library and how we can use that data, to draw things in the canvas, using p5.js.
Well, what if instead of drawing a sketch, we used audio to distort an image? Today we want to show you some demos that play around that idea.
We’ve created some experiments using the theme of movie trailers where the background image of the movie poster is being distorted using a sound sample. It kind of adds some drama to an otherwise static image in this case.
Here’s a short video of the beginning of one of the effects:
How it works
We analyze the sound and map the range of frequencies, to some uniforms we pass in our fragment shader. Then depending on the effect/distortion we have, we can tweak different parameters, using the audio frequencies which constantly change overtime.
In our first demo, we create a simple sinewave in our fragment shader, by using the bass frequencies of the audio track to control its frequency and the mid frequencies to control its amplitude. Then we add the distortion in both axes (x & y) of our uv and add that distortion to the initial texture coordinates.
It looks like this:
float wave = sin(uv.y * u_bass + u_time) * u_mid;
vec2 d = vec2(wave); // could be vec2(wave, 0.0) or vec2(0.0, wave) for distortion only in 1 axis.
vec4 image = texture2D(u_texture, uv + d);
gl_FragColor = image;
The possibilities are endless if you want to play around that idea, it’s just a matter of what effect you’re after. Make sure you’re mapping values to your uniforms, that are within a range that can distort your visual, and you can always use some generic uniforms like u_time, that can put some ‘overdrive’ to your distortion.
Head over to the demos and check out the variations we’ve made.
Hope you’ll have fun with this one and be sure to share any of your own versions!
Fantastic!
Fantastic… but why it works only from your server? Not on the pc or from my webserver.
Do you have any error messages? It’s very difficult to troubleshoot something when you only say it doesn’t work…
I hangs on the initial point. No Play button. Need to install something?
FANTASTIC!!!!!!!!!!!!!!!!
I downloaded the zip file, clicking on index but it just flashes the loading, why doesn’t it work?
I think you need to run it on a server environment. Cheers.
wow yes TU!
Love this! How would I go about using the “Two Pass Blur” shader instead of this sinewave distort? So, the audio triggers a blurring of the image that goes away as the sound dissipates… is that even possible?