Skip to main content

Making An Audio Visualizer In Unity With Blend Shapes & Audio API - Part 2

There is a Part 1, check that out before going through this one.
So now we will look at Unity's Audio API, not the entire thing just the part that allows us to do stuff like this.
Audio Frequency Spectrum Visualizer
This is a representation of the audio data based on it's frequency and the how loud (amplitude) that particular part of the frequency spectrum is.
Let's start coding the thing.
We will make class called AudioVisualizer.
Member variables:
 [Range(1.0f,4500.0f)]  
 public float multiplier;  
 public int minRange = 0;  
 public int maxRange = 64;  
 private SkinnedMeshRenderer skinnedMeshRenderer;  
 private float prevAvg = 0.0f;
Nothing special here, we will get to know what these are used for as we go on.
Start Function:
void Start ()   
{  
   skinnedMeshRenderer = GetComponent<SkinnedMeshRenderer> ();  
}  
Update Function:
void Update ()   
{      
     float[] spectrum = new float[64];  
     AudioListener.GetSpectrumData(spectrum, 0, FFTWindow.BlackmanHarris);  
     if (maxRange < minRange)  
       maxRange = minRange + 1;  
     minRange = Mathf.Clamp(minRange, 0, 63);  
     maxRange = Mathf.Clamp(maxRange, 0, 63);  
     float avg = 0;  
     for (int i = minRange; i < maxRange; i++)  
       avg += Mathf.Abs(spectrum[i]);      
     avg = avg / (float)Mathf.Abs(maxRange - minRange);  
     if (avg - prevAvg > 0.0012f)
            avg = prevAvg + 0.0012f;
     else if (avg - prevAvg < -0.0012f)
            avg = prevAvg - 0.0012f;
     skinnedMeshRenderer.SetBlendShapeWeight(0, avg*multiplier);  
     prevAvg = avg;
}  
Now time to break it down.
float[] spectrum = new float[64];  
AudioListener.GetSpectrumData(spectrum, 0, FFTWindow.BlackmanHarris);  
We allocate an array of 64 floats.
We then pass in that array as parameter to 'GetSpectrumData'.
'GetSpectrumData' requires 3 parameters -
1) The float array (has be power of 2 in size eg :32,64,128 etc).
2) Which audio channel it is. In case of stereo audio 2 channels exist, we will use the first one.
3) Type of FFT Window, We will use BlackmanHarris which is more expensive to compute when compared to Window type of Triangles or Hamming but has better quality.
So each index of the float[] is a higher frequency bracket.
With index 0 being lowest frequency bracket and 63 being highest frequency bracket.
The float value at each index represents how much amplitude that frequency had. 
 if (maxRange < minRange)  
       maxRange = minRange + 1;  
minRange = Mathf.Clamp(minRange, 0, 63);  
maxRange = Mathf.Clamp(maxRange, 0, 63);  
The above code keep parameters within the correct range.
float avg = 0;  
for (int i = minRange; i < maxRange; i++)  
     avg += Mathf.Abs(spectrum[i]);      
avg = avg / (float)Mathf.Abs(maxRange - minRange);  
if (avg - prevAvg > 0.0012f)
      avg = prevAvg + 0.0012f;
else if (avg - prevAvg < -0.0012f)
      avg = prevAvg - 0.0012f;
I am going through the specified range of frequencies and averaging them and also limiting how much change can happen each frame so that the motion is no that erratic.
skinnedMeshRenderer.SetBlendShapeWeight(0, avg*multiplier);  
prevAvg = avg;
Here we are setting the blend shape weight and assigning prevAvg as the currently calculated average.
That's it! Get source code HERE.
Get Unity package HERE.
For more Unity development tutorials go HERE
Support Bitshift Programmer by leaving a like on Bitshift Programmer Facebook Page and be updated as soon as there is a new blog post.
If you have any questions that you might have about shaders or unity development in general don't be shy and leave a message on my facebook page or down in the comments.

Assets Worth Checking Out

POPULAR POSTS

Introduction To Replacement Shaders & Shader Keywords

What is a replacement shader? A replacement shader is a shader that gets applied to every object being rendered.
Since the camera determines what objects end up being shown on screen, The functionality for setting up replacement shaders are in the camera class as well.

A good use case of a replacement shader would be in making effects like SSAO.
Here we need access to the normals and the depth information so a replacement shader that displays only the normals can be rendered ( stored in a render texture ) and then another shader that displays the depth information ( stored in a render texture ) and then the final image is rendered with the SSAO effect by taking the two render textures as input and doing a bunch of calculations.

Another use case would be to visualize the environment differently for various reasons like how they did in City Skylines.

The function that performs shader replacement is:
Camera.SetReplacementShader( Shader shader, string replacementTag ) Takes in a shader as …

Curved Surface Shader [ Unity Implementation ]

Curved Surface Shader This is the shader that we will be having at the end of this tutorial.
 The curved surface shader is capable of achieving really varied visual effects from showing space-time curve due to gravity to a generic curved world shader that is seen in endless runners like Subway Surfers.
The concepts that you learn here can open you up to a new way of looking at shaders and if you didn't think they were the coolest thing ever already, hopefully let this be the turning point.😝.

Both the examples show above use the same exact material is just that different values have been passed to the shader.
Start by creating a new unlit shader in Unity and we will work our way from there.
First we define what the properties are:
_MainTex("Texture", 2D) = "white" {} _BendAmount("Bend Amount", Vector) = (1,1,1,1) _BendOrigin("Bend Origin", Vector) = (0,0,0,0) _BendFallOff("Bend Falloff", float) = 1.0 _BendFallOffStr("Falloff s…

Noise Generation Algorithms : White Noise With Shader Example

White Noise Many of you may know what white noise is and how it can be used in code.
But there are a spectrum of  'Coloured Noises' as well these include ones like brown noise, pink, blue and violet noise all these have slightly different properties and can be very useful in creating different effects. More details on coloured noises can be found HERE.
A lot of movies as well as video games like City Skylines use white noise to add a feeling of realism and texture to the on screen content, it's also referred to as a grain filter or overlay. It is a really subtle effect and should not be over used.
Creating white noise boils down to the random function used.
While writing C++ code or C# code we have access to good pseudo-random functions from their standard libraries... but while writing code in shaders we usually have to create our own pseudo-random number generator.
One really good way of making one is by using the frac( in Unity & HLSL ) or fract( in GLSL ) functio…

Alto's Adventure Style Procedural Surface Generation Part 1

Alto's Adventure Style - Procedural Surface Generation This game appears to be a strictly 2D game but if you have played it enough you will notice that some of the art assets used look like it's 3D ( I don't know if they are tho ). If you haven't played the game you are missing out on one the most visually pleasing and calming games out there ( There is literally a mode called Zen mode in the game ).
Anyway, I am going to show you how to make a procedural 2D world ( without the trees, buildings and background ) like in Alto's Adventure.
But you may notice I have a plane which is in in the Z-axis giving a depth to the surface which is not there in Alto's Adventure but if you want to know how to do it then that will be in part 2.
To achieve the same effect of Alto's Adventure ( I'm leaving that up to you ) only minimal changes are needed to the code that I am going to explain.
We are going to be using the plane mesh in unity for creating the 2D surface as th…

Making An Audio Visualizer In Unity With Blend Shapes & Audio API - Part 1

Audio Visualizer with Unity Audio API & Blend Shapes This tutorial is going to be split into two parts, The first part (this one) we are going to be covering Blendshapes, what they are, what it's used for and how to use them in Unity and in Part 2 we will cover integrating that with Unity's audio API to get the desired result.
So now you got a feel as to what we will be making.
Blendshapes are just like transitions that the vertices go through to go from being in one position in space to another with the help of a transition value. All the major 3d modelling software from Blender to Maya has it.
As you can see that the model has 2 shape keys, one called MinValue and another called MaxValue, these names don't carry over into unity but the index of the shape key( blend shape ) matters.
We will be manipulating the MaxValue to make changes to the mesh.
This is how the model looks when it has it's MaxValue set at zero.
You may have guessed now that it is just a basic linear i…