Skip to main content

Making An Audio Visualizer In Unity With Blend Shapes & Audio API - Part 2

There is a Part 1, check that out before going through this one.
So now we will look at Unity's Audio API, not the entire thing just the part that allows us to do stuff like this.
Audio Frequency Spectrum Visualizer
This is a representation of the audio data based on it's frequency and the how loud (amplitude) that particular part of the frequency spectrum is.
Let's start coding the thing.
We will make class called AudioVisualizer.
Member variables:
 [Range(1.0f,4500.0f)]  
 public float multiplier;  
 public int minRange = 0;  
 public int maxRange = 64;  
 private SkinnedMeshRenderer skinnedMeshRenderer;  
 private float prevAvg = 0.0f;
Nothing special here, we will get to know what these are used for as we go on.
Start Function:
void Start ()   
{  
   skinnedMeshRenderer = GetComponent<SkinnedMeshRenderer> ();  
}  
Update Function:
void Update ()   
{      
     float[] spectrum = new float[64];  
     AudioListener.GetSpectrumData(spectrum, 0, FFTWindow.BlackmanHarris);  
     if (maxRange < minRange)  
       maxRange = minRange + 1;  
     minRange = Mathf.Clamp(minRange, 0, 63);  
     maxRange = Mathf.Clamp(maxRange, 0, 63);  
     float avg = 0;  
     for (int i = minRange; i < maxRange; i++)  
       avg += Mathf.Abs(spectrum[i]);      
     avg = avg / (float)Mathf.Abs(maxRange - minRange);  
     if (avg - prevAvg > 0.0012f)
            avg = prevAvg + 0.0012f;
     else if (avg - prevAvg < -0.0012f)
            avg = prevAvg - 0.0012f;
     skinnedMeshRenderer.SetBlendShapeWeight(0, avg*multiplier);  
     prevAvg = avg;
}  
Now time to break it down.
float[] spectrum = new float[64];  
AudioListener.GetSpectrumData(spectrum, 0, FFTWindow.BlackmanHarris);  
We allocate an array of 64 floats.
We then pass in that array as parameter to 'GetSpectrumData'.
'GetSpectrumData' requires 3 parameters -
1) The float array (has be power of 2 in size eg :32,64,128 etc).
2) Which audio channel it is. In case of stereo audio 2 channels exist, we will use the first one.
3) Type of FFT Window, We will use BlackmanHarris which is more expensive to compute when compared to Window type of Triangles or Hamming but has better quality.
So each index of the float[] is a higher frequency bracket.
With index 0 being lowest frequency bracket and 63 being highest frequency bracket.
The float value at each index represents how much amplitude that frequency had. 
 if (maxRange < minRange)  
       maxRange = minRange + 1;  
minRange = Mathf.Clamp(minRange, 0, 63);  
maxRange = Mathf.Clamp(maxRange, 0, 63);  
The above code keep parameters within the correct range.
float avg = 0;  
for (int i = minRange; i < maxRange; i++)  
     avg += Mathf.Abs(spectrum[i]);      
avg = avg / (float)Mathf.Abs(maxRange - minRange);  
if (avg - prevAvg > 0.0012f)
      avg = prevAvg + 0.0012f;
else if (avg - prevAvg < -0.0012f)
      avg = prevAvg - 0.0012f;
I am going through the specified range of frequencies and averaging them and also limiting how much change can happen each frame so that the motion is no that erratic.
skinnedMeshRenderer.SetBlendShapeWeight(0, avg*multiplier);  
prevAvg = avg;
Here we are setting the blend shape weight and assigning prevAvg as the currently calculated average.
That's it! Get source code HERE.
Get Unity package HERE.
For more Unity development tutorials go HERE
Support Bitshift Programmer by leaving a like on Bitshift Programmer Facebook Page and be updated as soon as there is a new blog post.
If you have any questions that you might have about shaders or unity development in general don't be shy and leave a message on my facebook page or down in the comments.

Comments

Assets Worth Checking Out

POPULAR POSTS

Curved Surface Shader [ Unity Implementation ]

Curved Surface Shader This is the shader that we will be having at the end of this tutorial.
 The curved surface shader is capable of achieving really varied visual effects from showing space-time curve due to gravity to a generic curved world shader that is seen in endless runners like Subway Surfers.
The concepts that you learn here can open you up to a new way of looking at shaders and if you didn't think they were the coolest thing ever already, hopefully let this be the turning point.😝.

Both the examples show above use the same exact material is just that different values have been passed to the shader.
Start by creating a new unlit shader in Unity and we will work our way from there.
First we define what the properties are:
_MainTex("Texture", 2D) = "white" {} _BendAmount("Bend Amount", Vector) = (1,1,1,1) _BendOrigin("Bend Origin", Vector) = (0,0,0,0) _BendFallOff("Bend Falloff", float) = 1.0 _BendFallOffStr("Falloff s…

Pixelation Shader - Unity Shader

Pixelation Shader This is the correct way (one of many) of showing pixelation as a post-processing effect. This effect will work in any aspect ratio without any pixel size scaling issues as well as it is very minimal in terms of coding it up.

In order to get this to work 2 components have to be set up:
1) The pixelation image effect
2) The script - which will be attached to the camera

So let's get started by creating a new image effect shader.
We will take a look at our Shaderlab properties :
_MainTex("Texture", 2D) = "white" {} That's it, Everything else will be private and not shown in the editor.
Now we will see what are defined along with the _MainTex but are private.
sampler2D _MainTex; int _PixelDensity; float2 _AspectRatioMultiplier; We will pass _PixelDensity & _AspectRatioMultiplier values from the script.
As this is an image effect there is no need to play around with the vertex shader.
Let's take a look at our fragment shader:
fixed4 frag (…

Toon Liquid Shader - Unity Shader

Toon Liquid Shader This is how the shader will end up looking :
This shader is pretty neat and somewhat easy to implement as well as to understand. Since we will be adding some basic physics to the toon water as it is moved about we will have to support that in the vertex shader as well.
So let's start by looking at the properties :
Properties { _Colour ("Colour", Color) = (1,1,1,1) _FillAmount ("Fill Amount", Range(-10,10)) = 0.0 [HideInInspector] _WobbleX ("WobbleX", Range(-1,1)) = 0.0 [HideInInspector] _WobbleZ ("WobbleZ", Range(-1,1)) = 0.0 _TopColor ("Top Color", Color) = (1,1,1,1) _FoamColor ("Foam Line Color", Color) = (1,1,1,1) _Rim ("Foam Line Width", Range(0,0.1)) = 0.0 _RimColor ("Rim Color", Color) = (1,1,1,1) _RimPower ("Rim Power", Range(0,10)) = 0.0 } Just the usual stuff that we are used to. The only thing that may stand out is the [HideInInspector] tag, This works j…

Alto's Adventure Style Procedural Surface Generation Part 1

Alto's Adventure Style - Procedural Surface Generation This game appears to be a strictly 2D game but if you have played it enough you will notice that some of the art assets used look like it's 3D ( I don't know if they are tho ). If you haven't played the game you are missing out on one the most visually pleasing and calming games out there ( There is literally a mode called Zen mode in the game ).
Anyway, I am going to show you how to make a procedural 2D world ( without the trees, buildings and background ) like in Alto's Adventure.
But you may notice I have a plane which is in in the Z-axis giving a depth to the surface which is not there in Alto's Adventure but if you want to know how to do it then that will be in part 2.
To achieve the same effect of Alto's Adventure ( I'm leaving that up to you ) only minimal changes are needed to the code that I am going to explain.
We are going to be using the plane mesh in unity for creating the 2D surface as th…

Gift Wrapping Convex Hull Algorithm With Unity Implementation

Convex Hull Algorithm Convex Hull algorithms are one of those algorithms that keep popping up from time to time in seemingly unrelated fields from big data to image processing to collision detection in physics engines, It seems to be all over the place. Why should you care? Cus you can do magic with it and it seems so simple to implement when you first hear about it, but when you start thinking about it, you will realize why it's not such a straightforward thing to do.
Now that I got you interested (hopefully) and now we will see just what a convex hull is.
As you may have noticed a perimeter was made with the same points that was given and these perimeter points enclose the entire set of points.
Now we have to clear up the term 'Convex'.
Convex means no part of the object is caved inwards or that none of the internal angles made by the points exceed 180 degrees.
In this example of a concave shape internal angles go beyond 180 degrees.
What are those red lines for? Well...…