Skip to main content

Making An Audio Visualizer In Unity With Blend Shapes & Audio API - Part 2

There is a Part 1, check that out before going through this one.
So now we will look at Unity's Audio API, not the entire thing just the part that allows us to do stuff like this.
Audio Frequency Spectrum Visualizer
This is a representation of the audio data based on it's frequency and the how loud (amplitude) that particular part of the frequency spectrum is.
Let's start coding the thing.
We will make class called AudioVisualizer.
Member variables:
 [Range(1.0f,4500.0f)]  
 public float multiplier;  
 public int minRange = 0;  
 public int maxRange = 64;  
 private SkinnedMeshRenderer skinnedMeshRenderer;  
 private float prevAvg = 0.0f;
Nothing special here, we will get to know what these are used for as we go on.
Start Function:
void Start ()   
{  
   skinnedMeshRenderer = GetComponent<SkinnedMeshRenderer> ();  
}  
Update Function:
void Update ()   
{      
     float[] spectrum = new float[64];  
     AudioListener.GetSpectrumData(spectrum, 0, FFTWindow.BlackmanHarris);  
     if (maxRange < minRange)  
       maxRange = minRange + 1;  
     minRange = Mathf.Clamp(minRange, 0, 63);  
     maxRange = Mathf.Clamp(maxRange, 0, 63);  
     float avg = 0;  
     for (int i = minRange; i < maxRange; i++)  
       avg += Mathf.Abs(spectrum[i]);      
     avg = avg / (float)Mathf.Abs(maxRange - minRange);  
     if (avg - prevAvg > 0.0012f)
            avg = prevAvg + 0.0012f;
     else if (avg - prevAvg < -0.0012f)
            avg = prevAvg - 0.0012f;
     skinnedMeshRenderer.SetBlendShapeWeight(0, avg*multiplier);  
     prevAvg = avg;
}  
Now time to break it down.
float[] spectrum = new float[64];  
AudioListener.GetSpectrumData(spectrum, 0, FFTWindow.BlackmanHarris);  
We allocate an array of 64 floats.
We then pass in that array as parameter to 'GetSpectrumData'.
'GetSpectrumData' requires 3 parameters -
1) The float array (has be power of 2 in size eg :32,64,128 etc).
2) Which audio channel it is. In case of stereo audio 2 channels exist, we will use the first one.
3) Type of FFT Window, We will use BlackmanHarris which is more expensive to compute when compared to Window type of Triangles or Hamming but has better quality.
So each index of the float[] is a higher frequency bracket.
With index 0 being lowest frequency bracket and 63 being highest frequency bracket.
The float value at each index represents how much amplitude that frequency had. 
 if (maxRange < minRange)  
       maxRange = minRange + 1;  
minRange = Mathf.Clamp(minRange, 0, 63);  
maxRange = Mathf.Clamp(maxRange, 0, 63);  
The above code keep parameters within the correct range.
float avg = 0;  
for (int i = minRange; i < maxRange; i++)  
     avg += Mathf.Abs(spectrum[i]);      
avg = avg / (float)Mathf.Abs(maxRange - minRange);  
if (avg - prevAvg > 0.0012f)
      avg = prevAvg + 0.0012f;
else if (avg - prevAvg < -0.0012f)
      avg = prevAvg - 0.0012f;
I am going through the specified range of frequencies and averaging them and also limiting how much change can happen each frame so that the motion is no that erratic.
skinnedMeshRenderer.SetBlendShapeWeight(0, avg*multiplier);  
prevAvg = avg;
Here we are setting the blend shape weight and assigning prevAvg as the currently calculated average.
That's it! Get source code HERE.
Get Unity package HERE.
For more Unity development tutorials go HERE
Support Bitshift Programmer by leaving a like on Bitshift Programmer Facebook Page and be updated as soon as there is a new blog post.
If you have any questions that you might have about shaders or unity development in general don't be shy and leave a message on my facebook page or down in the comments.

Comments

Assets Worth Checking Out

POPULAR POSTS

Curved Surface Shader [ Unity Implementation ]

Curved Surface Shader This is the shader that we will be having at the end of this tutorial.
 The curved surface shader is capable of achieving really varied visual effects from showing space-time curve due to gravity to a generic curved world shader that is seen in endless runners like Subway Surfers.
The concepts that you learn here can open you up to a new way of looking at shaders and if you didn't think they were the coolest thing ever already, hopefully let this be the turning point.😝.

Both the examples show above use the same exact material is just that different values have been passed to the shader.
Start by creating a new unlit shader in Unity and we will work our way from there.
First we define what the properties are:
_MainTex("Texture", 2D) = "white" {} _BendAmount("Bend Amount", Vector) = (1,1,1,1) _BendOrigin("Bend Origin", Vector) = (0,0,0,0) _BendFallOff("Bend Falloff", float) = 1.0 _BendFallOffStr("Falloff s…

Introduction To Replacement Shaders & Shader Keywords

What is a replacement shader? A replacement shader is a shader that gets applied to every object being rendered.
Since the camera determines what objects end up being shown on screen, The functionality for setting up replacement shaders are in the camera class as well.

A good use case of a replacement shader would be in making effects like SSAO.
Here we need access to the normals and the depth information so a replacement shader that displays only the normals can be rendered ( stored in a render texture ) and then another shader that displays the depth information ( stored in a render texture ) and then the final image is rendered with the SSAO effect by taking the two render textures as input and doing a bunch of calculations.

Another use case would be to visualize the environment differently for various reasons like how they did in City Skylines.

The function that performs shader replacement is:
Camera.SetReplacementShader( Shader shader, string replacementTag ) Takes in a shader as …

Toon Liquid Shader - Unity Shader

Toon Liquid Shader This is how the shader will end up looking :
This shader is pretty neat and somewhat easy to implement as well as to understand. Since we will be adding some basic physics to the toon water as it is moved about we will have to support that in the vertex shader as well.
So let's start by looking at the properties :
Properties { _Colour ("Colour", Color) = (1,1,1,1) _FillAmount ("Fill Amount", Range(-10,10)) = 0.0 [HideInInspector] _WobbleX ("WobbleX", Range(-1,1)) = 0.0 [HideInInspector] _WobbleZ ("WobbleZ", Range(-1,1)) = 0.0 _TopColor ("Top Color", Color) = (1,1,1,1) _FoamColor ("Foam Line Color", Color) = (1,1,1,1) _Rim ("Foam Line Width", Range(0,0.1)) = 0.0 _RimColor ("Rim Color", Color) = (1,1,1,1) _RimPower ("Rim Power", Range(0,10)) = 0.0 } Just the usual stuff that we are used to. The only thing that may stand out is the [HideInInspector] tag, This works j…

How To Animate A Fish Swimming With Shaders

Animate Fish Swimming With Shaders We are going to make swimming animation by using only shader code.
By the time we are done, it's going to look like this.
You will probably need the fish model used in this tutorial, that can be found HERE. Can use your own model but the shader code might have to be modified accordingly because of the orientation of the model that you might be using ( issues with whether the X axis & Z axis is flipped ).
The shader used way out performs a similar scene with skeletal animations applied on the fish models.
On a previous benchmark I did comparing the shader animation with the skeletal animation there was a difference of 28 FPS( on average ) with 50 fish.
The shader we are going to make is really powerful and flexible and don't think that it's limited to making fishes swim😀.


So this mesh oriented like this when imported into unity and this is important to understand because this means that the model's vertices have to be moved along the X-…

Access Reflection Probe Data For Custom Shaders

The Unity shader documentation regarding reflection probes is pretty minimal and not at all comprehensive.
This short tutorial is intended to bring reflection probe functionalities to the forefront your future shader writing endevors which is a fancy way of saying "Look at this cool stuff and go and use it somewhere" 😏
Here we will try just the bare minimum of making a shader that reflects the cubemap data from reflection probe and displays it on the object.

These reflection probes are basically objects that store a complete image of the environment surrounding it into a cubemap which then can be read by shaders to create various effects.
More information on how reflection probes work in Unity can be found here :
Using Reflection Probes In Unity

I am not going over how to set up Reflection Probes here only how to access them inside our custom shaders.
So this is what we will be making:
The reflection probe takes in the cubemap only if it is within it's range otherwise i…