Skip to main content

Making An Audio Visualizer In Unity With Blend Shapes & Audio API - Part 2

There is a Part 1, check that out before going through this one.
So now we will look at Unity's Audio API, not the entire thing just the part that allows us to do stuff like this.
Audio Frequency Spectrum Visualizer
This is a representation of the audio data based on it's frequency and the how loud (amplitude) that particular part of the frequency spectrum is.
Let's start coding the thing.
We will make class called AudioVisualizer.
Member variables:
 public float multiplier;  
 public int minRange = 0;  
 public int maxRange = 64;  
 private SkinnedMeshRenderer skinnedMeshRenderer;  
 private float prevAvg = 0.0f;
Nothing special here, we will get to know what these are used for as we go on.
Start Function:
void Start ()   
   skinnedMeshRenderer = GetComponent<SkinnedMeshRenderer> ();  
Update Function:
void Update ()   
     float[] spectrum = new float[64];  
     AudioListener.GetSpectrumData(spectrum, 0, FFTWindow.BlackmanHarris);  
     if (maxRange < minRange)  
       maxRange = minRange + 1;  
     minRange = Mathf.Clamp(minRange, 0, 63);  
     maxRange = Mathf.Clamp(maxRange, 0, 63);  
     float avg = 0;  
     for (int i = minRange; i < maxRange; i++)  
       avg += Mathf.Abs(spectrum[i]);      
     avg = avg / (float)Mathf.Abs(maxRange - minRange);  
     if (avg - prevAvg > 0.0012f)
            avg = prevAvg + 0.0012f;
     else if (avg - prevAvg < -0.0012f)
            avg = prevAvg - 0.0012f;
     skinnedMeshRenderer.SetBlendShapeWeight(0, avg*multiplier);  
     prevAvg = avg;
Now time to break it down.
float[] spectrum = new float[64];  
AudioListener.GetSpectrumData(spectrum, 0, FFTWindow.BlackmanHarris);  
We allocate an array of 64 floats.
We then pass in that array as parameter to 'GetSpectrumData'.
'GetSpectrumData' requires 3 parameters -
1) The float array (has be power of 2 in size eg :32,64,128 etc).
2) Which audio channel it is. In case of stereo audio 2 channels exist, we will use the first one.
3) Type of FFT Window, We will use BlackmanHarris which is more expensive to compute when compared to Window type of Triangles or Hamming but has better quality.
So each index of the float[] is a higher frequency bracket.
With index 0 being lowest frequency bracket and 63 being highest frequency bracket.
The float value at each index represents how much amplitude that frequency had. 
 if (maxRange < minRange)  
       maxRange = minRange + 1;  
minRange = Mathf.Clamp(minRange, 0, 63);  
maxRange = Mathf.Clamp(maxRange, 0, 63);  
The above code keep parameters within the correct range.
float avg = 0;  
for (int i = minRange; i < maxRange; i++)  
     avg += Mathf.Abs(spectrum[i]);      
avg = avg / (float)Mathf.Abs(maxRange - minRange);  
if (avg - prevAvg > 0.0012f)
      avg = prevAvg + 0.0012f;
else if (avg - prevAvg < -0.0012f)
      avg = prevAvg - 0.0012f;
I am going through the specified range of frequencies and averaging them and also limiting how much change can happen each frame so that the motion is no that erratic.
skinnedMeshRenderer.SetBlendShapeWeight(0, avg*multiplier);  
prevAvg = avg;
Here we are setting the blend shape weight and assigning prevAvg as the currently calculated average.
That's it! Get source code HERE.
Get Unity package HERE.
For more Unity development tutorials go HERE
Support Bitshift Programmer by leaving a like on Bitshift Programmer Facebook Page and be updated as soon as there is a new blog post.
If you have any questions that you might have about shaders or unity development in general don't be shy and leave a message on my facebook page or down in the comments.


Assets Worth Checking Out


Curved Surface Shader [ Unity Implementation ]

Curved Surface Shader This is the shader that we will be having at the end of this tutorial.
 The curved surface shader is capable of achieving really varied visual effects from showing space-time curve due to gravity to a generic curved world shader that is seen in endless runners like Subway Surfers.
The concepts that you learn here can open you up to a new way of looking at shaders and if you didn't think they were the coolest thing ever already, hopefully let this be the turning point.😝.

Both the examples show above use the same exact material is just that different values have been passed to the shader.
Start by creating a new unlit shader in Unity and we will work our way from there.
First we define what the properties are:
_MainTex("Texture", 2D) = "white" {} _BendAmount("Bend Amount", Vector) = (1,1,1,1) _BendOrigin("Bend Origin", Vector) = (0,0,0,0) _BendFallOff("Bend Falloff", float) = 1.0 _BendFallOffStr("Falloff s…

How To Animate A Fish Swimming With Shaders

Animate Fish Swimming With Shaders We are going to make swimming animation by using only shader code.
By the time we are done, it's going to look like this.
You will probably need the fish model used in this tutorial, that can be found HERE. Can use your own model but the shader code might have to be modified accordingly because of the orientation of the model that you might be using ( issues with whether the X axis & Z axis is flipped ).
The shader used way out performs a similar scene with skeletal animations applied on the fish models.
On a previous benchmark I did comparing the shader animation with the skeletal animation there was a difference of 28 FPS( on average ) with 50 fish.
The shader we are going to make is really powerful and flexible and don't think that it's limited to making fishes swim😀.

So this mesh oriented like this when imported into unity and this is important to understand because this means that the model's vertices have to be moved along the X-…

Toon Liquid Shader - Unity Shader

Toon Liquid Shader This is how the shader will end up looking :
This shader is pretty neat and somewhat easy to implement as well as to understand. Since we will be adding some basic physics to the toon water as it is moved about we will have to support that in the vertex shader as well.
So let's start by looking at the properties :
Properties { _Colour ("Colour", Color) = (1,1,1,1) _FillAmount ("Fill Amount", Range(-10,10)) = 0.0 [HideInInspector] _WobbleX ("WobbleX", Range(-1,1)) = 0.0 [HideInInspector] _WobbleZ ("WobbleZ", Range(-1,1)) = 0.0 _TopColor ("Top Color", Color) = (1,1,1,1) _FoamColor ("Foam Line Color", Color) = (1,1,1,1) _Rim ("Foam Line Width", Range(0,0.1)) = 0.0 _RimColor ("Rim Color", Color) = (1,1,1,1) _RimPower ("Rim Power", Range(0,10)) = 0.0 } Just the usual stuff that we are used to. The only thing that may stand out is the [HideInInspector] tag, This works j…

Pixelation Shader - Unity Shader

Pixelation Shader This is the correct way (one of many) of showing pixelation as a post-processing effect. This effect will work in any aspect ratio without any pixel size scaling issues as well as it is very minimal in terms of coding it up.

In order to get this to work 2 components have to be set up:
1) The pixelation image effect
2) The script - which will be attached to the camera

So let's get started by creating a new image effect shader.
We will take a look at our Shaderlab properties :
_MainTex("Texture", 2D) = "white" {} That's it, Everything else will be private and not shown in the editor.
Now we will see what are defined along with the _MainTex but are private.
sampler2D _MainTex; int _PixelDensity; float2 _AspectRatioMultiplier; We will pass _PixelDensity & _AspectRatioMultiplier values from the script.
As this is an image effect there is no need to play around with the vertex shader.
Let's take a look at our fragment shader:
fixed4 frag (…

Access Reflection Probe Data For Custom Shaders

The Unity shader documentation regarding reflection probes is pretty minimal and not at all comprehensive.
This short tutorial is intended to bring reflection probe functionalities to the forefront your future shader writing endevors which is a fancy way of saying "Look at this cool stuff and go and use it somewhere" 😏
Here we will try just the bare minimum of making a shader that reflects the cubemap data from reflection probe and displays it on the object.

These reflection probes are basically objects that store a complete image of the environment surrounding it into a cubemap which then can be read by shaders to create various effects.
More information on how reflection probes work in Unity can be found here :
Using Reflection Probes In Unity

I am not going over how to set up Reflection Probes here only how to access them inside our custom shaders.
So this is what we will be making:
The reflection probe takes in the cubemap only if it is within it's range otherwise i…