Let there be shadow!

Original post can be found here.

Shadows in Unity is something that in most cases is a given through the use of surface shaders, but sometimes you don't want to use a surface shader for whatever reason and create your own vertex/fragment shader. The biggest advantage is that everything is in your hands now, but this is also one of the drawbacks because you now have to handle a lot of stuff that Unity conveniently handled for you in a surface shader. Among such things are support for multiple lights and shadows.

Luckily, Unity provides you the means to get this working! The catch? Documentation on this is lacking or even non-existent. I was in the same position as most people and somewhat clueless on how to get shadows in my vertex/fragment shader, I did my fair share of googling and found some clues that didn't quit do the trick, but gave me a good impression on where to search. I also went through a compiled surface shader to see if I could figure out how they did it. All of the research combined and some trying out finally gave me the results I needed: Shadows! And now I will share it with whoever is interested.

Before I begin, I want to make note that as mentioned earlier, Unity solves a lot of cases for you when you are using surface shaders, among such things are the inner workings when you are using deferred or forward rendering. With your own vertex/fragment shaders, you will need to take that into account yourself for some cases. Truth is, I only needed to get this to work with forward rendering and only briefly tested how this works with deferred rendering and although I did not notice anything off, I can't guarantee it will work in all cases, so keep that in mind!

I will start off with showing you the shader that casts (and receives) a nice shadow and break it down, going over the different elements of interest. It's a simple diffuse shader that looks like this:

Shader "Sample/Diffuse" 
{
	Properties 
	{
		_DiffuseTexture ("Diffuse Texture", 2D) = "white" {}
		_DiffuseTint ( "Diffuse Tint", Color) = (1, 1, 1, 1)
	}

	SubShader 
	{
		Tags { "RenderType"="Opaque" }

		pass
		{		
			Tags { "LightMode"="ForwardBase"}

			CGPROGRAM

			#pragma target 3.0
			#pragma fragmentoption ARB_precision_hint_fastest

			#pragma vertex vertShadow
			#pragma fragment fragShadow
			#pragma multi_compile_fwdbase

			#include "UnityCG.cginc"
			#include "AutoLight.cginc"

			sampler2D _DiffuseTexture;
			float4 _DiffuseTint;
			float4 _LightColor0;

			struct v2f
			{
				float4 pos : SV_POSITION;
				float3 lightDir : TEXCOORD0;
				float3 normal : TEXCOORD1;
				float2 uv : TEXCOORD2;
				LIGHTING_COORDS(3, 4)
			};

			v2f vertShadow(appdata_base v)
			{
				v2f o;

				o.pos = mul(UNITY_MATRIX_MVP, v.vertex);
				o.uv = v.texcoord;
				o.lightDir = normalize(ObjSpaceLightDir(v.vertex));
				o.normal = normalize(v.normal).xyz;

				TRANSFER_VERTEX_TO_FRAGMENT(o);

				return o; 
			}

			float4 fragShadow(v2f i) : COLOR
			{					
				float3 L = normalize(i.lightDir);
				float3 N = normalize(i.normal);	 

				float attenuation = LIGHT_ATTENUATION(i) * 2;
				float4 ambient = UNITY_LIGHTMODEL_AMBIENT * 2;

				float NdotL = saturate(dot(N, L));
				float4 diffuseTerm = NdotL * _LightColor0 * _DiffuseTint * attenuation;

				float4 diffuse = tex2D(_DiffuseTexture, i.uv);

				float4 finalColor = (ambient + diffuseTerm) * diffuse;

				return finalColor;
			}

			ENDCG
		}		

	} 
	FallBack "Diffuse"
}

If you have ever worked with vertex/fragment shaders you will notice that there isn't much to be noted except for a few macros, but let's address the first things you will need to do to get those shadows.

The first thing you will need to define is the LightMode Pass Tag:

Tags { "LightMode"="ForwardBase"}

This will tell unity that this pass will make use of the main light that will cast our shadow (there's more to this tag, check the link for more info). Unity handles each light in their own pass, so if we want to work with multiple lights, this value in another pass would change to ForwardAdd.

Next to the tag, we also need to define the following:

#pragma multi_compile_fwdbase

This is to ensure the shader compiles properly for the needed passes. As with the tag, for any additional lights in their own pass, fwdbase becomes fwdadd.

To make use of all the needed code/macros to sample shadows in our shader, we will need to include the AutoLight.cginc that holds all the goodness:

#include "AutoLight.cginc"

Now Unity knows all it needs on how to handle the lights, now we just need to get the relevant data to get our shadow to appear and for that we only have to do 3 things:

To make Unity "generate" the values we need, all we have to do is add the LIGHTING_COORDS macro to our vertex to fragment struct like so:

struct v2f
{
	float4 pos : SV_POSITION;
	float3 lightDir : TEXCOORD0;
	float3 normal : TEXCOORD1;
	float2 uv : TEXCOORD2;
	LIGHTING_COORDS(3, 4)
};

The LIGHTING_COORDS macro defines the parameters needed to sample the shadow map and the light map depending on the light source. The numbers specified are the next 2 available TEXCOORD semantics available. So if I would need a viewing direction for a specular highlight, the struct would look like this:

struct v2f
{
	float4 pos : SV_POSITION;
	float3 lightDir : TEXCOORD0;
	float3 normal : TEXCOORD1;
	float2 uv : TEXCOORD2;
	float3 viewDir : TEXCOORD3;
	LIGHTING_COORDS(4, 5)
};

This is much like defining them yourself, except that now it's guaranteed for Unity that they're using the right values for the right light sources with perhaps also a cookie texture attached to them. If you're curious as to what gets defined exactly, check out the AutoLight.cginc file.

Next up is the vertex shader. Having the values is one thing, but we need them to hold the right data and Unity provides another macro that fills it up with the right data for the right situation, this is done with the TRANSFER_VERTEX_TO_FRAGMENT macro. This macro must be defined before returning the v2f struct, so your vertex shader would look something like this:

v2f vertShadow(appdata_base v)
{
	v2f o;

	o.pos = mul(UNITY_MATRIX_MVP, v.vertex);
	o.uv = v.texcoord;
	o.lightDir = normalize(ObjSpaceLightDir(v.vertex));
	o.normal = normalize(v.normal).xyz;

	TRANSFER_VERTEX_TO_FRAGMENT(o);

	return o; 
}

Not much is to be said about this, other than that it takes care of calculating the light and shadow coordinates for you for the different lights.

At this moment, all we have left is to create our fragment program that is able to use the LIGHT_ATTENUATION macro that returns the correct values we need for our shadow. You can use the attenuation value like you would normally, for diffuse shading I use it in the diffuse term like this in the fragment shader:

float4 fragShadow(v2f i) : COLOR
{					
	float3 L = normalize(i.lightDir);
	float3 N = normalize(i.normal);	 

	float attenuation = LIGHT_ATTENUATION(i) * 2;
	float4 ambient = UNITY_LIGHTMODEL_AMBIENT * 2;

	float NdotL = saturate(dot(N, L));
	float4 diffuseTerm = NdotL * _LightColor0 * _DiffuseTint * attenuation;

	float4 diffuse = tex2D(_DiffuseTexture, i.uv);

	float4 finalColor = (ambient + diffuseTerm) * diffuse;

	return finalColor;
}

And there you have it, everything you need to get that lovely shadow in your vertex/fragment shaders. The LIGHT_ATTENUATION samples the shadowmap and returns the value for you to use. Once again, if you want to know what LIGHT_ATTENUATION exactly does, check out the AutoLight.cginc.

There is still one little thing to be noted however. For Unity to have something cast and/or receive a shadow, you must provide a shadow receiver and caster pass which I didn't provide here. Instead of making them yourself, I simply added a fallback shader that has these passes so I don't have to add them myself and make the shader bigger than it already is. You can of course add this to a .cginc or put them all the way down and never look back at it, but just adding a fallback works just as well for our shadow purpose.

I hope this clears things up a bit for those struggling to get their shaders cast and/or receive shadows. Feel free to leave me a comment or mail me if you have any questions or remarks on this post!