Skeletal Animation (part 1 – shader)

In this article, I will describe my understanding of skeletal animation, which is used in all modern 3D engines to animate characters, game environments, etc.
I will start the description from the most simple part – the vertex shader, because the whole way of the calculations, no matter how complicated it was, ends with the transfer of the prepared data to render in the vertex shader.

Skeletal animation after calculating the CPU enters the vertex shader.
I recall the vertex formula without skeletal animation:
gl_Position = projectionMatrix * viewMatrix * modelMatrix * vertex;
For those who do not understand how this formula originated, you can read my article describing the principle of working with matrices to display 3D content in the context of OpenGL.
For the rest – the formula for the implementation of skeletal animation:
” vec4 animatedVertex = bone0matrix * vertex * bone0weight +”
“bone1matrix * vertex * bone1weight +”
“bone2matrix * vertex * bone2weight +”
“bone3matrix * vertex * bone3weight;\n”
” gl_Position = projectionMatrix * viewMatrix * modelMatrix * animatedVertex;\n”

Ie the final matrix of bone transformation is multiplied by the vertex and the weight of this matrix relative to the vertex. Each vertex can be animated with 4 bones, the force of the impact is regulated by the bone weight parameter, the sum of the effects should be equal to one.
What to do if the vertex is affected by less than 4 bones? It is necessary to divide the weight between them, the effect of the rest to be equal to zero.
Mathematically multiplying a weight by a matrix is ​​called “Multiplying a matrix by a scalar”. Multiplication by a scalar allows you to sum up the effects of the matrices on the final vertex.

The bone transformation matrixes themselves are passed in an array. Moreover, the array contains the matrix for the entire model as a whole, and not for each mesh separately.

But for each vertex the following information is transmitted separately:
– Matrix index that affects vertex
– The weight of the matrix that affects the vertex
More than one bone is transmitted, usually 4 bones per vertex are used.
Also the sum of the weights of the 4 bones should always be equal to one.
Next, consider how it looks in the shader.
Array of matrices:
“uniform mat4 bonesMatrices[kMaxBones];”

Information on the effect of 4 bones on each vertex:
“attribute vec2 bone0info;”
“attribute vec2 bone1info;”
“attribute vec2 bone2info;”
“attribute vec2 bone3info;”

vec2 – in the X coordinate, store the bone index (and translate to int in the shader), in the Y coordinate, the weight of the impact of the bone on the vertex. Why do you have to transfer this data in a two-dimensional vector? Because GLSL does not support the transfer of readable C structures with correct fields to the shader.

Below is an example of obtaining the necessary information from a vector, for further substitution in the formula animatedVertex:

“int bone0Index = int(bone0info.x);”
“float bone0weight = bone0info.y;”
“mat4 bone0matrix = bonesMatrices[bone0Index];”

“int bone1Index = int(bone1info.x);”
“float bone1weight = bone1info.y;”
“mat4 bone1matrix = bonesMatrices[bone1Index];”

“int bone2Index = int(bone2info.x);”
“float bone2weight = bone2info.y;”
“mat4 bone2matrix = bonesMatrices[bone2Index];”

“int bone3Index = int(bone3info.x);”
“float bone3weight = bone3info.y;”
“mat4 bone3matrix = bonesMatrices[bone3Index];”

Now the vertex structure that fills up on the CPU should look like this:
x, y, z, u, v, bone0index, bone0weight, bone1index, bone1weight, bone2index, bone2weight, bone3index, bone3weight

The structure of the vertex buffer is filled once at the time of loading the model, but the transformation matrices are transferred from the CPU to the shader at each rendering frame.

In the remaining parts, I will describe the principle of calculating animation on the CPU, before passing to the vertex shader, I will describe the tree of bones, the passage through the hierarchy of animation-model-node-mesh, matrix interpolation.


Source Code