Question: why does the worldFromModelNormalMatrix apply to tangent vector?

[Hearwindsaying]

Moved from issue: #3952

In main() of main.vs:

        // We don't need to normalize here, even if there's a scale in the matrix
        // because we ensure the worldFromModelNormalMatrix pre-scales the normal such that
        // all its components are < 1.0. This prevents the bitangent to exceed the range of fp16
        // in the fragment shader, where we renormalize after interpolation
        vertex_worldTangent.xyz = objectUniforms.worldFromModelNormalMatrix * vertex_worldTangent.xyz;
        vertex_worldTangent.w = mesh_tangents.w;
        material.worldNormal = objectUniforms.worldFromModelNormalMatrix * material.worldNormal;

This might be a graphics/math related problem.
I know that for normal vector transformation, it should use a fixed transform to ensure that it's perpendicular to the tangent plane after transformation.
But why do we apply this to tangent vector instead of using the trivial worldFromModelMatrix?

@romainguy I cannot understand why tangent is subject to the same problems as transform does.

[hoqhuuep]

I believe you are correct. And I believe this is still a bug in the Filament source code. I found your comment here after noticing the issue in the source code and googling to see if there was any discussion about it.

The offending source code in Filament is here: https://github.com/google/filament/blob/main/shaders/src/surface_main.vs#L87-L93

The vertex normals are correctly transformed using the inverse-transpose matrix (or actually, the potentially better cofactor matrix).

However, as @Hearwindsaying has, I believe, correctly pointed out, the vertex tangents should be transformed with the upper-left 3x3 portion of the worldFromModelMatrix, not the worldFromModelNormalMatrix.

This is because the normals need to remain 'normal' to the geometry when subjected to non-uniform scaling, however the tangents need to remain 'tangent' to the geometry. This is a different requirement.

@romainguy could this topic be revisited?

Disclaimer: I am currently learning these concepts myself, I am definitely not an expert, but my current investigations have lead me to believe that @Hearwindsaying is correct on this one.

---

This comment by Ben Golus on the Unity forums explains this concept, with helpful diagrams to consider:

https://discussions.unity.com/t/which-gets-the-inverse-multiplication-normals-or-tangents/899322/2

"Normals are transformed by the transposed inverse object to world matrix."
"Tangents are transformed by the object to world matrix."
"As for tangents, those vectors are parallel to each side, so you do want them to “squish” with the shape."

---

Also this wikibook:

https://en.wikibooks.org/wiki/Cg_Programming/Unity/Lighting_of_Bumpy_Surfaces#Normal_Mapping_in_Unity

"Note that the normal vector N is transformed with the transpose of the inverse model matrix from object space to world space (because it is orthogonal to a surface; see Section “Applying Matrix Transformations”) while the tangent vector T specifies a direction between points on a surface and is therefore transformed with the model matrix."

---

Also refer to the implementation in Bevy where this is done correctly:

https://github.com/bevyengine/bevy/blob/main/crates/bevy_pbr/src/render/mesh.wgsl#L83-L91
https://github.com/bevyengine/bevy/blob/main/crates/bevy_pbr/src/render/mesh_functions.wgsl#L101-L125

---

EDIT: Add another reference

Also refer to the glTF-Sample-Renderer where the is done correctly:

https://github.com/KhronosGroup/glTF-Sample-Renderer/blob/main/source/Renderer/shaders/primitive.vert#L123-L124

[romainguy]

@pixelflinger to answer this

[pixelflinger]

It is entirely possible this is an oversight. I don't remember how we made this choice back then. But it certainly looks wrong.

I'm happy to review a patch referencing an actual bug it would fix.

Thanks for the investigation and links.

[hoqhuuep]

Thank you for the replies!

A further point to be careful of if anyone wants to dig into this further is the use of the normal and tangent conforming to the MikkTSpace convention.

The current vertex shader does not normalize the normal and tangent (with a comment explaining why):

• https://github.com/google/filament/blob/main/shaders/src/surface_main.vs#L87-L93

The the matrix used to transform them (worldFromModelNormalMatrix) will cause some scaling:

• https://github.com/google/filament/blob/main/filament/src/details/InstanceBuffer.cpp#L109-L110
• https://github.com/google/filament/blob/main/libs/math/include/math/mat3.h#L473-L483

Then the non-normalized (interpolated) values are used for constructing the TBN matrix:

• https://github.com/google/filament/blob/main/shaders/src/surface_shading_parameters.fs

The potential issue here is that MikkTSpace (which I still have yet to fully wrap my head around) specifies (http://www.mikktspace.com/):

1. "keep the vertex normal and the vertex tangent normalized in the vertex shader"
2. "However, in the pixel shader the "unnormalized" and interpolated vertex normal and tangent are used to decode the tangent space normal."
3. "The bitangent is constructed here to avoid the use of an additional interpolater and again is NOT normalized."

The current implementation conforms with points 2 and 3. However it is not conforming with point 1. This is maybe OK in the current implementation (I'm really not sure, needs more investigation!) because the same matrix is used (incorrectly) to transform both the normal and the tangent?? But when the issue at hand is fixed, and the code is updated to use the correct, different matrices for transforming the normal and tangent, perhaps this will be broken (if it isn't already?)??

Not entirely sure, but I'd be uncomfortable to just make the naive changes here without a deeper understanding of the downstream consequences.

Sorry that I'm not an expert who can give an authoritative answer on all this, I'm just trying to learn it and doing a lot of studying existing code in the process... and this doesn't seem to add up to me.

I hope these observations are useful and perhaps will help someone to fix the issue without introducing any further issues, and then we can all enjoy perfectly non-skewed normal mapped normals on non-uniformly scaled meshes.