polysail commented at 2020-06-05T01:32:41Z

This doesn't seem to be a vertex normals issue at all.  If vertex normals were not altered by bone deformation literally no animation would work at all, nor would normal maps would display properly on rigged meshes.  You also specifically mention that this effect does not occur while using a Neutral Windlight Setting in your forum post:

"If you're using Second Life NOT with a shadowless WL presets like CalWL, this affects your avatar for years."

This further indicates that this issue has little to do with vertex normals at all ~ in fact it's isolated exclusively to the shadow calculations specifically ~  If I had to guess, something about the avatar shape sliders simply is bypassed in the simplified shadows calculation and that's why we're seeing this effect.

polysail commented at 2020-06-20T21:13:30Z, updated at 2020-06-20T21:17:57Z

So, there actually IS a vertex normals issue with meshes: https://jira.secondlife.com/browse/BUG-228952, but I don't think that it's directly related to this issue at all.  Vertex normals affect surface lighting calculations, and normal map display ( by extension ), but they're largely irrelevant in shadow calculations which seems to be the cause of the anomaly in this bug report.

Beev Fallen commented at 2020-06-21T09:55:55Z

Thank you polysail for your investigation and efforts, I personally think this IS the same issue. In my simple way of understanding this, the shape sliders are changing scale on some axis, so if we have 'broken' vertex normals after mesh upload, shape sliders change them into 'broken' ways too. I probably gave misleading terms in my JIRA text, I wasn't talking about shadows calculations at all, I was talking about 'shading', and you call that thing 'surface lighting'. Basically we are talking about the same thing.

polysail commented at 2020-06-21T12:32:29Z, updated at 2020-06-21T12:33:22Z

I mean~  if you give me the DAE files you uploaded~ I can try uploading them with the patched viewer that doesn't bork the surface normals ~ or you can just encase your test meshes within invisible boxes yourself and see if the problem persists~  I strongly suspect it will, implying that my assertion that these two issues are not directly related is correct.  I'll happily be proven wrong though!~  As it would kill 2 birds with one stone.

However~  I don't think..... this is the case.  Though you might be correct in your assessment about this yet.  When I was discussing the mesh uploader issue with Beq ~ we were using the Alice-in-Wonderland adventure as a kind of 'convention' to describe what meshes go through in order to be viewed in-world.   Upon upload they go through a "Drink Me" calculation ~ that is to say ~ they are taken from their original coordinate space ~ and squished into unit cubes.   The rest of the operation is the "Eat Me" operation ~ the undoing of the unitized scale operation ~ to expand the mesh into a presentable and viewable asset to the world.  The "Eat Me" calculation has to be arbitrary though ~ as the user can at any point decide to make an object very large ~ or very small.  We have NOT looked at that part of the code yet.  When I say our two issues are unrelated ~I mean that BUG-228952 is an issue with "Drink Me".  Whether "Eat Me" is also borked is essentially this report.  It's very very very possible that it is...

If I toss the most simplistic spherical prim on the ground in SL and start scaling it, the surface normals do NOT recalculate on the fly as the object undergoes a scale transform.  You know how surface normals would be recalculated based on object scale??  That would be an inverse_scale function...  This is how it's handled in Maya...

So there's a part of me.. sitting here going 'well $@#)%'

Beev Fallen commented at 2020-06-21T16:00:36Z

attached DAE files, head.dae is a rigged one, two others are unrigged with min and max nose sizes.

polysail commented at 2020-07-06T18:04:16Z

https://gyazo.com/3fc35d4aa5263dd8aa12926012d0b70c

Beq was looking at this bug again which reminded me about it's existence.

The screenshot above shows 3 objects worn by the avatar:
The Greens Tinted Object is an Unrigged Sphere ~ squashed into the shape of a disc, so it's vertex normals behave in a ~ what we assume to be proper functional manner.

The Purple object is one of my special test objects.  It's an Unrigged Sphere, that was squashed into a disc in Maya.  However, it has the vertex normals of a fully inflated sphere ~  despite being disc shaped.  It's designed to look like a sphere, even when it's a disc.

The last White tinted Object is another sphere, this one rigged to the LEFT_PEC bone.  As you can see, when the shapes are changed from 0 >> 100 breast size, the sphere changes shape, but it continues to reflect light in a manner that is more similar to that of the Purple Object, than the Green one  ~which would seem to indicate that it's got vertex normals more closely resembling that of the Purple test-object.

Ptolemy Linden commented at 2020-07-06T22:45:54Z

Beq mentioned that this whitepaper may be of interest:

Accurate and Efficient Lighting for Skinned Models
http://vcg.isti.cnr.it/deformFactors/

Beq Janus commented at 2020-07-07T01:08:59Z, updated at 2020-07-07T01:10:28Z

This is a normals issue, as best I can tell. However, it is not a bug so much as a deliberate choice. 

In order to calculate the correct normals of a mesh post-weighting you need to be able to average the normals of the associated triangles that join at that vertex. The vertex normal being the weighted average of the face normals. That data is not available at the vertex shader. as such it is commonplace to approximate things at least on older realtime systems. 

One example explanation for this is here https://computergraphics.stackexchange.com/questions/4656/calculating-skeleton-deformed-mesh-normals

There are solutions to this, as mentioned in the thread but none of them is particularly useful to us at this time, reverting back to CPU rendering, for example, is not exactly where we want to go. Furthermore, there are more modern implementations (See Ptolemy's note above for one such example) that allow a more efficient calculation to be done at the vertex shader, however, it effectively doubles the amount of data sent from the CPU to the GPU, per-vertex, and given the average (excessive) complexity of rigged mesh items currently sold in SL the potential performance implications are rather scary and would need to be carefully considered before implementation. It is something that should be reviewed as things evolve so it is good to see this accepted.