Fix out-of-bounds reads in Validate()'s bow-tie and adjacency checks

[Roland Shum (ShumWengSang)]

Problem

Validate() is meant to diagnose untrusted meshes, but two inputs it is specifically supposed to catch instead made it read out of bounds:

• Bow-tie check (DirectXMeshValidate.cpp:335). A vertex index equal to the unused-vertex sentinel index_t(-1) is used directly as the subscript faceIds[j], where faceIds is sized nVerts. ValidateIndices intentionally lets the sentinel through (it marks an unused vertex), so it reaches the bow-tie walk and indexes past the end of the array — an out-of-bounds read and write.

• Adjacency check (DirectXMeshP.h:409, under VALIDATE_ASYMMETRIC_ADJ). An out-of-range adjacency index k is used to read adjacency[k*3 .. +2], guarded only by assert(k < nFaces). In diagnostic-message mode the out-of-range value is recorded but does not stop processing, so it reaches find_edge — and the assert is compiled out under NDEBUG, so release builds had no check at all.

Fix

Both are one-line bounds guards, each mirroring a check the library already uses elsewhere:

• ComputeTangentFrameEx #1 — skip the sentinel before subscripting, exactly as CleanImpl already does on the equivalent loop: if (j == index_t(-1)) continue;
• OptimizeFacesLRU #2 — replace the debug-only assert with a runtime check: if (k >= nFaces) { result = false; continue; }

Total change is +7 / −1 in one file. A valid mesh is unaffected; a genuinely invalid one still returns E_FAIL — only the out-of-bounds access is removed.

Behavior

Each row replays an input against unpatched vs. patched builds (Release /O2 /DNDEBUG, AddressSanitizer). The fault site is the same line the diff touches.

#1 under VALIDATE_BOWTIES; #2 under VALIDATE_ASYMMETRIC_ADJ + diagnostic messages.

#	Fault site	Input	Before	After
1	DirectXMeshValidate.cpp:335	{ 0xFFFF, 1, 2 }, nVerts = 3	heap OOB r/w	S_OK
1	DirectXMeshValidate.cpp:335	{ 0xFFFFFFFF, 1, 2 } (32-bit index)	heap OOB r/w	S_OK
2	DirectXMeshP.h:409	adj { 5, -1, -1 }, nFaces = 1	heap OOB r	E_FAIL
2	DirectXMeshP.h:409	adj { 1, -1, -1 } — k == nFaces boundary	heap OOB r	E_FAIL

Why #1 returns S_OK, not E_FAIL: index_t(-1) marks an unused vertex, not an invalid mesh — CleanImpl skips it the same way, so Validate must agree or the two APIs disagree on the same input. A genuinely out-of-range index still returns E_FAIL (vertex 99, below). The k == nFaces row confirms the new bound is >=, not >.

Regression check	Before → After
valid quad / triangle / unused-face	S_OK → S_OK
genuine bow-tie	E_FAIL → E_FAIL
out-of-range vertex 99 (nVerts = 3)	E_FAIL → E_FAIL

[azure-pipelines]

Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.