Introduction
Recently, I was approached by a member of the Silent Hill 2 Enhancements project with a proposition to work on solving an issue with some models in one of SH2 maps. Given my busy schedule, I could not commit to such an endeavor, but offered to take a look whenever I had time. Over the course of a week, I was able to replicate the issue and lay the groundwork for a solution.
Reconnaissance
One of the tools mentioned in the bug report is a fan created map viewer. Given the map file in question, I was interested in seeing how the viewer rendered the map. The viewer used OpenGL, so I added apitrace's proxy dll to the executable's directory, changed some paths in its config file and ran the viewer. One of the objects in question is an object whose vertices can be seen being used by call 94611. A key piece of information is that the glDrawElements render call used to draw the object uses triangle strip mode. I exported vertices, texture coordinates and indices from the apitrace tool and used my python script to convert the binary files to an OBJ which I could import into Blender.
python3 raw2obj.py --input-type bin --type positions vertex_call_94611_buffer.raw --stride 32 > vert.obj
python3 raw2obj.py --input-type bin --type texcoords texcord_call_94610_buffer.raw --stride 32 >> vert.obj
python3 raw2obj.py --input-type bin --type indices indices_call_94612_buffer.raw --mode TRIANGLE_STRIP --format UNSIGNED_INT >> vert.obj
In this Silent Hill 2 map, and I suspect in all of them, the objects are in world space, so I had to translate, scale, and change the origin of the imported object to make things a bit easier. Here is the imported object with the UV map.
It was time to find the location of the object data in the map file. I searched for the first vertex and texture coordinate, getting a hit at 0x1d05d8 and 0x1c04b0 respectively. At offset 0x01D05D0 is a uint32 of value 32, followed by a uint32 of value 59712 which is 1866*32 which corresponds to the number of vertices in the model. I suspected that the three floats between the vertex position and texture coordinate was the vertex normal, a suspicion that was later proven true.
To find the indices, I searched for the last index with the pattern 0x70057205. It was found at 0x01E07DF, after which I looked backward to the end of the vertex data to find the beginning of indices at 0x01DEF18.
Now that I had the offsets of all the pieces in the map file, I used my python script again:
python3 raw2obj.py --input-type bin --type positions ap76.map --offset 0x1d05d8 --stride 32 --count 1866 > object.obj
python3 raw2obj.py --input-type bin --type texcoords ap76.map --offset 0x1d05d8 --stride 32 --skip 24 --count 1866 >> object.obj
python3 raw2obj.py --input-type bin --type indices ap76.map --offset 0x01DEF18 --mode TRIANGLE_STRIP --format UNSIGNED_SHORT --count 3043 >> object.obj
python3 raw2obj.py --input-type bin --type normals ap76.map --offset 0x1d05d8 --stride 32 --skip 12 --count 1866 >> object.obj
It can be plainly seen here that the normals are a bit off, just like they are in the game. To fix this, the normals can be recalculated and written to the map file.