7.62 High Calibre on Linux

Install the game using command line steam client:
$ steamcmd +login username password +@sSteamCmdForcePlatformType windows app_update 289890 validate

Create 32 bit wine prefix with DirectX 9:
$ WINEPREFIX=”$HOME/prefix32″ WINEARCH=win32 wine wineboot
$ export WINEPREFIX=$HOME/prefix32/
$ winetricks d3dx9

Launch the game:
$ wine .steam/steamapps/common/7.62/E6.exe

762_high_calibre_linux

Hooking Lua Part 3

Introduction
After learning a bit about how Syrian Warfare handles Lua context, I wanted to figure out how I could execute my own scripts from within the game.

Enabling io
The game does not use the Lua io library, so I modified my injected DLL to have the capability to execute luaopen_io() when F5 key is pressed. I tested the newly enabled io library by executing the following from console:

dump = io.open("testing_io_lua.txt","w")
dump:write("Wrote from lua")
dump:flush()
dump:close()

The file appeared in the game directory with the correct text.

Adding new scripts
I tried to execute dofile() and loadfile() from the command console, but I kept on getting file not found errors. To debug this, I put a breakpoint on lua_load. This led me to the discovery that Galileo loads up scripts from scripts/ and scripts/lua. Looking at strings in SyrianWarefare.exe, I noticed references to “scripts/triggers.lua” and “scripts/global_map.lua.”

global_map.lua uses dofile() to read and execute other lua files:

--************************************************************************
--* Вспомогательные функции и константы                                  *
--************************************************************************
-- utility variables
-- constants
SCRIPTS_PATH = "scripts/lua/"
dofile(SCRIPTS_PATH.."constants.lua")
-- соответствие объекта спауну
dofile(SCRIPTS_PATH.."spawns.lua")
-- соответстиве объекта резервам
dofile(SCRIPTS_PATH.."reserves.lua")
-- вспомогательные функции
dofile(SCRIPTS_PATH.."tools.lua")

I put test.lua into the main.pak zip file and ran the game from Steam. The game started up normally. When I executed the following on console:
dofile(SCRIPTS_PATH..”test.lua”)
the following error printed in DebugView:
[11060] mll::debug::exception: [ml_encrypted_zip: unknown zlib error while inflating]

It appears that the game expects all the files in the zip to be encrypted. Since I know the password used to encrypt the game files, I used 7zip to append the file to main.pak:

$ 7z.exe a main.pak test.lua -pm,nw0rdk1s;ldscj

Conclusion
By appending a Lua script to the game zip files and forcing the game to load Lua io library, I was able to successfully execute test.lua from inside the game. The approach could be used to add and execute custom scripts. The source code for the injected DLL can be found here:
https://github.com/sbobovyc/SyrianWarfare_lua_hacks

Reversing a DirectX Game Part 3

Reproduced From https://sites.google.com/site/sbobovyc/writing/reverse-engineering/reversing-a-directx-game-part-3

DISCLAIMER: The information provided here is for educational purposes only.

Reconnaissance

Looking at calls to CreateFile, this is one of the first files to be accessed. Next the data.pak gets accessed. File mapping object is anonymous.
read_files

“Data\XmlFiles\resources.xml” is:

<?xml version = "1.0" encoding = "UTF-8"?>
<!-- Copyright (c)2004 Lesta Studio -->

<root>
    <!-- Пути для ресурсов. -->
    <object name = "Resources">
        
        <object name = "VFS">
            <object name = "vfs1">
                <string name = "Type" value = "filesystem"/>
                <string name = "Path" value = "Data"/>
                <boolean name = "Recursive" value = "true"/>
            </object>
            

            <object name = "vfsPakFiles">
                <string name = "Type" value = "sma2"/>
                <string name = "Path" value = "data"/>
                <string name = "Mask" value = "*.pak"/>
                <boolean name = "Recursive" value = "true"/>
            </object>

            <object name = "z">
                <string name = "Type" value = "filesystem"/>
                <string name = "Path" value = "Patch"/>
                <boolean name = "Recursive" value = "true"/>
            </object>
            <object name = "z2">
                <string name = "Type" value = "sma2"/>
                <string name = "Path" value = "Patch"/>
                <string name = "Mask" value = "*.pak"/>
                <boolean name = "Recursive" value = "true"/>
            </object>

        </object>
    </object>

        
</root>

The Russian text is “Paths for resources”.

Data.pak gets read here.
Data_Pak
In the last tutorial I showed you how I looked at the 3D models being used by the game using PIX. Now it was time to look at what files the game was using. Games tend to bundle all their files into archives. These are archives tend to be big and 9th Company had one file that stood out: 9th Company\Data\data.pak. This file is 1.32 GB and when opened with a hex editor has a bunch of file names at the top of the file and a bunch of data at the bottom of the file. I searched for DXT (DDS texture) and the hex array “89 50 4E 47 0D 0A 1A 0A” (png) in the pack file and saw many occurring instances. Great! The files are not compressed or encrypted. Aside from games using standard packing formats, it does not get any easier than this.

From this point, it was a matter of experimenting to try to figure out the structure of the header. I looked for the length of the file names, file offset, the total number of files, etc. Once I thought I had a pretty good idea, I wrote a simple loop that followed a pattern to determine how many files are in the header. Once I got to 500 files, i thought to look at the beginning of the file again. Scanning over the first few bytes, integer at offset 0x5 seemed to be the only plausible value. And sure enough, looking at the offset after reading, this is where the file description ended and data began. Also, the integer after number of files was an offset where the first file started.

Pak_Format

The format looked fairly straight forward.

The header was:

struct {
    char type[] ="SMA";
    int8 type_version;
    int16 unknown;
    int32 total_files;
    int32 file_offset;
    int8 file_name_length;
    char file_name[];
}

The main body had the following repeating structure:

struct {
    int32 file_size;
    int32 unknown;
    int32 file_offset;
    int8 file_name_length;
    char file_name[];
}

I had my script print out the file names and offsets:
SMA 2
Number of files: 2904
Data\Art\Buildings\AminPalace\AminPalace.lm, unknown 0x39046, file size 0x85b9d, offset: 0x262c6
Data\Art\Buildings\AminPalace\AminPalace.xml, unknown 0x3f7, file size 0x946, offset: 0x5f30c
Data\Art\Buildings\Army_Buildings\kazarma.lm, unknown 0x12bfd, file size 0x47883, offset: 0x5f703
Data\Art\Buildings\Army_Buildings\kazarma.xml, unknown 0x3afd, file size 0xf832, offset: 0x72300

Using this information, I wrote each file to the disk. Looking at DDS texture file and XML files, it was clear that the files where either ciphered or encrypted in some way. A few of the files were not and the unknown field in those files was zero. At the end of the game manual, it says this:

Zlib, 1995-2002 Jean-loup Gailly & Mark Adler.
Please visit: www.gzip.org/zlib

Looking through the strings in the exe, I found lots of references like these:
007A0B0C=9-Pota.007A0B0C (ASCII “unknown compression method”)
007A0AE0=9-Pota.007A0AE0 (ASCII “incorrect header check”)
007A0AC4=9-Pota.007A0AC4 (ASCII “unknown header flags set”)
007A0AB0=9-Pota.007A0AB0 (ASCII “header crc mismatch”)
deflate 1.2.3 Copyright 1995-2005 Jean-loup Gailly
inflate 1.2.3 Copyright 1995-2005 Mark Adler

They are from zlib inflate.c. Using zlib.decompress() on each dumped file yielded an uncompressed file!

import struct
import os
import sys
import errno
import zlib

class File(object):
    def __init__(self, path, offset, size, unk):
        self.path = path
        self.offset = offset
        self.size = size
        self.unk = unk

    def create(self, file_pointer):
        directory = os.path.dirname(self.path)
        try: os.makedirs(directory)
        except OSError, err:
            # Reraise the error unless it's about an already existing directory 
            if err.errno != errno.EEXIST or not os.path.isdir(directory): 
                raise
        if self.size > 0:
            file_pointer.seek(self.offset)
            data = file_pointer.read(self.size)            
            with open(self.path, "wb") as f:
                # decompress
                if self.unk != 0:
                    data = zlib.decompress(data)
                f.write(data)
    


files = []
with open("data.pak", "rb") as f:
    magic, = struct.unpack("3s", f.read(3))
    version, = struct.unpack("<H", f.read(2))
    print magic, version
    if version != 2:
        print "Wrong version detected!"
        sys.exit()

    num_files, = struct.unpack("<I", f.read(4))
    print "Number of files:", num_files

    unknown, = struct.unpack("<I", f.read(4))

    for i in range(0, num_files):
        count, = struct.unpack("B", f.read(1))
        path, = struct.unpack("%is" % count, f.read(count))
        file_size, = struct.unpack("<I", f.read(4))
        unk1, = struct.unpack("<I", f.read(4))
        offset, = struct.unpack("<I", f.read(4))
        # unk1 is not adler32 or crc32
        print "%s, unknown %s, file size %s, offset: %s" % (path,  hex(unk1), hex(file_size), hex(offset))
        
        files.append(File(path, offset, file_size, unk1))


    for fil in files:
        fil.create(f)

Conclusion
Code for this work can be found herehttps://github.com/sbobovyc/GameTools/tree/master/9rota

References
http://msdn.microsoft.com/en-us/library/windows/desktop/aa366556%28v=vs.85%29.aspx
http://forums.steampowered.com/forums/showthread.php?t=2554597http://www.pacificstorm.net/forum/viewtopic.php?t=2975
http://www.opensource.apple.com/source/zlib/zlib-23.0.1/zlib/inflate.c

Reversing a DirectX Game Part 2

Reproduced from https://sites.google.com/site/sbobovyc/writing/reverse-engineering/reversing-a-directx-game-part-20

DISCLAIMER: The information provided here is for educational purposes only.

Foreword
In the last tutorial, I showed you how to get at the geometry information of a scene using PIX. This tutorial will show you how to dump this geometry information and import it into Blender.

Dumping geometry
I did not have a Blender script ready when I first started this process, so I choose to dump a simple object in order to simplify my debugging steps. Clicking on a draw command and clicking on the mesh tab will show the geometry information, as shown in the previous tutorial. Right clicking on the geometry info will bring up an option to export the current geometry to a CSV file. The pre-Vertex shader CSV dump of a simple object like a hat looks like this:

VTX,IDX,Position[0],Position[1],Position[2],Position[3],Normal[0],Normal[1],Normal[2],TexCoord0[0],TexCoord0[1]
0,0,-0.044,1.747,0.114,1.000,0.643,-0.173,-0.746,0.637,0.335
1,45,0.005,1.743,0.129,1.000,-0.035,-0.010,-0.999,0.652,0.335
2,1,-0.032,1.792,0.117,1.000,0.460,-0.546,-0.700,0.642,0.322
3,1,-0.032,1.792,0.117,1.000,0.460,-0.546,-0.700,0.642,0.322
4,45,0.005,1.743,0.129,1.000,-0.035,-0.010,-0.999,0.652,0.335
5,23,0.005,1.788,0.128,1.000,-0.027,-0.439,-0.898,0.649,0.323
...
etc

PIX tells us that the format of this list is three vertices per primitive.

Importing into Blender
It is a simple matter of writing a python script to import this CSV into Blender. The full script can be found here. I will not explain the script in detail since the code is straight forward. I will explain the basic process and a few caveats. The first thing I did was import the geometry. This consists of the vertex indices and their XYZ positions. Once that was done, I imported the texture (UV) coordinates. After debugging those, I imported the normals.

The first is that DirectX and Blender use different coordinate systems. DirectX uses a left handed system with the Y axis being up. Blender uses a right handed system with the Z axis being up. The importer script takes care of this conversion. When importing the object, each triangle had its own set of vertices. That is, vertices were not shared between faces of the mesh.

As you can see in the following image, the V component is wrong because the UV map does not line up correctly with the texture.
uv_map

To fix this, simply subtract the V of each vertex from 1. Now the UV map is correct.

uv_map2

Here is the render of the hat.
hat_render

Another thing to watch out for is how normals are oriented. Taking the normals straight from PIX, they are wrong:
wrong_normals

DirectX traverses vertices in clockwise order while Blender does it in counter-clockwise order. This differences causes the normal to point the wrong way. The provided script takes care of vertex reordering by default.

After importing unsymmetrical meshes, it was clear that they where mirrored across the x axis. The mirroring is fixed by negating the x coordinate in each vertex.

Here you can see a correct render AKS-74N. This render includes both the diffuse texture and normal map.

aks74n_dumped

Once I have the mesh and the textures, it is pretty simple to port them to another game. Here is the same AKS-74N that has been port to Jagged Alliance: Back in Action:
aks74n_in_jabia

Conclusion
There some problems with this process. PIX rounds all the floating point numbers to the 3rd decimal place. Another problem is that the material properties are not preserved. Finally, this process is on directional. To be able to export your own meshes so that they are usable by the game, the 3D format has to be reverse engineered. In the next tutorial, I will show you how I reverse engineer the packing format used by this game to get at the 3D and texture files.

Reversing a DirectX Game Part 1

Reproduced from https://sites.google.com/site/sbobovyc/writing/reverse-engineering/reversing-a-directx-game-part-1

DISCLAIMER: The information provided here is for educational purposes only.

Foreword
Over the last year I’ve been learning the art of reverse engineering games for modding purposes. It has been a fun process, but full of trial and error. I want to share my experiences here. The goal of this tutorial is to take you through the steps of reverse engineering game formats. Why is this useful? Most games these days don’t support modding. For example, Jagged Alliance: Back in Action did not officially support modding, but I liked the game so much that I wanted to change that. The result was the JABIA Tools project, which included a myriad of tools including importer and exporter plugins for Blender.

Instead of showing you what I’ve done before, I decided to start a new reversing project for the purposes of this tutorial. I choose an obscure game, 9th Company: Roots of Terror. It is based on a Russian movie set during the Soviet-Afghan war. Why did I choose this game? I bought it, played the first few missions and uninstalled. This is a chance to recoup the $10 I spent. In addition, it has some nice low poly models.

This tutorial is aimed at intermediate users. I will not be explaining hexadecimal number representation, DirectX 9 or how 3D graphics work. These topics are very large and covered much better in other places. I am to help you build on top of your knowledge to learn how to do something that is not taught in a computer science curriculum. If you are not a technical person, but you enjoy video games and are interested in how they work, I encourage you to continue reading.

Reconnaissance
9th Company is developed by a Russian studio, Lesta Studio and uses the AdicoEngine2. Not much can be found online about this engine, but apparently there are some Russian modding tools. I spent a few moments trying to find them, but only found a few screenshots.

Graphics debugging
It was time to start up the tools and do some investigating. One of my favorite tools for this task is DirectX PIX. It comes with the DirectX 9 SDK and is used for profiling/debugging. It is also great for reversing.
I started the game under PIX and did a single frame capture during the tutorial mission. Here you can see the the rendered frame.

pix_frame_render-1

With PIX, you can debug an individual pixel. This will show you all the draw calls to that pixel. However, this game uses multiple render targets, so debugging a pixel will only show the calls for the last render target. I simply picked a draw call in the middle of the trace. From there, I debugged a pixel that was part of the AKS-74N. This led me to the draw call for this mesh. It is interesting to note that the engine uses a single draw call to draw all the instances of this mesh. You can tell that this is happening because there are multiple copies of the rifle in the post-vertex output. PIX tells us a treasure trove of information: faces, vertices, vertex positions, surface normals and UV texture coordinates. This information could be exported to a CSV file and turned into a 3D model
pix_ak74s-1

A few function calls before the draw, the vertex declaration is set. PIX prints it out really nice for us. This is an important piece of information that gives us a hint of how the game’s custom 3D file format is structured.
pix_vertex_declaration-1

Getting the textures to the model is also pretty simple. PIX can do this, but in the interested of showing more tools I will show you how to do it with Intel GPA. Simply right click on the texture and save the texture as a DDS or PNG.

gpa_ak74s_texture-1

At this point, I could simply dump the mesh information and textures and be done with it. If all you care about is ripping a model, this is a way to do it. However, if you want to build modding tools, you have to pretty much fully reverse engineer the 3D file format. In the next tutorial I will show you how to dump the mesh geometry with PIX and import it into Blender.

Conclusion

The code for this work can be found here https://github.com/sbobovyc/GameTools/tree/master/9rota