Hooking Lua Part 3

Introduction
After learning a bit about how Syrian Warfare handles Lua context, I wanted to figure out how I could execute my own scripts from within the game.

Enabling io
The game does not use the Lua io library, so I modified my injected DLL to have the capability to execute luaopen_io() when F5 key is pressed. I tested the newly enabled io library by executing the following from console:

dump = io.open("testing_io_lua.txt","w")
dump:write("Wrote from lua")
dump:flush()
dump:close()

The file appeared in the game directory with the correct text.

Adding new scripts
I tried to execute dofile() and loadfile() from the command console, but I kept on getting file not found errors. To debug this, I put a breakpoint on lua_load. This led me to the discovery that Galileo loads up scripts from scripts/ and scripts/lua. Looking at strings in SyrianWarefare.exe, I noticed references to “scripts/triggers.lua” and “scripts/global_map.lua.”

global_map.lua uses dofile() to read and execute other lua files:

--************************************************************************
--* Вспомогательные функции и константы                                  *
--************************************************************************
-- utility variables
-- constants
SCRIPTS_PATH = "scripts/lua/"
dofile(SCRIPTS_PATH.."constants.lua")
-- соответствие объекта спауну
dofile(SCRIPTS_PATH.."spawns.lua")
-- соответстиве объекта резервам
dofile(SCRIPTS_PATH.."reserves.lua")
-- вспомогательные функции
dofile(SCRIPTS_PATH.."tools.lua")

I put test.lua into the main.pak zip file and ran the game from Steam. The game started up normally. When I executed the following on console:
dofile(SCRIPTS_PATH..”test.lua”)
the following error printed in DebugView:
[11060] mll::debug::exception: [ml_encrypted_zip: unknown zlib error while inflating]

It appears that the game expects all the files in the zip to be encrypted. Since I know the password used to encrypt the game files, I used 7zip to append the file to main.pak:

$ 7z.exe a main.pak test.lua -pm,nw0rdk1s;ldscj

Conclusion
By appending a Lua script to the game zip files and forcing the game to load Lua io library, I was able to successfully execute test.lua from inside the game. The approach could be used to add and execute custom scripts. The source code for the injected DLL can be found here:
https://github.com/sbobovyc/SyrianWarfare_lua_hacks

Reversing a DirectX Game Part 3

Reproduced From https://sites.google.com/site/sbobovyc/writing/reverse-engineering/reversing-a-directx-game-part-3

DISCLAIMER: The information provided here is for educational purposes only.

Reconnaissance

Looking at calls to CreateFile, this is one of the first files to be accessed. Next the data.pak gets accessed. File mapping object is anonymous.
read_files

“Data\XmlFiles\resources.xml” is:

<?xml version = "1.0" encoding = "UTF-8"?>
<!-- Copyright (c)2004 Lesta Studio -->

<root>
    <!-- Пути для ресурсов. -->
    <object name = "Resources">
        
        <object name = "VFS">
            <object name = "vfs1">
                <string name = "Type" value = "filesystem"/>
                <string name = "Path" value = "Data"/>
                <boolean name = "Recursive" value = "true"/>
            </object>
            

            <object name = "vfsPakFiles">
                <string name = "Type" value = "sma2"/>
                <string name = "Path" value = "data"/>
                <string name = "Mask" value = "*.pak"/>
                <boolean name = "Recursive" value = "true"/>
            </object>

            <object name = "z">
                <string name = "Type" value = "filesystem"/>
                <string name = "Path" value = "Patch"/>
                <boolean name = "Recursive" value = "true"/>
            </object>
            <object name = "z2">
                <string name = "Type" value = "sma2"/>
                <string name = "Path" value = "Patch"/>
                <string name = "Mask" value = "*.pak"/>
                <boolean name = "Recursive" value = "true"/>
            </object>

        </object>
    </object>

        
</root>

The Russian text is “Paths for resources”.

Data.pak gets read here.
Data_Pak
In the last tutorial I showed you how I looked at the 3D models being used by the game using PIX. Now it was time to look at what files the game was using. Games tend to bundle all their files into archives. These are archives tend to be big and 9th Company had one file that stood out: 9th Company\Data\data.pak. This file is 1.32 GB and when opened with a hex editor has a bunch of file names at the top of the file and a bunch of data at the bottom of the file. I searched for DXT (DDS texture) and the hex array “89 50 4E 47 0D 0A 1A 0A” (png) in the pack file and saw many occurring instances. Great! The files are not compressed or encrypted. Aside from games using standard packing formats, it does not get any easier than this.

From this point, it was a matter of experimenting to try to figure out the structure of the header. I looked for the length of the file names, file offset, the total number of files, etc. Once I thought I had a pretty good idea, I wrote a simple loop that followed a pattern to determine how many files are in the header. Once I got to 500 files, i thought to look at the beginning of the file again. Scanning over the first few bytes, integer at offset 0x5 seemed to be the only plausible value. And sure enough, looking at the offset after reading, this is where the file description ended and data began. Also, the integer after number of files was an offset where the first file started.

Pak_Format

The format looked fairly straight forward.

The header was:

struct {
    char type[] ="SMA";
    int8 type_version;
    int16 unknown;
    int32 total_files;
    int32 file_offset;
    int8 file_name_length;
    char file_name[];
}

The main body had the following repeating structure:

struct {
    int32 file_size;
    int32 unknown;
    int32 file_offset;
    int8 file_name_length;
    char file_name[];
}

I had my script print out the file names and offsets:
SMA 2
Number of files: 2904
Data\Art\Buildings\AminPalace\AminPalace.lm, unknown 0x39046, file size 0x85b9d, offset: 0x262c6
Data\Art\Buildings\AminPalace\AminPalace.xml, unknown 0x3f7, file size 0x946, offset: 0x5f30c
Data\Art\Buildings\Army_Buildings\kazarma.lm, unknown 0x12bfd, file size 0x47883, offset: 0x5f703
Data\Art\Buildings\Army_Buildings\kazarma.xml, unknown 0x3afd, file size 0xf832, offset: 0x72300

Using this information, I wrote each file to the disk. Looking at DDS texture file and XML files, it was clear that the files where either ciphered or encrypted in some way. A few of the files were not and the unknown field in those files was zero. At the end of the game manual, it says this:

Zlib, 1995-2002 Jean-loup Gailly & Mark Adler.
Please visit: www.gzip.org/zlib

Looking through the strings in the exe, I found lots of references like these:
007A0B0C=9-Pota.007A0B0C (ASCII “unknown compression method”)
007A0AE0=9-Pota.007A0AE0 (ASCII “incorrect header check”)
007A0AC4=9-Pota.007A0AC4 (ASCII “unknown header flags set”)
007A0AB0=9-Pota.007A0AB0 (ASCII “header crc mismatch”)
deflate 1.2.3 Copyright 1995-2005 Jean-loup Gailly
inflate 1.2.3 Copyright 1995-2005 Mark Adler

They are from zlib inflate.c. Using zlib.decompress() on each dumped file yielded an uncompressed file!

import struct
import os
import sys
import errno
import zlib

class File(object):
    def __init__(self, path, offset, size, unk):
        self.path = path
        self.offset = offset
        self.size = size
        self.unk = unk

    def create(self, file_pointer):
        directory = os.path.dirname(self.path)
        try: os.makedirs(directory)
        except OSError, err:
            # Reraise the error unless it's about an already existing directory 
            if err.errno != errno.EEXIST or not os.path.isdir(directory): 
                raise
        if self.size > 0:
            file_pointer.seek(self.offset)
            data = file_pointer.read(self.size)            
            with open(self.path, "wb") as f:
                # decompress
                if self.unk != 0:
                    data = zlib.decompress(data)
                f.write(data)
    


files = []
with open("data.pak", "rb") as f:
    magic, = struct.unpack("3s", f.read(3))
    version, = struct.unpack("<H", f.read(2))
    print magic, version
    if version != 2:
        print "Wrong version detected!"
        sys.exit()

    num_files, = struct.unpack("<I", f.read(4))
    print "Number of files:", num_files

    unknown, = struct.unpack("<I", f.read(4))

    for i in range(0, num_files):
        count, = struct.unpack("B", f.read(1))
        path, = struct.unpack("%is" % count, f.read(count))
        file_size, = struct.unpack("<I", f.read(4))
        unk1, = struct.unpack("<I", f.read(4))
        offset, = struct.unpack("<I", f.read(4))
        # unk1 is not adler32 or crc32
        print "%s, unknown %s, file size %s, offset: %s" % (path,  hex(unk1), hex(file_size), hex(offset))
        
        files.append(File(path, offset, file_size, unk1))


    for fil in files:
        fil.create(f)

Conclusion
Code for this work can be found herehttps://github.com/sbobovyc/GameTools/tree/master/9rota

References
http://msdn.microsoft.com/en-us/library/windows/desktop/aa366556%28v=vs.85%29.aspx
http://forums.steampowered.com/forums/showthread.php?t=2554597http://www.pacificstorm.net/forum/viewtopic.php?t=2975
http://www.opensource.apple.com/source/zlib/zlib-23.0.1/zlib/inflate.c

Reversing a DirectX Game Part 1

Reproduced from https://sites.google.com/site/sbobovyc/writing/reverse-engineering/reversing-a-directx-game-part-1

DISCLAIMER: The information provided here is for educational purposes only.

Foreword
Over the last year I’ve been learning the art of reverse engineering games for modding purposes. It has been a fun process, but full of trial and error. I want to share my experiences here. The goal of this tutorial is to take you through the steps of reverse engineering game formats. Why is this useful? Most games these days don’t support modding. For example, Jagged Alliance: Back in Action did not officially support modding, but I liked the game so much that I wanted to change that. The result was the JABIA Tools project, which included a myriad of tools including importer and exporter plugins for Blender.

Instead of showing you what I’ve done before, I decided to start a new reversing project for the purposes of this tutorial. I choose an obscure game, 9th Company: Roots of Terror. It is based on a Russian movie set during the Soviet-Afghan war. Why did I choose this game? I bought it, played the first few missions and uninstalled. This is a chance to recoup the $10 I spent. In addition, it has some nice low poly models.

This tutorial is aimed at intermediate users. I will not be explaining hexadecimal number representation, DirectX 9 or how 3D graphics work. These topics are very large and covered much better in other places. I am to help you build on top of your knowledge to learn how to do something that is not taught in a computer science curriculum. If you are not a technical person, but you enjoy video games and are interested in how they work, I encourage you to continue reading.

Reconnaissance
9th Company is developed by a Russian studio, Lesta Studio and uses the AdicoEngine2. Not much can be found online about this engine, but apparently there are some Russian modding tools. I spent a few moments trying to find them, but only found a few screenshots.

Graphics debugging
It was time to start up the tools and do some investigating. One of my favorite tools for this task is DirectX PIX. It comes with the DirectX 9 SDK and is used for profiling/debugging. It is also great for reversing.
I started the game under PIX and did a single frame capture during the tutorial mission. Here you can see the the rendered frame.

pix_frame_render-1

With PIX, you can debug an individual pixel. This will show you all the draw calls to that pixel. However, this game uses multiple render targets, so debugging a pixel will only show the calls for the last render target. I simply picked a draw call in the middle of the trace. From there, I debugged a pixel that was part of the AKS-74N. This led me to the draw call for this mesh. It is interesting to note that the engine uses a single draw call to draw all the instances of this mesh. You can tell that this is happening because there are multiple copies of the rifle in the post-vertex output. PIX tells us a treasure trove of information: faces, vertices, vertex positions, surface normals and UV texture coordinates. This information could be exported to a CSV file and turned into a 3D model
pix_ak74s-1

A few function calls before the draw, the vertex declaration is set. PIX prints it out really nice for us. This is an important piece of information that gives us a hint of how the game’s custom 3D file format is structured.
pix_vertex_declaration-1

Getting the textures to the model is also pretty simple. PIX can do this, but in the interested of showing more tools I will show you how to do it with Intel GPA. Simply right click on the texture and save the texture as a DDS or PNG.

gpa_ak74s_texture-1

At this point, I could simply dump the mesh information and textures and be done with it. If all you care about is ripping a model, this is a way to do it. However, if you want to build modding tools, you have to pretty much fully reverse engineer the 3D file format. In the next tutorial I will show you how to dump the mesh geometry with PIX and import it into Blender.

Conclusion

The code for this work can be found here https://github.com/sbobovyc/GameTools/tree/master/9rota

Discovering Zip password with signature search

Introduction

After playing an interesting RTS game Syrian Warfare, I wanted to take a look at the internals of this game. Curiously, the game files with extension .pak were actually zip archives that contained ZipCrypto encrypted files. The developers of the game stated that in the future modding tools would be released, but I took it as a challenge to figure out the password.

Signature search

I used signsrch to scan the game’s DLLs for ZipCrypto signatures. This tools searches binary files for specific byte patterns, and in case of ZipCrypto the byte patterns are integer constants in the password update function. In one of the DLLs, zfs.dll, the following signatures were found:

C:\Users\sbobovyc\Tools\signsrch>signsrch.exe -e "C:\Steam\steamapps\common\Syri
an Warfare\bin\zfs.dll"

Signsrch 0.2.4
by Luigi Auriemma
e-mail: aluigi@autistici.org
web:    aluigi.org
  optimized search function by Andrew http://www.team5150.com/~andrew/
  disassembler engine by Oleh Yuschuk

- open file "C:\Steam\steamapps\common\Syrian Warfare\bin\zfs.dll"
- 90112 bytes allocated
- load signatures
- open file C:\Users\sbobovyc\Tools\signsrch\signsrch.sig
- 3075 signatures in the database
- start 4 threads
- start signatures scanning:

  offset   num  description [bits.endian.size]
  --------------------------------------------
  100021a6 3052 function where is handled the ZipCrypto password [32.le.12&]
  1000220e 1847 Zip Crypto [32.le.16&]
  1000e328 641  CRC-32-IEEE 802.3 [crc32.0x04c11db7 le rev int_min.1024]
  1000e328 648  CRC-32-IEEE 802.3 [crc32.0xedb88320 lenorev 1.1024]
  100130ca 2545 anti-debug: IsDebuggerPresent [..17]
  100172b0 3032 PADDINGXXPADDING [..16]

- 6 signatures found in the file in 0 seconds
- done

The function where the ZipCrypto password is handled in the DLL looks like this:
zipcrypto_update_keys

Compare with a C++ implementation of the password update function found in ZipCrypto.cpp:

STDMETHODIMP CCipher::CryptoSetPassword(const Byte *password, UInt32 passwordLen)
{
  Keys[0] = 0x12345678;
  Keys[1] = 0x23456789;
  Keys[2] = 0x34567890;
  UInt32 i;
  for (i = 0; i < passwordLen; i++)
    UpdateKeys(password[i]);
  for (i = 0; i < 3; i++)
    Keys2[i] = Keys[i];
  return S_OK;
}

Starting the game with a debugger caused the game to crash. The game executable (SyrianWarfare.exe) resides in bin directory and is normally started by start.cmd batch script. After further investigation, I realized that the game looks for files in the current working directory which of course differs based on whether it’s started by the batch script or by a debugger. Copying all the pak, war, and txt files into /bin fixed the crash problem.

$ tree -L 1
.
├── _CommonRedist
├── basis
├── bin
├── effects.war
├── effectstex.war
├── lands.pak
├── lang.pak
├── langs.pak
├── main.pak
├── models.pak
├── OST
├── sounds.pak
├── start.cmd
├── steam_appid.txt
├── textures.pak
└── workdir.root

Once the game started up inside the debugger, I let it run so that the DLL would be loaded into memory. I then found the address of the function which signsrch found earlier, put a breakpoint on the function entry, restarted the game and let the game execute till the breakpoint was hit.

zip_crypto_password

Examining the values in the registers at the breakpoint, it is clear that register ESI contains the plaintext password needed to decrypt the files in lands.pak. The same password is used to decrypt files in every other .pak file. Having what I wanted, I became curious if I could find the password in game memory. Searching the game memory yielded zero results. I backtraced the execution from the “CryptoSetPassword” function in zfs.dll to a place in SyrianWarfare.exe where the obfuscated password is de-obfuscated. The obfuscated password m,ew0rdk1a;ldsdj is stored in the .rdata section of the executable.

zip_crypto_obfuscated

Looking at the code listing, one can see that in lines 10 and 16, the third and tenth characters are substituted with n and s while at line 22 the fifteenth character is decremented by one resulting in the plaintext password of m,nw0rdk1s;ldscj.

00408A01 | 68 08 8D 41 00           | push syrianwarfare.418D08                                             | 418D08:&"m,ew0rdk1a;ldsdj"
00408A06 | 8D 4C 24 20              | lea ecx,dword ptr ss:[esp+20]                                         |
00408A0A | FF 15 E8 12 41 00        | call dword ptr ds:[<&stlp_std::basic_string<char,stlp_std::char_trait |
00408A10 | 8D 54 24 2C              | lea edx,dword ptr ss:[esp+2C]                                         | [esp+2C]:".pak"
00408A14 | C7 44 24 70 00 00 00 00  | mov dword ptr ss:[esp+70],0                                           |
00408A1C | 8D 44 24 1C              | lea eax,dword ptr ss:[esp+1C]                                         |
00408A20 | 39 54 24 30              | cmp dword ptr ss:[esp+30],edx                                         |
00408A24 | 74 04                    | je syrianwarfare.408A2A                                               |
00408A26 | 8B 44 24 1C              | mov eax,dword ptr ss:[esp+1C]                                         |
00408A2A | C6 40 02 6E              | mov byte ptr ds:[eax+2],6E                                            | 6E:'n'
00408A2E | 8D 44 24 2C              | lea eax,dword ptr ss:[esp+2C]                                         | [esp+2C]:".pak"
00408A32 | 39 44 24 30              | cmp dword ptr ss:[esp+30],eax                                         |
00408A36 | 8D 44 24 1C              | lea eax,dword ptr ss:[esp+1C]                                         |
00408A3A | 74 04                    | je syrianwarfare.408A40                                               |
00408A3C | 8B 44 24 1C              | mov eax,dword ptr ss:[esp+1C]                                         |
00408A40 | C6 40 09 73              | mov byte ptr ds:[eax+9],73                                            | 73:'s'
00408A44 | 8D 4C 24 2C              | lea ecx,dword ptr ss:[esp+2C]                                         | [esp+2C]:".pak"
00408A48 | 8D 44 24 1C              | lea eax,dword ptr ss:[esp+1C]                                         |
00408A4C | 39 4C 24 30              | cmp dword ptr ss:[esp+30],ecx                                         |
00408A50 | 74 04                    | je syrianwarfare.408A56                                               |
00408A52 | 8B 44 24 1C              | mov eax,dword ptr ss:[esp+1C]                                         |
00408A56 | FE 48 0E                 | dec byte ptr ds:[eax+E]                                               |
00408A59 | 6A 24                    | push 24                                                               |
00408A5B | E8 F4 55 00 00           | call <syrianwarfare.operator new>                                     |

Notes

signsrch http://aluigi.altervista.org/mytoolz.htm
Retrieving ZIP passwords from games – the debugger way http://zenhax.com/viewtopic.php?f=4&t=59