Hooking Lua Part 3

Introduction
After learning a bit about how Syrian Warfare handles Lua context, I wanted to figure out how I could execute my own scripts from within the game.

Enabling io
The game does not use the Lua io library, so I modified my injected DLL to have the capability to execute luaopen_io() when F5 key is pressed. I tested the newly enabled io library by executing the following from console:

dump = io.open("testing_io_lua.txt","w")
dump:write("Wrote from lua")
dump:flush()
dump:close()

The file appeared in the game directory with the correct text.

Adding new scripts
I tried to execute dofile() and loadfile() from the command console, but I kept on getting file not found errors. To debug this, I put a breakpoint on lua_load. This led me to the discovery that Galileo loads up scripts from scripts/ and scripts/lua. Looking at strings in SyrianWarefare.exe, I noticed references to “scripts/triggers.lua” and “scripts/global_map.lua.”

global_map.lua uses dofile() to read and execute other lua files:

--************************************************************************
--* Вспомогательные функции и константы                                  *
--************************************************************************
-- utility variables
-- constants
SCRIPTS_PATH = "scripts/lua/"
dofile(SCRIPTS_PATH.."constants.lua")
-- соответствие объекта спауну
dofile(SCRIPTS_PATH.."spawns.lua")
-- соответстиве объекта резервам
dofile(SCRIPTS_PATH.."reserves.lua")
-- вспомогательные функции
dofile(SCRIPTS_PATH.."tools.lua")

I put test.lua into the main.pak zip file and ran the game from Steam. The game started up normally. When I executed the following on console:
dofile(SCRIPTS_PATH..”test.lua”)
the following error printed in DebugView:
[11060] mll::debug::exception: [ml_encrypted_zip: unknown zlib error while inflating]

It appears that the game expects all the files in the zip to be encrypted. Since I know the password used to encrypt the game files, I used 7zip to append the file to main.pak:

$ 7z.exe a main.pak test.lua -pm,nw0rdk1s;ldscj

Conclusion
By appending a Lua script to the game zip files and forcing the game to load Lua io library, I was able to successfully execute test.lua from inside the game. The approach could be used to add and execute custom scripts. The source code for the injected DLL can be found here:
https://github.com/sbobovyc/SyrianWarfare_lua_hacks

Reversing a DirectX Game Part 3

Reproduced From https://sites.google.com/site/sbobovyc/writing/reverse-engineering/reversing-a-directx-game-part-3

DISCLAIMER: The information provided here is for educational purposes only.

Reconnaissance

Looking at calls to CreateFile, this is one of the first files to be accessed. Next the data.pak gets accessed. File mapping object is anonymous.
read_files

“Data\XmlFiles\resources.xml” is:

<?xml version = "1.0" encoding = "UTF-8"?>
<!-- Copyright (c)2004 Lesta Studio -->

<root>
    <!-- Пути для ресурсов. -->
    <object name = "Resources">
        
        <object name = "VFS">
            <object name = "vfs1">
                <string name = "Type" value = "filesystem"/>
                <string name = "Path" value = "Data"/>
                <boolean name = "Recursive" value = "true"/>
            </object>
            

            <object name = "vfsPakFiles">
                <string name = "Type" value = "sma2"/>
                <string name = "Path" value = "data"/>
                <string name = "Mask" value = "*.pak"/>
                <boolean name = "Recursive" value = "true"/>
            </object>

            <object name = "z">
                <string name = "Type" value = "filesystem"/>
                <string name = "Path" value = "Patch"/>
                <boolean name = "Recursive" value = "true"/>
            </object>
            <object name = "z2">
                <string name = "Type" value = "sma2"/>
                <string name = "Path" value = "Patch"/>
                <string name = "Mask" value = "*.pak"/>
                <boolean name = "Recursive" value = "true"/>
            </object>

        </object>
    </object>

        
</root>

The Russian text is “Paths for resources”.

Data.pak gets read here.
Data_Pak
In the last tutorial I showed you how I looked at the 3D models being used by the game using PIX. Now it was time to look at what files the game was using. Games tend to bundle all their files into archives. These are archives tend to be big and 9th Company had one file that stood out: 9th Company\Data\data.pak. This file is 1.32 GB and when opened with a hex editor has a bunch of file names at the top of the file and a bunch of data at the bottom of the file. I searched for DXT (DDS texture) and the hex array “89 50 4E 47 0D 0A 1A 0A” (png) in the pack file and saw many occurring instances. Great! The files are not compressed or encrypted. Aside from games using standard packing formats, it does not get any easier than this.

From this point, it was a matter of experimenting to try to figure out the structure of the header. I looked for the length of the file names, file offset, the total number of files, etc. Once I thought I had a pretty good idea, I wrote a simple loop that followed a pattern to determine how many files are in the header. Once I got to 500 files, i thought to look at the beginning of the file again. Scanning over the first few bytes, integer at offset 0x5 seemed to be the only plausible value. And sure enough, looking at the offset after reading, this is where the file description ended and data began. Also, the integer after number of files was an offset where the first file started.

Pak_Format

The format looked fairly straight forward.

The header was:

struct {
    char type[] ="SMA";
    int8 type_version;
    int16 unknown;
    int32 total_files;
    int32 file_offset;
    int8 file_name_length;
    char file_name[];
}

The main body had the following repeating structure:

struct {
    int32 file_size;
    int32 unknown;
    int32 file_offset;
    int8 file_name_length;
    char file_name[];
}

I had my script print out the file names and offsets:
SMA 2
Number of files: 2904
Data\Art\Buildings\AminPalace\AminPalace.lm, unknown 0x39046, file size 0x85b9d, offset: 0x262c6
Data\Art\Buildings\AminPalace\AminPalace.xml, unknown 0x3f7, file size 0x946, offset: 0x5f30c
Data\Art\Buildings\Army_Buildings\kazarma.lm, unknown 0x12bfd, file size 0x47883, offset: 0x5f703
Data\Art\Buildings\Army_Buildings\kazarma.xml, unknown 0x3afd, file size 0xf832, offset: 0x72300

Using this information, I wrote each file to the disk. Looking at DDS texture file and XML files, it was clear that the files where either ciphered or encrypted in some way. A few of the files were not and the unknown field in those files was zero. At the end of the game manual, it says this:

Zlib, 1995-2002 Jean-loup Gailly & Mark Adler.
Please visit: www.gzip.org/zlib

Looking through the strings in the exe, I found lots of references like these:
007A0B0C=9-Pota.007A0B0C (ASCII “unknown compression method”)
007A0AE0=9-Pota.007A0AE0 (ASCII “incorrect header check”)
007A0AC4=9-Pota.007A0AC4 (ASCII “unknown header flags set”)
007A0AB0=9-Pota.007A0AB0 (ASCII “header crc mismatch”)
deflate 1.2.3 Copyright 1995-2005 Jean-loup Gailly
inflate 1.2.3 Copyright 1995-2005 Mark Adler

They are from zlib inflate.c. Using zlib.decompress() on each dumped file yielded an uncompressed file!

import struct
import os
import sys
import errno
import zlib

class File(object):
    def __init__(self, path, offset, size, unk):
        self.path = path
        self.offset = offset
        self.size = size
        self.unk = unk

    def create(self, file_pointer):
        directory = os.path.dirname(self.path)
        try: os.makedirs(directory)
        except OSError, err:
            # Reraise the error unless it's about an already existing directory 
            if err.errno != errno.EEXIST or not os.path.isdir(directory): 
                raise
        if self.size > 0:
            file_pointer.seek(self.offset)
            data = file_pointer.read(self.size)            
            with open(self.path, "wb") as f:
                # decompress
                if self.unk != 0:
                    data = zlib.decompress(data)
                f.write(data)
    


files = []
with open("data.pak", "rb") as f:
    magic, = struct.unpack("3s", f.read(3))
    version, = struct.unpack("<H", f.read(2))
    print magic, version
    if version != 2:
        print "Wrong version detected!"
        sys.exit()

    num_files, = struct.unpack("<I", f.read(4))
    print "Number of files:", num_files

    unknown, = struct.unpack("<I", f.read(4))

    for i in range(0, num_files):
        count, = struct.unpack("B", f.read(1))
        path, = struct.unpack("%is" % count, f.read(count))
        file_size, = struct.unpack("<I", f.read(4))
        unk1, = struct.unpack("<I", f.read(4))
        offset, = struct.unpack("<I", f.read(4))
        # unk1 is not adler32 or crc32
        print "%s, unknown %s, file size %s, offset: %s" % (path,  hex(unk1), hex(file_size), hex(offset))
        
        files.append(File(path, offset, file_size, unk1))


    for fil in files:
        fil.create(f)

Conclusion
Code for this work can be found herehttps://github.com/sbobovyc/GameTools/tree/master/9rota

References
http://msdn.microsoft.com/en-us/library/windows/desktop/aa366556%28v=vs.85%29.aspx
http://forums.steampowered.com/forums/showthread.php?t=2554597http://www.pacificstorm.net/forum/viewtopic.php?t=2975
http://www.opensource.apple.com/source/zlib/zlib-23.0.1/zlib/inflate.c

Reversing a DirectX Game Part 1

Reproduced from https://sites.google.com/site/sbobovyc/writing/reverse-engineering/reversing-a-directx-game-part-1

DISCLAIMER: The information provided here is for educational purposes only.

Foreword
Over the last year I’ve been learning the art of reverse engineering games for modding purposes. It has been a fun process, but full of trial and error. I want to share my experiences here. The goal of this tutorial is to take you through the steps of reverse engineering game formats. Why is this useful? Most games these days don’t support modding. For example, Jagged Alliance: Back in Action did not officially support modding, but I liked the game so much that I wanted to change that. The result was the JABIA Tools project, which included a myriad of tools including importer and exporter plugins for Blender.

Instead of showing you what I’ve done before, I decided to start a new reversing project for the purposes of this tutorial. I choose an obscure game, 9th Company: Roots of Terror. It is based on a Russian movie set during the Soviet-Afghan war. Why did I choose this game? I bought it, played the first few missions and uninstalled. This is a chance to recoup the $10 I spent. In addition, it has some nice low poly models.

This tutorial is aimed at intermediate users. I will not be explaining hexadecimal number representation, DirectX 9 or how 3D graphics work. These topics are very large and covered much better in other places. I am to help you build on top of your knowledge to learn how to do something that is not taught in a computer science curriculum. If you are not a technical person, but you enjoy video games and are interested in how they work, I encourage you to continue reading.

Reconnaissance
9th Company is developed by a Russian studio, Lesta Studio and uses the AdicoEngine2. Not much can be found online about this engine, but apparently there are some Russian modding tools. I spent a few moments trying to find them, but only found a few screenshots.

Graphics debugging
It was time to start up the tools and do some investigating. One of my favorite tools for this task is DirectX PIX. It comes with the DirectX 9 SDK and is used for profiling/debugging. It is also great for reversing.
I started the game under PIX and did a single frame capture during the tutorial mission. Here you can see the the rendered frame.

pix_frame_render-1

With PIX, you can debug an individual pixel. This will show you all the draw calls to that pixel. However, this game uses multiple render targets, so debugging a pixel will only show the calls for the last render target. I simply picked a draw call in the middle of the trace. From there, I debugged a pixel that was part of the AKS-74N. This led me to the draw call for this mesh. It is interesting to note that the engine uses a single draw call to draw all the instances of this mesh. You can tell that this is happening because there are multiple copies of the rifle in the post-vertex output. PIX tells us a treasure trove of information: faces, vertices, vertex positions, surface normals and UV texture coordinates. This information could be exported to a CSV file and turned into a 3D model
pix_ak74s-1

A few function calls before the draw, the vertex declaration is set. PIX prints it out really nice for us. This is an important piece of information that gives us a hint of how the game’s custom 3D file format is structured.
pix_vertex_declaration-1

Getting the textures to the model is also pretty simple. PIX can do this, but in the interested of showing more tools I will show you how to do it with Intel GPA. Simply right click on the texture and save the texture as a DDS or PNG.

gpa_ak74s_texture-1

At this point, I could simply dump the mesh information and textures and be done with it. If all you care about is ripping a model, this is a way to do it. However, if you want to build modding tools, you have to pretty much fully reverse engineer the 3D file format. In the next tutorial I will show you how to dump the mesh geometry with PIX and import it into Blender.

Conclusion

The code for this work can be found here https://github.com/sbobovyc/GameTools/tree/master/9rota

Reversing 7.62 High Calibre AZP and localization files

Introduction
Going through my Steam backlog of games, I saw 7.62 High Calibre and decided to give it another try.  Looking around, I noticed that this game had a very active modding community, but that the available modding tools were closed source.  I wanted to create an open source tool for unpacking/packing AZP files used by 7.62 High Calibre.  I was successful in accomplishing that and more.  The results of this work can be found here https://github.com/sbobovyc/GameTools/tree/master/762_HighCalibre

Reconnaissance
The initial step of searching for signatures in game executable revealed that zlib is likely statically linked.

$ /c/Users/sbobovyc/Tools/signsrch/signsrch.exe /c/Steam/steamapps/common/7.62/E6.exe

Signsrch 0.2.4
by Luigi Auriemma
e-mail: aluigi@autistici.org
web: aluigi.org
optimized search function by Andrew http://www.team5150.com/~andrew/
disassembler engine by Oleh Yuschuk

- done
- open file "C:/Steam/steamapps/common/7.62/E6.exe"
- 3547136 bytes allocated
- load signatures
- open file C:\Users\sbobovyc\Tools\signsrch\signsrch.sig
- 3075 signatures in the database
- start 4 threads
- start signatures scanning:

offset num description [bits.endian.size]
--------------------------------------------
00017aee 3048 DMC compression [32.le.16&]
002be682 1299 classical random incrementer 0x343FD 0x269EC3 [32.le.8&]
00314e90 2289 zinflate_lengthStarts [16.le.58]
00314f10 2296 zinflate_distanceStarts [16.le.60]
00314f90 641 CRC-32-IEEE 802.3 [crc32.0x04c11db7 le rev int_min.1024]
00314f90 648 CRC-32-IEEE 802.3 [crc32.0xedb88320 lenorev 1.1024]
00315390 129 Adler CRC32 (0x191b3141) [32.le.1024]
00315790 131 Adler CRC32 (0x01c26a37) [32.le.1024]
00315b90 133 Adler CRC32 (0xb8bc6765) [32.le.1024]
00315f90 645 CRC-32-IEEE 802.3 [crc32.0x04c11db7 be rev int_min.1024]
00315f90 652 CRC-32-IEEE 802.3 [crc32.0xedb88320 benorev 1.1024]
00316390 130 Adler CRC32 (0x191b3141) [32.be.1024]
00316790 132 Adler CRC32 (0x01c26a37) [32.be.1024]
00316b90 134 Adler CRC32 (0xb8bc6765) [32.be.1024]
00316f90 2294 zinflate_lengthExtraBits [32.le.116]
00317005 2304 zinflate_distanceExtraBits [32.be.120]
00317008 2303 zinflate_distanceExtraBits [32.le.120]
003175d8 1086 Zlib dist_code [..512]
003177d8 1087 Zlib length_code [..256]
003178d8 1089 Zlib base_length [32.le.116]
00317950 1091 Zlib base_dist [32.le.120]
0031f460 639 CRC-32-IEEE 802.3 [crc32.0x04c11db7 lenorev int_min.1024]
0031f460 650 CRC-32-IEEE 802.3 [crc32.0xedb88320 le rev 1.1024]
0031f868 3038 unlzx table_three [32.le.64]
0031f868 1605 Generic bitmask table [32.le.128]
0031f86c 2588 bitmask [32.le.128]
0031f8dc 3051 compression algorithm seen in the game DreamKiller [32.be.12&]
0031f8df 3050 compression algorithm seen in the game DreamKiller [32.le.12&]
0031fb28 2876 libavcodec ff_mjpeg_val_ac_chrominance [..162]
0031fbf8 2875 libavcodec ff_mjpeg_val_ac_luminance [..162]
00321298 1115 Jpeg dct 14 bit aanscales [16.le.128]
00321348 1119 Jpeg dct AA&N scale factor [double.le.64]
00326753 2417 MBC2 [32.le.248&]
00336030 1933 Vorbis FLOOR1_fromdB_LOOKUP [float.le.1024]
$ strings /c/Steam/steamapps/common/7.62/E6.exe | grep flate
deflate 1.2.3 Copyright 1995-2005 Jean-loup Gailly
inflate 1.2.3 Copyright 1995-2005 Mark Adler

It’s very common for games to use zlib inflate() to save disk space.  Deflated (aka compressed) data looks like random array of bytes, which is how the AZP archives appear to look.

Here are the first few bytes of HWM.azp:
azp_data

First three bytes of the file correspond to the string “AZP.” Interestingly enough, file names and directory names do not appear inside AZP files. Either the table of contents exists somewhere else or it’s obfuscated.

In order to start following the data access pattern, I put a breakpoint on CreateFileA. The first hit that created an AZP file occured here:

006CB5AB | push eax                
006CB5AC | push dword ptr ss:[ebp-8]
006CB5AF | push dword ptr ss:[ebp-10]
006CB5B2 | push dword ptr ss:[ebp+10]         // [ebp+10]:"HWM.azp"
006CB5B5 | call dword ptr ds:[<&CreateFileA>]

I chose to use HWM.azp as the candidate for reverse engineering. This file is read by ReadFile at this instruction:

006C6D2B | push edx
006C6D2C | push dword ptr ds:[eax+esi]
006C6D2F | call dword ptr ds:[<&ReadFile>]

The call stack before the call to ReadFile:
1: [esp] 00000104       // file handle
2: [esp+4] 032D1530  // pointer to destination buffer
3: [esp+8] 00001000  // number of bytes to read
4: [esp+C] 0018F060
5: [esp+10] 00000000

After ReadFile executed, I put a hardware access breakpoint on address 0x32D1530, which is ‘A’ in the buffer. This breakpoint triggered by this instruction:

006C6C97 | movzx eax,byte ptr ds:[ecx]  // ecx is 032D1530
006C6C9A | inc ecx
006C6C9B | mov dword ptr ds:[esi],ecx   // esi is a pointer to some data structure
006C6C9D | pop esi
006C6C9E | ret

After tracing through the code, I figured out that this code copies the first four bytes, “AZP” plus 0x01, to some buffer:

006C4D84 | mov al,byte ptr ds:[esi]
006C4D86 | mov byte ptr ds:[edi],al
006C4D88 | mov al,byte ptr ds:[esi+1]
006C4D8B | mov byte ptr ds:[edi+1],al
006C4D8E | mov al,byte ptr ds:[esi+2]
006C4D91 | mov byte ptr ds:[edi+2],al
006C4D94 | mov eax,dword ptr ss:[ebp+8]
006C4D97 | pop esi
006C4D98 | pop edi
006C4D99 | leave
006C4D9A | ret

I then put a hardware access breakpoint on the next two bytes in the buffer, 0x42 0xED. This breakpoint was triggered here:

006C4D0C | mov dword ptr ds:[edi+ecx*4-18],eax
006C4D10 | mov eax,dword ptr ds:[esi+ecx*4-14]
006C4D14 | mov dword ptr ds:[edi+ecx*4-14],eax
006C4D18 | mov eax,dword ptr ds:[esi+ecx*4-10]
006C4D1C | mov dword ptr ds:[edi+ecx*4-10],eax | edi+ecx*4-10:",Aq"
006C4D20 | mov eax,dword ptr ds:[esi+ecx*4-C] 
006C4D24 | mov dword ptr ds:[edi+ecx*4-C],eax  | [edi+ecx*4-C]:"HWM.azp"
006C4D28 | mov eax,dword ptr ds:[esi+ecx*4-8]
006C4D2C | mov dword ptr ds:[edi+ecx*4-8],eax
006C4D30 | mov eax,dword ptr ds:[esi+ecx*4-4]
006C4D34 | mov dword ptr ds:[edi+ecx*4-4],eax  // copy 0x42 0xED here
006C4D38 | lea eax,dword ptr ds:[ecx*4]

The same code is hit when the next four bytes, 0x6 0x00 0x00 0x00, are accessed. After I put a hardware breakpoint on 0x3C 0x05 0x00 0x00, this instruction was hit:

006C4C33 | rep movsd dword ptr es:[edi],dword ptr ds:[esi]
006C4C35 | jmp dword ptr ds:[edx*4+6C4D4C]

Next, another ReadFile happened in which 0xD000 bytes are read:

006C6D2C | push dword ptr ds:[eax+esi]
006C6D2F | call dword ptr ds:[<&ReadFile>]

The call stack:
1: [esp] 00000104        // file handle
2: [esp+4] 032D3528   // pointer to destination buffer
3: [esp+8] 0000D000  // number of bytes to read
4: [esp+C] 0018F074
5: [esp+10] 00000000

At this point I decided to switch gears and take a look at the community provided tool that is used to unpack/pack AZP files.

Reversing azp.exe
In the process of reverse engineering a cipher, it is useful to have access to the plain text version. The community tool is able to provide a listing of files in an AZP archive:

$ ./azp.exe l HWM.azp | head

7.62 resource archiver  (c) 2007 by Novik  v 1.3

     41564/41580      100.04% LOCAL.TXT
      3910/1804        46.14% TIPS.TXT
     53682/8514        15.86% ACTORS\ITEMS\ACOG-11.ACT
       163/83          50.92% ACTORS\ITEMS\ACOG-11.ACT.INF
     62839/11530       18.35% ACTORS\ITEMS\ACOG-11WR.ACT
       163/82          50.31% ACTORS\ITEMS\ACOG-11WR.ACT.INF
     46710/7363        15.76% ACTORS\ITEMS\AEK919K_SCOPE.ACT

Listing of files in HWM.azp yields the count of 1340 files.  This number can be found at file offset 0xC.

Put breakpoints on open() and read():
azp file is opened here

00404D7F | call dword ptr ds:[<&open>]
00404D85 | add esp,C

The first read from the file occurs here:

00404C81 | call dword ptr ds:[<&read>]
00404C87 | add esp,C

The call stack:
1: [esp] 00000003       // file handle
2: [esp+4] 01B80048  // pointer to destination buffer
3: [esp+8] 00010000  // number of bytes to read
4: [esp+C] 7C36C01B
5: [esp+10] 0018FD74

After the first 0x10000 bytes are read from the file, I put a hardware breakpoint on the ‘A’ in the buffer. The following instruction was hit:

7C342FF4 | mov dword ptr ds:[edi+ecx*4-4],eax

Looking at the callstack, this instruction is part of a call chain of memcopy():
1: [esp] 0018FC58 // pointer to destination buffer
2: [esp+4] 01B80048 // pointer to source buffer
3: [esp+8] 00000004 // number of bytes to copy
4: [esp+C] 7C36C01B
5: [esp+10] 0018FD74

Setting a breakpoint on this call to memcopy, I was able to see the pattern. First, the magic identifier “AZP1” is read, then unknown 4 bytes, then another unknown 4 bytes, then total number of files, then a uint32 which is the length of file name, then nine bytes which are the ciphered version of “LOCAL.TXT”. Both the length of the file name and the file name itself are ciphered, therefore the next step is to figure out the cipher. I kept track of where the bytes 0xFC 0x38 0xB8 0xE8 were copied and put a hardware breakpoint on 0xFC. The following instruction was hit:

00402DF0 | push ebp
....
00402E67 | mov al,byte ptr ds:[edi] // esi is a pointer to a copy of 0xFC 0x38 0xB8 0xE8
00402E69 | xor edx,eax
00402E6B | mov byte ptr ds:[edi],dl // dl has decoded byte 9
00402E6D | inc edi

Watching this code execute, it became clear that this is where the bytes were deciphered.

00402E3C | mov eax,dword ptr ds:[esi+30]                       // [esi+30] is an unknown value, possibly cipher key
00402E3F | mov dword ptr ss:[ebp+C],eax                        |
00402E42 | push edi                                            |
00402E43 | push ecx                                            |
00402E44 | push edx                                            |
00402E45 | push eax                                            |
00402E46 | mov edi,dword ptr ss:[ebp+8]                        // edi is a pointer to ciphered data
00402E49 | mov ecx,dword ptr ss:[ebp-4]                        // ecx is a counter variable
00402E4C | mov eax,dword ptr ss:[ebp+C]                        // eax is probably contains cipher key
00402E4F | cld                                                 |
00402E50 | push eax                                            |
00402E51 | xor eax,eax                                         |
00402E53 | xor edx,edx                                         |
00402E55 | pop dx                                              |
00402E57 | pop ax                                              |
00402E59 | mul dx                                              |
00402E5C | dec ax                                              |
00402E5E | xor eax,edx                                         |
00402E60 | mov edx,FF                                          |
00402E65 | and edx,eax                                         |
00402E67 | mov al,byte ptr ds:[edi]                            |
00402E69 | xor edx,eax                                         |
00402E6B | mov byte ptr ds:[edi],dl                            // dl has clear text byte
00402E6D | inc edi                                             |
00402E6E | loop azp.402E59                                     |

I searched for the constant 0xF69DA025 in the azp.exe address space, and found two locations:
0040304D mov dword ptr ss:[esp+40],F69DA025
004033AE push F69DA025

The second result is hit when AZP file is deciphered:

004033AE | push F69DA025                // hard coded key!
004033B3 | push esi   
004033B4 | lea ecx,dword ptr ss:[esp+24]
004033B8 | call azp.402D80

The cipher algorithm lies between instruction address 00402E51 and 00402E6B. After porting this cipher algorithm to python, I was able to decipher the length of the first file. Watching the execution of this cipher for subsequent data, it became clear that in addition to deciphered data this function also stored a key that should be used to decipher the next data.

00402E70 | shl eax,10
00402E73 | or ax,dx
00402E76 | mov dword ptr ss:[ebp+C],eax     // after next key is generated, store it

With this information, I was able to decipher and parse the header.  The structure of the entries looks like this:

struct {
    uint32 name_length;
    char file_name[];
    uint32 offset;
    uint32 compressed_size;
    uint32 uncompressed_size;
}

With the header parsed, it was time to decompress the data.  From strings found in the game and azp.exe, I knew that zlib was compiled in the executable. Also, I was pretty sure that sub_40D750 is zlib inflate based on strings found in the body of the function:

0040D7D2 | mov dword ptr ds:[eax],D                            | D:'\r'
0040D7D8 | mov dword ptr ds:[esi+18],azp.414BB4                | 414BB4:"unknown compression method"
0040D7DF | jmp azp.40D9BB                                      |
0040D7E4 | mov ecx,dword ptr ds:[eax+4]                        |
0040D7E7 | mov edx,dword ptr ds:[eax+10]                       |
0040D7EA | shr ecx,4                                           |
0040D7ED | add ecx,8                                           |
0040D7F0 | cmp ecx,edx                                         |
0040D7F2 | jbe azp.40D806                                      |
0040D7F4 | mov dword ptr ds:[eax],D                            | D:'\r'
0040D7FA | mov dword ptr ds:[esi+18],azp.414BA0                | 414BA0:"invalid window size"

This function was hit when azp.exe extracted files which indicates that indeed, zlib inflate is used.  I dumped the first file from the AZP archive and the first two bytes where 0x78 0x9C which correspond to a zlib header.  After this, it was a matter of reading the compressed data from the AZP archive, zlib inflating it, and saving the decompressed data to the appropriately named file. Since the cipher is symmetric, the same secret key is used to cipher the table of contents, so I was able to add packing functionality to the tool I developed.

E5DEC
HWM.azp contained two interesting files: LOCAL.txt and TIPS.txt. Using ed5dec.exe, I was able to decipher LOCAL.txt into a human readable file. Following a similar process as reverse engineering azp.exe, I found that these text files are ciphered using the exact same algorithm as the AZP table of contents. It was a simple matter of reading the whole file and deciphering it with the function I developed for azp files.

00401260 | push ebp                                            |
00401261 | mov ebp,esp                                         |
00401263 | push edi                                            |
00401264 | push edi                                            |
00401265 | push ecx                                            |
00401266 | push edx                                            |
00401267 | push eax                                            |
00401268 | mov edi,dword ptr ss:[ebp+8]                        |
0040126B | mov ecx,dword ptr ss:[ebp+C]                        |
0040126E | mov eax,dword ptr ss:[ebp+10]                       |
00401271 | cld                                                 |
00401272 | push eax                                            |
00401273 | xor eax,eax                                         |
00401275 | xor edx,edx                                         |
00401277 | pop dx                                              |
00401279 | pop ax                                              |
0040127B | mul dx                                              |
0040127E | dec ax                                              |
00401280 | xor eax,edx                                         |
00401282 | mov edx,FF                                          |
00401287 | and edx,eax                                         |
00401289 | mov al,byte ptr ds:[edi]                            |
0040128B | xor edx,eax                                         |
0040128D | mov byte ptr ds:[edi],dl             // dl contain deciphered byte
0040128F | inc edi                                             |
00401290 | loop e5dec.40127B                                   |
00401292 | shl eax,10                                          |
00401295 | or ax,dx                                            |
00401298 | mov dword ptr ss:[ebp+10],eax                       |
0040129B | pop eax                                             |
0040129C | pop edx                                             |
0040129D | pop ecx                                             |
0040129E | pop edi                                             |
0040129F | mov eax,dword ptr ss:[ebp+10]                       |
004012A2 | pop edi                                             |
004012A3 | pop ebp                                             |
004012A4 | ret                                                 |

Notes

http://unix.superglobalmegacorp.com/xnu/newsrc/bsd/net/zlib.h.htmlhttps://stackoverflow.com/questions/9050260/what-does-a-zlib-header-look-like#17176881
https://tools.ietf.org/html/rfc1950
http://compgroups.net/comp.compression/what-is-this-compression-format/185138

Discovering Zip password with signature search

Introduction

After playing an interesting RTS game Syrian Warfare, I wanted to take a look at the internals of this game. Curiously, the game files with extension .pak were actually zip archives that contained ZipCrypto encrypted files. The developers of the game stated that in the future modding tools would be released, but I took it as a challenge to figure out the password.

Signature search

I used signsrch to scan the game’s DLLs for ZipCrypto signatures. This tools searches binary files for specific byte patterns, and in case of ZipCrypto the byte patterns are integer constants in the password update function. In one of the DLLs, zfs.dll, the following signatures were found:

C:\Users\sbobovyc\Tools\signsrch>signsrch.exe -e "C:\Steam\steamapps\common\Syri
an Warfare\bin\zfs.dll"

Signsrch 0.2.4
by Luigi Auriemma
e-mail: aluigi@autistici.org
web:    aluigi.org
  optimized search function by Andrew http://www.team5150.com/~andrew/
  disassembler engine by Oleh Yuschuk

- open file "C:\Steam\steamapps\common\Syrian Warfare\bin\zfs.dll"
- 90112 bytes allocated
- load signatures
- open file C:\Users\sbobovyc\Tools\signsrch\signsrch.sig
- 3075 signatures in the database
- start 4 threads
- start signatures scanning:

  offset   num  description [bits.endian.size]
  --------------------------------------------
  100021a6 3052 function where is handled the ZipCrypto password [32.le.12&]
  1000220e 1847 Zip Crypto [32.le.16&]
  1000e328 641  CRC-32-IEEE 802.3 [crc32.0x04c11db7 le rev int_min.1024]
  1000e328 648  CRC-32-IEEE 802.3 [crc32.0xedb88320 lenorev 1.1024]
  100130ca 2545 anti-debug: IsDebuggerPresent [..17]
  100172b0 3032 PADDINGXXPADDING [..16]

- 6 signatures found in the file in 0 seconds
- done

The function where the ZipCrypto password is handled in the DLL looks like this:
zipcrypto_update_keys

Compare with a C++ implementation of the password update function found in ZipCrypto.cpp:

STDMETHODIMP CCipher::CryptoSetPassword(const Byte *password, UInt32 passwordLen)
{
  Keys[0] = 0x12345678;
  Keys[1] = 0x23456789;
  Keys[2] = 0x34567890;
  UInt32 i;
  for (i = 0; i < passwordLen; i++)
    UpdateKeys(password[i]);
  for (i = 0; i < 3; i++)
    Keys2[i] = Keys[i];
  return S_OK;
}

Starting the game with a debugger caused the game to crash. The game executable (SyrianWarfare.exe) resides in bin directory and is normally started by start.cmd batch script. After further investigation, I realized that the game looks for files in the current working directory which of course differs based on whether it’s started by the batch script or by a debugger. Copying all the pak, war, and txt files into /bin fixed the crash problem.

$ tree -L 1
.
├── _CommonRedist
├── basis
├── bin
├── effects.war
├── effectstex.war
├── lands.pak
├── lang.pak
├── langs.pak
├── main.pak
├── models.pak
├── OST
├── sounds.pak
├── start.cmd
├── steam_appid.txt
├── textures.pak
└── workdir.root

Once the game started up inside the debugger, I let it run so that the DLL would be loaded into memory. I then found the address of the function which signsrch found earlier, put a breakpoint on the function entry, restarted the game and let the game execute till the breakpoint was hit.

zip_crypto_password

Examining the values in the registers at the breakpoint, it is clear that register ESI contains the plaintext password needed to decrypt the files in lands.pak. The same password is used to decrypt files in every other .pak file. Having what I wanted, I became curious if I could find the password in game memory. Searching the game memory yielded zero results. I backtraced the execution from the “CryptoSetPassword” function in zfs.dll to a place in SyrianWarfare.exe where the obfuscated password is de-obfuscated. The obfuscated password m,ew0rdk1a;ldsdj is stored in the .rdata section of the executable.

zip_crypto_obfuscated

Looking at the code listing, one can see that in lines 10 and 16, the third and tenth characters are substituted with n and s while at line 22 the fifteenth character is decremented by one resulting in the plaintext password of m,nw0rdk1s;ldscj.

00408A01 | 68 08 8D 41 00           | push syrianwarfare.418D08                                             | 418D08:&"m,ew0rdk1a;ldsdj"
00408A06 | 8D 4C 24 20              | lea ecx,dword ptr ss:[esp+20]                                         |
00408A0A | FF 15 E8 12 41 00        | call dword ptr ds:[<&stlp_std::basic_string<char,stlp_std::char_trait |
00408A10 | 8D 54 24 2C              | lea edx,dword ptr ss:[esp+2C]                                         | [esp+2C]:".pak"
00408A14 | C7 44 24 70 00 00 00 00  | mov dword ptr ss:[esp+70],0                                           |
00408A1C | 8D 44 24 1C              | lea eax,dword ptr ss:[esp+1C]                                         |
00408A20 | 39 54 24 30              | cmp dword ptr ss:[esp+30],edx                                         |
00408A24 | 74 04                    | je syrianwarfare.408A2A                                               |
00408A26 | 8B 44 24 1C              | mov eax,dword ptr ss:[esp+1C]                                         |
00408A2A | C6 40 02 6E              | mov byte ptr ds:[eax+2],6E                                            | 6E:'n'
00408A2E | 8D 44 24 2C              | lea eax,dword ptr ss:[esp+2C]                                         | [esp+2C]:".pak"
00408A32 | 39 44 24 30              | cmp dword ptr ss:[esp+30],eax                                         |
00408A36 | 8D 44 24 1C              | lea eax,dword ptr ss:[esp+1C]                                         |
00408A3A | 74 04                    | je syrianwarfare.408A40                                               |
00408A3C | 8B 44 24 1C              | mov eax,dword ptr ss:[esp+1C]                                         |
00408A40 | C6 40 09 73              | mov byte ptr ds:[eax+9],73                                            | 73:'s'
00408A44 | 8D 4C 24 2C              | lea ecx,dword ptr ss:[esp+2C]                                         | [esp+2C]:".pak"
00408A48 | 8D 44 24 1C              | lea eax,dword ptr ss:[esp+1C]                                         |
00408A4C | 39 4C 24 30              | cmp dword ptr ss:[esp+30],ecx                                         |
00408A50 | 74 04                    | je syrianwarfare.408A56                                               |
00408A52 | 8B 44 24 1C              | mov eax,dword ptr ss:[esp+1C]                                         |
00408A56 | FE 48 0E                 | dec byte ptr ds:[eax+E]                                               |
00408A59 | 6A 24                    | push 24                                                               |
00408A5B | E8 F4 55 00 00           | call <syrianwarfare.operator new>                                     |

Notes

signsrch http://aluigi.altervista.org/mytoolz.htm
Retrieving ZIP passwords from games – the debugger way http://zenhax.com/viewtopic.php?f=4&t=59