Hooking Lua Part 1

Reproduced from https://sites.google.com/site/sbobovyc/writing/reverse-engineering/hooking-lua-part-1


Lua is a widely used scripting language in games. When it comes to modding/hacking games, getting access to the scripting system of a game is a huge win. I decided to learn Lua and set up a few experiments in order to learn how to hook into it. The following is a write up of my adventures in Lua.

Compiling, linking, hooking

There are a few different ways to use Lua in a game. Most games are written in C++, and the Lua interpreter is called from inside native code. A game can also register functions with Lua, and after registration these native functions can be used from inside Lua scripts. My little toy application registers two native functions and executes some Lua scripts. The injectable DLL calls a Lua script that dumps the Lua global variables to a file. The globals will contain all the functions that can be called, and include non-standard Lua function that the game registered.

I compiled Lua 5.2.3 with mingw g++ 4.8.1:
$ make mingw
This generated src/liblua.a and src/lua52.dll.

My application was linked to the static Lua library. The application registered two internal functions that simply take a Lua state as a parameter and return some integer values. The application also prints out the address of the Lua state, various Lua C API functions, and the two registered functions:

This is the address of lua state: 0x009123B8
This is the address of luaL_newstate: 0x00405C40
This is the address of luaL_loadstring: 0x00405650
This is the address of luaL_loadfilex: 0x004053E0
This is the address of luaL_loadbufferx: 0x00405600
This is the address of lua_pcallk: 0x00403000
[Exe] This is a dostring print
[Lua] internal_function_1: function: 004016C0
[Lua] internal_function_2: function: 00401790
[Exe] Called internal_function_1
[Lua] got 10
[Exe] Called internal_function_2

After the application is started, the DLL is injected and its DllMain function is called. The DLL uses the DllMain to start a new thread and hook the Lua state that is created in the main application. Most of the Lua C API function require a pointer to a Lua state, so the DLL needs to know its address in memory. No special memory allocator was used, so Lua uses realloc is used to allocate the state on the heap and the address of the state is not known ahead of time. To make things easier, I had the application write the address to a file and then the DLL reads the address from the file. Since the DLL is statically linked to Lua, it will use it’s own Lua functions instead of the application’s functions. If the application and the DLL were dynamically linked to Lua, when the DLL tried to access the Lua state it would cause a “multiple Lua VMs detected” error. The DLL executes a Lua script that dumps _G, the global Lua variables to a log file named state_dump.txt. This variable is a dictionary, and it contains all the functions that could be called from Lua. The addresses of the two internal functions can be seen in the dump:

internal_function_2 = function: 00401790
internal_function_1 = function: 004016C0

A game would contain many registered functions, and their names and addresses could be used to reverse engineer some interesting parts of the game.

Debugging symbols where included which makes finding Lua function in the application easy.
This is the location of luaL_newstate. Notice that the function luaL_checkversion_ is right below it:

00405C40 >/$ 53             PUSH EBX
00405C41  |. 83EC 18        SUB ESP,18
00405C44  |. C74424 04 0000>MOV DWORD PTR SS:[ESP+4],0
00405C4C  |. C70424 D042400>MOV DWORD PTR SS:[ESP],lua_simp.004042D0
00405C53  |. E8 C8DDFFFF    CALL lua_simp.lua_newstate
00405C58  |. 85C0           TEST EAX,EAX
00405C5A  |. 89C3           MOV EBX,EAX
00405C5C  |. 74 10          JE SHORT lua_simp.00405C6E
00405C5E  |. C74424 04 8042>MOV DWORD PTR SS:[ESP+4],lua_simp.00404280
00405C66  |. 890424         MOV DWORD PTR SS:[ESP],EAX
00405C69  |. E8 52BFFFFF    CALL lua_simp.lua_atpanic
00405C6E  |> 83C4 18        ADD ESP,18
00405C71  |. 89D8           MOV EAX,EBX
00405C73  |. 5B             POP EBX
00405C74  \. C3             RETN
00405C75     8D7426 00      LEA ESI,DWORD PTR DS:[ESI]
00405C79     8D             DB 8D
00405C7A     BC             DB BC
00405C7B     27             DB 27                                                  ;  CHAR '''
00405C7C     00             DB 00
00405C7D     00             DB 00
00405C7E     00             DB 00
00405C7F     00             DB 00
00405C80 > $ 56             PUSH ESI
00405C81   . 53             PUSH EBX
00405C82   . 83EC 24        SUB ESP,24
00405C85   . 8B5C24 30      MOV EBX,DWORD PTR SS:[ESP+30]
00405C89   . DD4424 34      FLD QWORD PTR SS:[ESP+34]
00405C8D   . DD5C24 18      FSTP QWORD PTR SS:[ESP+18]
00405C91   . 891C24         MOV DWORD PTR SS:[ESP],EBX
00405C94   . E8 47BFFFFF    CALL lua_simp.lua_version
00405C99   . C70424 0000000>MOV DWORD PTR SS:[ESP],0
00405CA0   . 89C6           MOV ESI,EAX
00405CA2   . E8 39BFFFFF    CALL lua_simp.lua_version
00405CA7   . 39F0           CMP EAX,ESI
00405CA9   . 74 75          JE SHORT lua_simp.00405D20
00405CAB   . C74424 04 3636>MOV DWORD PTR SS:[ESP+4],lua_simp.00423636             ;  ASCII "multiple Lua VMs detected"
00405CB3   . 891C24         MOV DWORD PTR SS:[ESP],EBX
00405CB6   . E8 75EAFFFF    CALL lua_simp.luaL_error

This is the source code of the two functions:

LUALIB_API lua_State *luaL_newstate (void) {
  lua_State *L = lua_newstate(l_alloc, NULL);
  if (L) lua_atpanic(L, &panic);
  return L;

LUALIB_API void luaL_checkversion_ (lua_State *L, lua_Number ver) {
  const lua_Number *v = lua_version(L);
  if (v != lua_version(NULL))
    luaL_error(L, "multiple Lua VMs detected");
  else if (*v != ver)
    luaL_error(L, "version mismatch: app. needs %f, Lua core provides %f",
                  ver, *v);

Depending on how Lua is compiled into a game, this could be used to find and hook the luaL_newstate function and get the pointer to any Lua states that are created.

Source for these experiments can be found here https://github.com/sbobovyc/WinHookInject/tree/master/lua_experiments



7 thoughts on “Hooking Lua Part 1”

  1. Hi Stan,

    This is very interesting stuff. I have a quick, related question for you.

    A friend and I have been working on a similar project for a game we are interested in manipulating its Lua engine. We have successfully hooked the lua_State of a game client using lua_gettop and like in your second part of this tutorial we have two states, both with the same _G environment that update and persist across states.

    We are curious if it is possible to determine the address of the C++ function that is registered to the lua_State somehow? We have checked your WinHookInject project on Github and used it as a reference (thank you for sharing that). While we did have to make a few changes due to the version of Lua we are using, unless we implemented it incorrectly, it appears like we only determined the address of the function in Lua.

    Would you be able to provide some guidance if you have some free time? Thank you very much in advance!

    1. I am very close to releasing a 3rd post about Lua hooking with accompanying source code. How about this, I will try to clean up the code and publish it on github before this weekend.

      1. That sounds great; no rush though!

        We are really interested in what you know. The game client we are working with has introduced some obstacles that we are hoping you have run into so I will keep an eye on your blog/Github!

        If by chance we still have some questions that aren’t addressed by your findings, are you cool with us picking your brain a little bit?

        Thanks for the quick reply also! Much appreciated.

          1. It looks like we will have some more questions for you when you have time. I’d be happy to take this conversation somewhere else besides spamming your comments section if you like; just let me know. I will try to keep this as brief as possible…

            For comparison, the game client we are reversing has a static Lua lib embedded in the client’s executable. We have also embedded the lib in our hook.

            Overall, it appears like we have actually followed almost the exact same procedure as you have to dump the global environment and we actually did dump the custom functions found in the global table (the ones defined by the game developer and not found in standard Lua source) addresses the same way. However, where we are possibly lost is when we attempt to look at the addresses in memory, it appears to be a block of pointers and random values and not the C/C++ function that we want to hook.

            One of our first questions is whether or not the addresses that you were able to print were actually to the C/C++ function registered to the Lua function? In theory, is this possible? If, in your dump, they are in fact the C/C++ function addresses registered to the lua_State, did you try to hook them directly in your DLL? The underlying reason is that we have found a Lua function that sends chat to the client and we want to be able to hook and intercept any commands entered into the chatlog.

            Another major topic of interest for us at the moment is whether you had any issue with locking/multithreading? We believe we need to lock because the game client is modifying the stack at the same time as our own DLL thread, so whenever we push/pop values, the game does too and when we try to grab the value on the stack, we can get unexpected values returned. In addition, much of the game client we are reversing is driven by Lua. This includes everything from the UI to game content, which results in the client freezing until the lock is released. Occasionally the locks will deadlock and the thread just stops so we do have some more research to do on this end. Did you experience any of this? Essentially we are exploring locking both the game client’s Lua state and our own to prevent any overlap. We have tried to use lua_newthread, however when we try to use the new state created by this call, all of the functions in the original lua_State’s global environment start to return nil for some reason.

            Thanks for entertaining these questions! It is very much appreciated Stan!

          2. In order to call C functions from Lua, one must use lua_register(). The third parameter is the function’s address. Check out how Lua state and parameters are handled http://pgl.yoyo.org/luai/i/lua_CFunction
            As to why you are not seeing the function body at the addresses you extract from the state, I do not know. Perhaps it’s a jump table that Lua references. I have not taken a close look at the function I discovered in Syrian Warfare. Instead of hooking the function you want in native code, you could do it at the Lua level. Is it possible for you to replace the game scripts with your own? I have not implemented any locking because I am at an early stage in the process and it has caused some instability. It looks like you guys are farther ahead in the process than me, but it’s good that we can compare notes.

  2. I can’t believe neither of us thought about that earlier. We will just launch the process in suspended state to give us time to hook lua_register and dump the pointer of the C++ functions that way. We won’t know just yet until we try, but I think we may even be able to go as far as to register those functions to our own Lua state and not have to deal with the locking problem. Credits to you for giving me the idea!

    As far as custom scripts, it’s definitely possible. We called our own custom scripts and printed out the returned values to the chat log. An early example of this can be seen here: https://cdn.discordapp.com/attachments/349753828532420609/350824235624693780/unknown.png. We called our own script from the client’s Lua engine just to print out the player’s health value. Simple test, but opens up the door to pretty powerful things for us.

    Definitely happy to share our results with you and help each other. If you’d like to idle in our Discord server you’re more than welcome to! Just let me know if you’re interested and I’ll generate an invite for you.

Leave a Reply

Your email address will not be published. Required fields are marked *