Reproduced from https://sites.google.com/site/sbobovyc/writing/reverse-engineering/hooking-lua-part-1
Introduction
Lua is a widely used scripting language in games. When it comes to modding/hacking games, getting access to the scripting system of a game is a huge win. I decided to learn Lua and set up a few experiments in order to learn how to hook into it. The following is a write up of my adventures in Lua.
Compiling, linking, hooking
There are a few different ways to use Lua in a game. Most games are written in C++, and the Lua interpreter is called from inside native code. A game can also register functions with Lua, and after registration these native functions can be used from inside Lua scripts. My little toy application registers two native functions and executes some Lua scripts. The injectable DLL calls a Lua script that dumps the Lua global variables to a file. The globals will contain all the functions that can be called, and include non-standard Lua function that the game registered.
I compiled Lua 5.2.3 with mingw g++ 4.8.1:
$ make mingwThis generated src/liblua.a and src/lua52.dll.
My application was linked to the static Lua library. The application registered two internal functions that simply take a Lua state as a parameter and return some integer values. The application also prints out the address of the Lua state, various Lua C API functions, and the two registered functions:
This is the address of lua state: 0x009123B8
This is the address of luaL_newstate: 0x00405C40
This is the address of luaL_loadstring: 0x00405650
This is the address of luaL_loadfilex: 0x004053E0
This is the address of luaL_loadbufferx: 0x00405600
This is the address of lua_pcallk: 0x00403000
[Exe] This is a dostring print
[Lua] internal_function_1: function: 004016C0
[Lua] internal_function_2: function: 00401790
[Exe] Called internal_function_1
[Lua] got 10
[Exe] Called internal_function_2After the application is started, the DLL is injected and its DllMain function is called. The DLL uses the DllMain to start a new thread and hook the Lua state that is created in the main application. Most of the Lua C API function require a pointer to a Lua state, so the DLL needs to know its address in memory. No special memory allocator was used, so Lua uses realloc is used to allocate the state on the heap and the address of the state is not known ahead of time. To make things easier, I had the application write the address to a file and then the DLL reads the address from the file. Since the DLL is statically linked to Lua, it will use it's own Lua functions instead of the application's functions. If the application and the DLL were dynamically linked to Lua, when the DLL tried to access the Lua state it would cause a "multiple Lua VMs detected" error. The DLL executes a Lua script that dumps _G, the global Lua variables to a log file named state_dump.txt. This variable is a dictionary, and it contains all the functions that could be called from Lua. The addresses of the two internal functions can be seen in the dump:
internal_function_2 = function: 00401790
internal_function_1 = function: 004016C0A game would contain many registered functions, and their names and addresses could be used to reverse engineer some interesting parts of the game.
Debugging symbols where included which makes finding Lua function in the application easy. This is the location of luaL_newstate. Notice that the function luaLcheckversion is right below it:
00405C40 >/$ 53 PUSH EBX
00405C41 |. 83EC 18 SUB ESP,18
00405C44 |. C74424 04 0000>MOV DWORD PTR SS:[ESP+4],0
00405C4C |. C70424 D042400>MOV DWORD PTR SS:[ESP],lua_simp.004042D0
00405C53 |. E8 C8DDFFFF CALL lua_simp.lua_newstate
00405C58 |. 85C0 TEST EAX,EAX
00405C5A |. 89C3 MOV EBX,EAX
00405C5C |. 74 10 JE SHORT lua_simp.00405C6E
00405C5E |. C74424 04 8042>MOV DWORD PTR SS:[ESP+4],lua_simp.00404280
00405C66 |. 890424 MOV DWORD PTR SS:[ESP],EAX
00405C69 |. E8 52BFFFFF CALL lua_simp.lua_atpanic
00405C6E |> 83C4 18 ADD ESP,18
00405C71 |. 89D8 MOV EAX,EBX
00405C73 |. 5B POP EBX
00405C74 \. C3 RETN
00405C75 8D7426 00 LEA ESI,DWORD PTR DS:[ESI]
00405C79 8D DB 8D
00405C7A BC DB BC
00405C7B 27 DB 27 ; CHAR '''
00405C7C 00 DB 00
00405C7D 00 DB 00
00405C7E 00 DB 00
00405C7F 00 DB 00
00405C80 > $ 56 PUSH ESI
00405C81 . 53 PUSH EBX
00405C82 . 83EC 24 SUB ESP,24
00405C85 . 8B5C24 30 MOV EBX,DWORD PTR SS:[ESP+30]
00405C89 . DD4424 34 FLD QWORD PTR SS:[ESP+34]
00405C8D . DD5C24 18 FSTP QWORD PTR SS:[ESP+18]
00405C91 . 891C24 MOV DWORD PTR SS:[ESP],EBX
00405C94 . E8 47BFFFFF CALL lua_simp.lua_version
00405C99 . C70424 0000000>MOV DWORD PTR SS:[ESP],0
00405CA0 . 89C6 MOV ESI,EAX
00405CA2 . E8 39BFFFFF CALL lua_simp.lua_version
00405CA7 . 39F0 CMP EAX,ESI
00405CA9 . 74 75 JE SHORT lua_simp.00405D20
00405CAB . C74424 04 3636>MOV DWORD PTR SS:[ESP+4],lua_simp.00423636 ; ASCII "multiple Lua VMs detected"
00405CB3 . 891C24 MOV DWORD PTR SS:[ESP],EBX
00405CB6 . E8 75EAFFFF CALL lua_simp.luaL_errorThis is the source code of the two functions:
LUALIB_API lua_State *luaL_newstate (void) {
lua_State *L = lua_newstate(l_alloc, NULL);
if (L) lua_atpanic(L, &panic);
return L;
}
LUALIB_API void luaL_checkversion_ (lua_State *L, lua_Number ver) {
const lua_Number *v = lua_version(L);
if (v != lua_version(NULL))
luaL_error(L, "multiple Lua VMs detected");
else if (*v != ver)
luaL_error(L, "version mismatch: app. needs %f, Lua core provides %f",
ver, *v);
....Depending on how Lua is compiled into a game, this could be used to find and hook the luaL_newstate function and get the pointer to any Lua states that are created.
Source for these experiments can be found here https://github.com/sbobovyc/WinHookInject/tree/master/lua_experiments
Notes
http://stackoverflow.com/questions/11324117/how-do-modern-vms-handle-memory-allocation