Eilam E.Reversing.Secrets of reverse engineering.2005

Breaking Protections 401

After KERNEL32.DLL is loaded, Defender goes through the familiar sequence of allocating a random address in memory and produces the same name checksum/RVA table from all the KERNEL32.DLL exports. After the copied module is ready for use the function makes one other call to NtDelay Execution for good luck and then you get to another funny jump that skips 30 bytes or so. Dumping the memory that immediately follows the CALL instruction as text reveals the following:

Finally, you’re looking at something familiar. This is Defender’s welcome message, and Defender is obviously preparing to print it out. The CALL instruction skips the string and takes us to the following code.

00404167 PUSH DWORD PTR SS:[ESP]

0040416A CALL Defender.004012DF

The code is taking the “return address” pushed by the CALL instruction and pushes it into the stack (even though it was already in the stack) and calls a function. You don’t even have to look inside this function (which is undoubtedly full of indirect calls to copied KERNEL32.DLL code) to know that this function is going to be printing that welcome message that you just pushed into the stack. You just step over it and unsurprisingly Defender prints its welcome message.

Reencrypting the Function

Immediately afterward you have yet another call to 6DEF20—NtDelay Execution and that brings us to what seems to be the end of this function. OllyDbg shows us the following code:

402 Chapter 11

If you look closely at the address that the JMP at 004041F4 is going to you’ll notice that it’s very far from where you are at the moment—right at the beginning of this function actually. To refresh your memory, here’s the code at that location:

00403401 CMP DWORD PTR SS:[EBP-8],0

00403405 JE SHORT Defender.0040346D

You may or may not remember this, but the line immediately preceding 00403401 was setting [EBP-8] to 1, which seemed a bit funny considering it was immediately checked. Well, here’s the answer—there is encrypted code at the end of the function that sets this variable to zero and jumps back to that same position. Since the conditional jump is taken this time, you land at 40346D, which is a sequence that appears to be very similar to the decryption sequence you studied in the beginning. Still, it is somewhat different, and observing its effect in the debugger reveals the obvious: it is reencrypting the code in this function.

There’s no reason to get into the details of this logic, but there are several details that are worth mentioning. After the encryption sequence ends, the following code is executed:

004034D0 MOV DWORD PTR DS:[406008],EAX

004034D5 PUSH Defender.004041FD

004034DA POP EBX

004034DB JMP EBX

The first line saves the value in EAX into a global variable. EAX seems to contain some kind of a checksum of the encrypted code. Also, the PUSH, POP, JMP sequence is the exact same code that originally jumped into the decrypted code, only it has been modified to jump to the end of the function.

Back at the Entry Point

After the huge function you’ve just dissected returns, the entry point routine makes the traditional call into NtDelayExecution and calls into another internal function, at 404202. The following is a full listing for this function:

Breaking Protections 403

This function performs another one of the familiar copied export table searches, this time on the copied KERNEL32 memory block (whose pointer is stored at 406004). It then immediately calls the found function. You’ll use the function index trick that you used before in order to determine which API is being called. For this you put a breakpoint on 404227 and observe the address loaded into ECX. You then subtract KERNEL32’s copied base address (which is stored at 406004) from this address and divide the result by 8. This gives us the current API’s index. You quickly run DUMPBIN /EXPORTS on

KERNEL32.DLL and find the API name: SetUnhandledExceptionFilter. It looks like Defender is setting up 0040322E as its unhandled exception filter. Unhandled exception filters are routines that are called when a process generates an exception and no handlers are available to handle it. You’ll worry about this exception filter and what it does later on.

Let’s proceed to another call to NtDelayExecution, followed by a call to another internal function, 401746. This function starts with a very familiar sequence that appears to be another decryption sequence; this function is also encrypted. I won’t go over the decryption sequence, but there’s one detail I want to discuss. Before the code starts decrypting, the following two lines are executed:

00401785 MOV EAX,DWORD PTR DS:[406008]

0040178A MOV DWORD PTR SS:[EBP-9C0],EAX

The reason I’m mentioning this is that the variable [EBP-9C0] is used a few lines later as the decryption key (the value against which the code is XORed to decrypt it). You probably don’t remember this, but you’ve seen this global variable 406008 earlier. Remember when the first encrypted function was about to return, how it reencrypted itself? During encryption the code calculated a checksum of the encrypted data, and the resulting checksum was stored in a global variable at 406008. The reason I’m telling you all of this is that this is an unusual property in this code—the decryption key is calculated at runtime. One side effect this has is that any breakpoint installed on encrypted code that is not removed before the function is reencrypted would change this checksum, preventing the next function from properly decrypting! Defender is doing as its name implies: It’s defending!

404 Chapter 11

Let’s proceed to investigate the newly decrypted function. It starts with two calls to the traditional NtDelayExecution . Then the function proceeds to call what appears to be NtOpenFile through the obfuscated interface, with the string “\??\C:” hard-coded right there in the middle of the code. After

NtOpenFile the function calls NtQueryVolumeInformationFile with the FileFsVolumeInformation information level flag. It then reads offset +8 from the returned data structure and stores it in the local variable

[406020]. Offset +8 in data structure FILE_FS_VOLUME_INFORMATION is

VolumeSerialNumber (this information was also obtained at http:// undocumented.ntinternals.net).

This is a fairly typical copy protection sequence, in a slightly different flavor. The primary partition’s volume serial number is a good way to create com- puter-specific dependencies. It is a 32-bit number that’s randomly assigned to a partition when it’s being formatted. The value is retained until the partition is formatted. Utilizing this value in a serial-number-based copy protection means that serial numbers cannot be shared between users on different computers— each computer has a different serial number. One slightly unusual thing about this is that Defender is obtaining this value directly using the native API. This is typically done using the GetVolumeInformation Win32 API.

You’ve pretty much reached the end of the current function. Before returning it makes yet another call to NtDelayExecution, invokes RDTSC, loads the low-order word into EAX as the return value (to make for a garbage return value), and goes back to the beginning to reencrypt itself.

Parsing the Program Parameters

Back at the main entry point function, you find another call to NtDelay Execution which is followed by a call into what appears to be the final function call (other than that apparently useless call to IsDebuggerPresent) in the program entry point, 402082.

Naturally, 402082 is also encrypted, so you will set a breakpoint on 402198, which is right after the decryption code is done decrypting. You immediately start seeing familiar bits of code (if Olly is still showing you junk instead of code at this point, you can either try stepping into that code and see if automatically fixes itself or you can specifically tell Olly to treat these bytes as code by right-clicking the first line and selecting Analysis. During next analysis, treat selection as Command). You will see a call to NtDelayExecution, followed by a sequence that loads a new DLL: SHELL32.DLL. The loading is followed by the creation of the obfuscated module interface: allocating memory at a random address, creating checksums for each of the exported SHELL32.DLL names, and copying the entire code section into the newly allocated memory block. After all of this the program calls a KERNEL32.DLL that

Breaking Protections 405

has a pure user-mode implementation, which forces you to use the function index method. It turns out the API is GetCommandLineW. Indeed, it returns a pointer to our test command line.

The next call is to a SHELL32.DLL API. Again, a SHELL32 API would probably never make a direct call down into the kernel, so you’re just stuck with some long function and you’ve no idea what it is. You have to use the function’s index again to figure out which API Defender is calling. This time it turns out that it’s CommandLineToArgvW. CommandLineToArgvW performs parsing on a command-line string and returns an array of strings, each containing a single parameter. Defender must call this function directly because it doesn’t make use of a runtime library, which usually takes care of such things.

After the CommandLineToArgvW call, you reach an area in Defender that you’ve been trying to get to for a really long time: the parsing of the commandline arguments.

You start with simple code that verifies that the parameters are valid. The code checks the total number of arguments (sent back from CommandLine ToArgvW) to make sure that it is three (Defender.EXE’s name plus username and serial number). Then the third parameter is checked for a 16-character length. If it’s not 16 characters, defender jumps to the same place as if there aren’t three parameters. Afterward Defender calls an internal function, 401CA8 that verifies that the hexadecimal string only contains digits and letters (either lowercase or uppercase). The function returns a Boolean indicating whether the serial is a valid hexadecimal number. Again, if the return value is 0 the code jumps to the same position (40299C), which is apparently the “bad parameters” code sequence. The code proceeds to call another function (401CE3) that confirms that the username only contains letters (either lowercase or uppercase). After this you reach the following three lines:

00402994 TEST EAX,EAX

00402996 JNZ Defender.00402AC4

0040299C CALL Defender.004029EC

When this code is executed EAX contains the returns value from the username verification sequence. If it is zero, the code jumps to the failure code, at 40299C, and if not it jumps to 402AC4, which is apparently the success code. One thing to notice is that 4029EC again uses the CALL instruction to skip a string right in the middle of the code. A quick look at the address right after the CALL instruction in OllyDbg’s data view reveals the following:

406 Chapter 11

So, you’ve obviously reached the “bad parameters” message display code. There is no need to examine this code – you should just get into the “good parameters” code sequence and see what it does. Looks like you’re close!

Processing the Username

Jumping to 402AC4, you will see that it’s not that simple. There’s quite a bit of code still left to go. The code first performs some kind of numeric processing sequence on the username string. The sequence computes a modulo 48 on each character, and that modulo is used for performing a left shift on the character. One interesting detail about this left shift is that it is implemented in a dedicated, somewhat complicated function. Here’s the listing for the shifting function:

This code appears to be a 64-bit left-shifting logic. CL contains the number of bits to shift, and EDX:EAX contains the number being shifted. In the case of a full-blown 64-bit left shift, the function uses the SHLD instruction. The SHLD instruction is not exactly a 64-bit shifting instruction, because it doesn’t shift the bits in EAX; it only uses EAX as a “source” of bits to shift into EDX. That’s why the function also needs to use a regular SHL on EAX in case it’s shifting less than 32 bits to the left.

Breaking Protections 407

After the 64-bit left-shifting function returns, you get into the following code:

Figure 11.16 shows what this sequence does in mathematical notation. Essentially, Defender is preparing a 64-bit integer that uniquely represents the username string by taking each character and adding it at a unique bit position in the 64-bit integer.

The function proceeds to perform a similar, but slightly less complicated conversion on the serial number. Here, it just takes the 16 hexadecimal digits and directly converts them into a 64-bit integer. Once it has that integer it calls into 401EBC, pushing both 64-bit integers into the stack. At this point, you’re hoping to find some kind of verification logic in 401EBC that you can easily understand. If so, you’ll have cracked Defender!

Validating User Information

Of course, 401EBC is also encrypted, but there’s something different about this sequence. Instead of having a hard-coded decryption key for the XOR operation or read it from a global variable, this function is calling into another function (at 401D18) to obtain the key. Once 401D18 returns, the function stores its return value at [EBP-1C] where it is used during the decryption process.

len

Sum = ΣCn × 2Cn mod48 n = 0

Figure 11.16 Equation used by Defender to convert username string to a 64-bit value.

408 Chapter 11

Let’s step into this function at 401D18 to determine how it produces the decryption key. As soon as you enter this function, you realize that you have a bit of a problem: It is also encrypted. Of course, the question now is where does the decryption key for this function come from? There are two code sequences that appear to be relevant. When the function starts, it performs the following:

00401D1F MOV EAX,DWORD PTR SS:[EBP+8]

00401D22 IMUL EAX,DWORD PTR DS:[406020]

00401D29 MOV DWORD PTR SS:[EBP-10],EAX

This sequence takes the low-order word of the name integer that was produced earlier and multiplies it with a global variable at [406020]. If you go back to the function that obtained the volume serial number, you will see that it was stored at [406020]. So, Defender is multiplying the low part of the name integer with the volume serial number, and storing the result in [EBP10]. The next sequence that appears related is part of the decryption loop:

This sequence subtracts the parameter at [EBP+10] from the result of the previous multiplication, and XORs that value against the encrypted function! Essentially Defender is doing Key = (NameInt * VolumeSerial) – LOWPART(SerialNumber). Smells like trouble! Let the decryption routine complete the decryption, and try to step into the decrypted code. Here’s what the beginning of the decrypted code looks like (this is quite random—your milage may vary).

It is quite easy to see that this is meaningless junk. It looks like the decryption failed. But still, it looks like Defender is going to try to execute this code! What happens now really depends on which debugger you’re dealing with, but Defender doesn’t just go away. Instead it prints its lovely “Sorry . . . Bad Key.” message. It looks like the top-level exception handler installed earlier is the one generating this message. Defender is just crashing because of the bad code in the function you just studied, and the exception handler is printing the message.

Breaking Protections 409

Unlocking the Code

It looks like you’ve run into a bit of a problem. You simply don’t have the key that is needed in order to decrypt the “success” path in Defender. It looks like Defender is using the username and serial number information to generate this key, and the user must type the correct information in order to unlock the code. Of course, closely observing the code that computes the key used in the decryption reveals that there isn’t just a single username/serial number pair that will unlock the code. The way this algorithm works there could probably be a valid serial number for any username typed. The only question is what should the difference be between the VolumeSerial * NameLowPart and the low part of the serial number? It is likely that once you find out that difference, you will have successfully cracked Defender, but how can you do that?

Brute-Forcing Your Way through Defender

It looks like there is no quick way to get that decryption key. There’s no evidence to suggest that this decryption key is available anywhere in Defender.EXE; it probably isn’t. Because the difference you’re looking for is only 32 bits long, there is one option that is available to you: brute-forcing. Brute-forcing means that you let the computer go through all possible keys until it finds one that properly decrypts the code. Because this is a 32-bit key there are only 4,294,967,296 possible options. To you this may sound like a whole lot, but it’s a piece of cake for your PC.

To find that key, you’re going to have to create a little brute-forcer program that takes the encrypted data from the program and tries to decrypt it using every key, from 0 to 4,294,967,296, until it gets back valid data from the decryption process. The question that arises is: What constitutes valid data? The answer is that there’s no real way to know what is valid and what isn’t. You could theoretically try to run each decrypted block and see if it works, but that’s extremely complicated to implement, and it would be difficult to create a process that would actually perform this task reliably.

What you need is to find a “token”—a long-enough sequence that you know is going to be in the encrypted block. This will allow you to recognize when you’ve actually found the correct key. If the token is too generic, you will get thousands or even millions of hits, and you’ll have no idea which is the correct key. In this particular function, you don’t need an incredibly long token because it’s a relatively short function. It’s likely that 4 bytes will be enough if you can find 4 bytes that are definitely going to be a part of the decrypted code.

You could look for something that’s likely to be in the code such as those repeated calls to NtDelayExecution, but there’s one thing that might be a bit easier. Remember that funny variable in the first function that was set to one and then immediately checked for a zero value? You later found that the

410Chapter 11

encrypted code contained code that sets it back to zero and jumps back to that address. If you go back to look at every encrypted function you’ve gone over, they all have this same mechanism. It appears to be a generic mechanism that reencrypts the function before it returns. The local variable is apparently required to tell the prologue code whether the function is currently being encrypted or decrypted. Here are those two lines from 401D18, the function you’re trying to decrypt.

00401D49 MOV DWORD PTR SS:[EBP-4],1

00401D50 CMP DWORD PTR SS:[EBP-4],0

00401D54 JE SHORT Defender.00401DBF

As usual, a local variable is being set to 1, and then checked for a zero value. If I’m right about this, the decrypted code should contain an instruction just like the first one in the preceding sequence, except that the value being loaded is 0, not 1. Let’s examine the code bytes for this instruction and determine exactly what you’re looking for.

Here’s the OllyDbg output that includes the instruction’s code bytes. It looks like this is a 7-byte sequence—should be more than enough to find the key. All you have to do is modify the 01 byte to 00, to create the following sequence:

C7 45 FC 00 00 00 00

The next step is to create a little program that contains a copy of the encrypted code (which you can rip directly from OllyDbg’s data window) and decrypts the code using every possible key from 0 to FFFFFFFF. With each decrypted block the program must search for the token—that 7-byte sequence you just prepared . As soon as you find that sequence in a decrypted block, you know that you’ve found the correct decryption key. This is a pretty short block so it’s unlikely that you’d find the token in the wrong decrypted block.

You start by determining the starting address and exact length of the encrypted block. Both addresses are loaded into local variables early in the decryption sequence:

00401D2C PUSH Defender.00401E32

00401D32 MOV DWORD PTR SS:[EBP-14],EAX

00401D35 PUSH Defender.00401EB6

00401D3B MOV DWORD PTR SS:[EBP-C],EAX