Eilam E.Reversing.Secrets of reverse engineering.2005

Deciphering File Formats 223

This function starts out by reading a fixed size (4,104-byte) chunk of data from the archive file. The interesting thing about this read operation is how the starting address is calculated. The function receives a parameter that is multiplied by 4,104, adds 0x28, and is then used as the file offset from where to start reading. This exposes an important detail about the internal organization of Cryptex files: they appear to be divided into data blocks that are 4,104 bytes long. Adding 0x28 to the file offset is simply a way to skip the file header. The second parameter that this function takes appears to be some kind of a block number that the function must read.

After the data is read into memory, the function proceeds to decrypt it using the CryptDecrypt function. As expected, the data-length parameter (which is the sixth parameter passed to this function) is again hard-coded to 4104. It is interesting to look at the error message that is printed if this function fails. It reveals that this function is attempting to read and decrypt a cluster, which is probably just a fancy name for what I classified as those fixed-sized data blocks. If CryptDecrypt is successful, the function simply returns to the caller while returning the address of the newly decrypted block.

Analyzing a File Entry

Since you’re working under the assumption that the block that was just read is the archive’s file directory or some other part of its header, your next step is to take the decrypted block and attempt to study it and establish how it’s structured. The following memory dump shows the contents of the decrypted block I obtained while trying to list the files in the Test1.crx archive created earlier.

As you can see, you’re in the right place; the memory block contains the file name asterisks.txt, which you encrypted into this archive earlier. The question now is: What is in those 28 bytes that precede the file name? One thing to do right away is to try and view the data in a different way. In the preceding dump I used an ASCII view because I wanted to be able to see the file name string, but it might be easier to make out other fields by using a 32-bit view on this entry. The following are the first 28 bytes viewed as a sequence of 32-bit hexadecimal numbers:

224 Chapter 6

With this view, you can immediately see a somewhat improved picture. The first three DWORDs are obviously some kind of 32-bit fields. The last four DWORDs are not as obvious, and seem to be some kind of a random 16-byte sequence. This is easy to tell because they do not contain text (you would have seen that in the previous dump), and they are not pointers or offsets into the file (the numbers are far too large, and some of them are not 32-bit aligned). This is a classic case where stepping into the code that deciphers this data should really simplify the process of deciphering the file format.

The code that actually reads the file table and displays the file list is shown in Listing 6.6 and is actually quite simple to analyze because the fields that it reads are both printed into the screen, so it’s very easy to tell what they stand for. Let’s go back to that code sequence and see what it’s doing with this file entry.

This sequence starts out by loading ESI with the newly decrypted block’s starting address, adding 8 to that, and reading a 32-bit member at that address into EAX. If you go back to the previous memory dump, you’ll see that the third DWORD contains 00000001. At this point, the code confirms that EAX is nonzero, and proceeds to perform an interesting series of arithmetic operations on it.

First, EDX is shifted left by 0xA (10) bits, then the original value (from EAX) is subtracted from the result. At that point, the value of EDX is added to itself (which is the equivalent of multiplying it by two). This operation is performed again in 00401A82, and is followed by a right-shift of 0xA (10) bits. Now let’s go over these operations step by step and try to determine their purpose.

1.EDX is shifted left by 10, which is equivalent to edx = edx × 1,024.

2.The original number at EAX is subtracted from EDX. This means that instead of 1,024, you have essentially performed edx = edx × 1,024 – edx, which is the equivalent of edx = edx × 1,023.

Deciphering File Formats 225

3.EDX is then added to itself, twice. This is equivalent of edx = edx × 4, which means that so far you’ve essentially calculated edx = edx × 4,092.

4.Finally, EDX is shifted back right by 10 bits, which is the equivalent of dividing by 1,024. The final formula is edx = edx × 4092 ÷ 1024.

You might be wondering why Cryptex didn’t just use the MUL instruction to multiply EDX by 4,092 and then apply the DIV instruction to divide the result by 1,024. The answer is that such code would run far more slowly than the one we’ve just analyzed. MUL and DIV are both relatively slow instructions, whereas ADD, SUB, and the shifting instructions are much faster. It is important to note that this sequence reveals an interesting fact about Cryptex: It was most likely compiled using some kind of an optimize-for-fast-code switch, rather than with an optimize-for-small-code switch. That’s because using the direct arithmetic instructions for division and multiplication would have produced smaller, yet slower, code. The compiler was clearly aiming at the generation of high-performance code, even at the cost of a somewhat larger executable.

The result of this little arithmetic sequence goes right into the printf call that prints the current file entry. This is quite illuminating because it tells you exactly what Cryptex was trying to calculate in the preceding arithmetic sequence: the file size. In fact, it is quite obvious that because the file size is printed in kilobytes, the final division by 1,024 simply converts the file size from bytes to kilobytes. The question now is, what was that original number and why was Cryptex multiplying it by 4,092? Well, it would seem that the file size is maintained by using some kind of fixed block size, which is probably somehow related to the cluster you saw earlier while decrypting the buffer. The problem is that the cluster you were dealing with earlier was 4,104 bytes long, and here you’re dealing with clusters that are only 4,092 bytes long. The difference is not clear at this point.

The original number of clusters that you multiplied was taken from offset +8 in the current file entry structure, so you know that offset +8 contains the file size in clusters. This raises the question of where does Cryptex store the actual file size? It would not be possible to accurately recover encrypted files without creating them with the exact size they had originally. Therefore Cryptex must also maintain the accurate file size somewhere in the archive file.

Other than the file size, the printf call also takes the file name, which is easily obtained by taking the address of offset +14 from ESI. Keep in mind that ESI was incremented by 8 earlier, so this is actually offset +1C from the original data structure, which matches what you saw in our data dump, where the string started at offset +1C.

After printing the file name and size, the program loops back to print the next entry. To reach the next item, Cryptex increments ESI by 0x98 bytes (152 in decimal), which is clearly the length of each entry. This indicates that there is a fixed number of characters reserved for each file name. Since you know