is converted into the following one-liner:

Pretty simple, you may say... Well, here's a longer one (sorry for the length of the assembler listing, please scroll down):

The above code is translated into:

I strongly prefer the second listing to the first. In fact, the more I use the decompiler, the less I want to return to the assembly level (this means that you may expect source level debugging and other similar improvements in the future ;)

In order to handle floating point, we also had to improve many other aspects of the decompiler. Here are the things I remember offhand:

• We changed the stack variable allocation mechanism to use data flow information. In practice this means that reused stack frame slots are recognized and multiple variables are created for them. No more funny casts because of a stack slot reuse!
• The stack variables are considered as first class citizens by the propagation and other algorithms. Previous versions of the decompiler were optimizing registers but stack variables were not optimized much. In practice: shorter and cleaner output. This improvement, combined with the previous one, allows us to handle reused function stack arguments very smoothly. It goes without saying that aliased stack variables are still not optimized (unfortunately, it can not be done automatically)
• Made the optimization rules more robust and more efficient
• Added more rules to remove unnecessary casts
• Add a new algorithm to recognize call arguments
• Better user interface (as usual, improving ui is always a good idea ;)

Here's the download link:

http://www.hex-rays.com/idapro/idadowndemo.htm

Enjoy!

VMWare's GDB stub is very basic, it doesn't know anything about processes or threads (for Windows guests), so for anything high-level we'll need to do some extra work. We will show how to get the loaded module list and load symbols for all them using IDAPython.

Let's assume that you already have a VM with Windows (32-bit) installed. Before starting the debugging, copy files for which you want to see symbols to the host. If you're not sure, copy nt*.exe and hal.dll from System32, and the whole System32\drivers directory.

Edit the VM's .vmx file to enable GDB debugger stub:

Add these lines to the file:

debugStub.listen.guest32 = "TRUE"
debugStub.hideBreakpoints= "TRUE"

Save the file.

In VMWare, click "Power on this virtual machine" or click the green Play button on the toolbar.

Wait until the VM boots.

Debugging in IDA

Start IDA.

If you get the welcome dialog, choose "Go".

Choose Debugger | Attach | Remote GDB debugger.

Enter "localhost" for hostname and 8832 for the port number.

Choose <attach to the process started on target> and click OK.

The execution should stop somewhere in the kernel (address above 0x80000000). You can step through the code, but it's not very convenient without any names. Let's try to gather some more information.

Getting the module list

The list of kernel modules is stored in the list pointed to by the PsLoadedModuleList symbol in the kernel. To find its address, we will use the so-called "KPCR trick". KPCR stands for Kernel Processor Control Region. It is used by the kernel to store various information about each processor. It is placed at the base of the segment pointed to by the fs register (similar to TEB in user mode). One of the fields in it is KdVersionBlock which points to a structure used by the kernel debugger. It, in turn, has various pointers to kernel structures, including PsLoadedModuleList.

Definition of the KPCR structure can be found in many places, including IDA's ntddk.til. Right now we just need to know that KdVersionBlock field is situated at offset 0x34 from the start of KPCR. It points to DBGKD_GET_VERSION64, which has PsLoadedModuleList pointer at offset 0x18.

Let's write a small Python function to find the value of that pointer. To retrieve the base of the segment pointed to by fs, we can use the VMWare's debug monitor "r" command. GDB debugger plugin registers an IDC function SendGDBMonitor() to send commands to the monitor, and we can use IDAPython's Eval() function to call it:

fs_str = Eval('SendGDBMonitor("r fs")')

Returned string has the following format:

fs 0x30 base 0x82744a00 limit 0x00002008 type 0x3 s 1 dpl 0 p 1 db 1

We need the address specified after "base":

kpcr = int(fs_str[13:23], 16) #extract and convert as base 16 (hexadecimal) number

Then get the value of KdVersionBlock:

kdversionblock = Dword(kpcr+0x34)

And finally PsLoadedModuleList:

PsLoadedModuleList = Dword(kdversionblock+0x18)

Walking the module list

PsLoadedModuleList is declared as PLIST_ENTRY. LIST_ENTRY is a structure which represents a member of a double-linked list:

typedef struct _LIST_ENTRY
{
     PLIST_ENTRY Flink;
     PLIST_ENTRY Blink;
} LIST_ENTRY, *PLIST_ENTRY;

So, we just need to follow the Flink pointer until we come back to where we started. A single entry of the list has the following structure:

struct LDR_MODULE
{
  LIST_ENTRY InLoadOrderModuleList;
  LIST_ENTRY InMemoryOrderModuleList;
  LIST_ENTRY InInitializationOrderModuleList;
  PVOID BaseAddress;
  PVOID EntryPoint;
  ULONG SizeOfImage;
  UNICODE_STRING FullDllName;
  UNICODE_STRING BaseDllName;
  ULONG Flags;
  SHORT LoadCount;
  SHORT TlsIndex;
  LIST_ENTRY HashTableEntry;
  ULONG TimeDateStamp;
};

Now we can write a small function to walk this list and create a segment for each module:

#get the first module
cur_mod = Dword(PsLoadedModuleList)
while cur_mod != PsLoadedModuleList and cur_mod != BADADDR:
  BaseAddress  = Dword(cur_mod+0x18)
  SizeOfImage  = Dword(cur_mod+0x20)
  FullDllName  = get_unistr(cur_mod+0x24)
  BaseDllName  = get_unistr(cur_mod+0x2C)
  #create a segment for the module
  SegCreate(BaseAddress, BaseAddress+SizeOfImage, 0, 1, saRelByte, scPriv)
  #set its name
  SegRename(BaseAddress, BaseDllName)
  #get next entry
  cur_mod = Dword(cur_mod)

Loading symbols

Having the module list is nice, but not very useful without symbols. We can load the symbols manually for each module using File | Load File | PDB file... command, but it would be better to automate it.

For that we can use the PDB plugin. From looking at its sources (available in the SDK), we can see that it supports three "call codes":

//call_code==0: user invoked 'load pdb' command, load pdb for the input file
//call_code==1: ida decided to call the plugin itself
//call_code==2: load pdb for an additional exe/dll
//              load_addr: netnode("$ pdb").altval(0)
//              dll_name:  netnode("$ pdb").supstr(0)

Call code 2 looks just like what we need. However, current IDAPython includes a rather basic implementation of netnode class and it is not possible to set supvals from Python. However, if we look at handling of the other call codes, we can see that the plugin retrieves module base from "$ PE header" netnode and module path using get_input_file_path() function. IDAPython's netnode.altset() function does work, and we can use set_root_filename() to set the input file path. Also, if we pass a call code 3, we will avoid the "Do you want to load the symbols?" prompt.

#new netnode instance
penode = idaapi.netnode()
#create netnode the in database if necessary
penode.create("$PE header")
#set the imagebase (-2 == 0xFFFFFFFE)
penode.altset(0xFFFFFFFE, BaseAddress)
#set the module filename
idaapi.set_root_filename(filename)
#run the plugin
RunPlugin("pdb",3)

However, we need to replace the kernel-mode path by the local path beforehand:

#path to the local copy of System32 directory
local_sys32 = r"D:\VmWareShared\w7\System32"
if FullDllName.lower().startswith(r"\systemroot\system32"):
#translate into local filename
filename = local_sys32 + FullDllName[20:]

Now we can gather all pieces into a single script. Download it here

After running it, you should have a nice memory map:

...and name list:

Looks much better now. Happy debugging!

http://www.hex-rays.com/idapro/debugger/index.htm

Direct links to tutorials are available here:

http://www.hex-rays.com/idapro/idasupport.htm

I know, I know - we need to add 64-bit support for all platforms, port the Bochs debugger module to Linux, and... any other suggestions? I personally would love to have source level debugging, yet it requires some substantial changes to the kernel. We probably will move in this direction, sooner or later...

For user mode debugging the Windbg debugger plugin beats the win32 debugger plugin, by providing you access to a wide range of extensions that ship with the debugging tools from Microsoft.
For kernel debugging, you can use Bochs/Disk Image loader or GDB plugin to debug the whole operating system from Bios code and on.
However when Windbg plugin is used, you get the raw power of the debugging engine (extensions / built-in commands, symbols, ...).

We prepared a video showing how to debug kernel mode and user mode at the same time with full symbolic information (provided from the PDB files).
The video also demonstrates how to set breakpoints on user mode APIs and see them get triggered when any application in the system uses those APIs.

Before viewing the video, for those willing to experiment with the Windbg debugger plugin to debug kernel mode and user mode at the same time, here is how to prepare a database:

1. If you never used the Windbg debugger plugin before please visit the Windbg plugin tutorial page
2. Setup a process server inside the VM and attach to it from IDA to debug just any user mode application
3. Once attached, go to desired segments (kernel32, user32, advapi32, gdi32, etc...) and convert them to loader segments
4. If symbol retrieval mechanism was properly configured then most system DLLs will have symbol information, otherwise only exported names will available
5. Now we have a database with all user mode components we wish to inspect from the live kernel debugging session
6. Using the same database, change the connection string so that it connects to the same VM for the purpose of live kernel debugging this time
7. Once attached to the kernel, IDA will present loaded drivers and kernel mode modules in the debugger / modules list
8. It is possible to convert to loader segments the kernel mode components of interest
9. That's it! The database is now suited for kernel debugging, yet contains names and addresses of user mode components

The video will put everything into perspective!

In addition to numerous small and not that small improvements, the new version will have three debugger modules: bochs, gdb, and windbg, selectable on the fly (the active debugger session will be closed, though ;))

• With the bochs debugger, we offer three different worlds: run-any-code-snippet facility, windows-like-environment for PE files, and any-bochs-image bare-bone machine emulation mode. You can read more about this module in our blog: http://hexblog.com/2008/11/bochs_plugin_goes_alpha.html
  
• With gdb, x86 and arm targets are supported. Among other things, it is possible to connect IDA to QEMU or debug a virtual machine inside VMWare. We tried it iPhone as well. However, while it works in some curcimstances, there were some problems on the gdbserver side.
  
• With windbg, user and kernel mode debugging is available. The debugger engine from Microsoft, which is currently the only choice for driver and kernel mode debugging, can be used from IDA. It can automatically load required PDB files and populate the listing with meaningful names, types, etc. Speaking of PDB files, IDA imports more information from them: local function variables and types are retrieved too, c++ base classes are handled, etc.
  

The gdb and windbg debugger modules support local and remote debugging. We tried to make the debugger modules as open as possible: target-specific commands can be sent to all backend engines in a very easy and user-friendly way.

As usual, better analysis and many minor changes have been made. If you spend plenty of time analyzing gcc generated binaries, you'll certainly appreciate that IDA handles its weird way of preparing outgoing function arguments. Now it can trace and find arguments copies to the stack with mov statements.

The new IDA will support Python out of box, thanks to Gergely Erdelyi, who kindly agreed the Python plugin to be included in the official distribution. In fact, the main IDA window will have a command line to enter any python (or other language) expressions and immediately get a result in the message window.

We will prepare the detailed list of improvements later this week.

http://www.binary-art.net/?p=1002

This is MIPS emulator for Linux. It can generate an IDC script after emulation, which then can be applied to the database and make it more readable.

Throughout the video you will see numerous tricks used by this malware and you will see how the debugger works smoothly with them.

Here are some of the features supported by it:

  download_winivstr();  // Download a program from the internet
                        // It will be launched later
                        // Create a thread to scare the user
  icon_thread = CreateThread(0, 0, icon_thread_entry, 0, 0, &Data);
  Sleep(1000u);         // 1 second
  while ( 1 )
  {
    if ( check_security_guards() == GUARD_EXISTS )
      SendMessageA(main_hwnd, WM_DESTROY, 0, 0);
    else
      scare_user();    // Tell the user that his computer is infected
    Sleep(180000u);    // 3 minutes
  }

http://www.hex-rays.com/decompilation/files/lighty_fraudload.zip

The next time I'll try to find something more complicated to show you more advanced features of the decompiler. Something really difficult to understand even for seasoned reverse engineers. For example, can you make sense out the following code? How much time does it take for you?

push ebp mov ebp, esp sub esp, 8 mov edx, [ebp+arg_C] mov eax, [ebp+arg_8] push edx push eax fild [esp+10h+var_10] add esp, 8 test edx, edx js short loc_2A23 fstp [ebp+var_8] fld [ebp+var_8] fld [ebp+arg_0] leave fucompp fnstsw ax sahf setnz al setp dl or al, dl movzx eax, al retn loc_2A23: fadd ds:flt_AEEC fstp [ebp+var_8] fld [ebp+var_8] fld [ebp+arg_0] leave fucompp fnstsw ax sahf setnz al setp dl or al, dl movzx eax, al retn

This mess was originally a one-line statement: bool cmpeq_double_longlong(double a, unsigned __int64 b) { return a == b; }

(you knew that it would be that simple, didn't you? ;)

As you see, we are playing with the floating point arithmetic now. Who knows, maybe the decompiler will handle it in the nearest future. Do not hold your breath yet: there is a long way ahead and many problems to solve. The above listing is from our sample test file. The test file has ~750 trivial functions and we compile it with 3 different compilers in optimized and non-optimized modes. So we 'just' need to make sure that all 750*3*2=4500 functions decompile correctly and we will have the first decompilation step over. Then we will need to make sure that all possible combinations of integer and floating point arithmetic decompile well, type conversions do not spoil the result, and the output is generated correctly. For integer arithmetic, a similar test file took more than one month, I bet that floating point will take longer... But we will eventually be there, with a good result (it must be portable too, since we do not plan to stay forever with x86). Stay tuned! :) ]]> tag:hexblog.com,2008://1.84 2008-10-02T23:11:14Z 2008-10-02T23:50:49Z

The next version of IDA will be released with a bochs debugger plugin, and what is nice about is that you will be able to use it easily by just downloading bochs executables and telling IDA where to find it.

]]> IDA's bochs debugger is a plugin that allow you to use bochs' emulation/debugger inside IDA's interface, but not just only that, but to make your debugging experience easier.

The plugin will come with three of the what we dubbed as "bochs loaders", so here is a brief explanation:

The first loader, disk image loader, is probably the most simple but yet the most powerful one. It allows you to debug any bochs image of your choice. For example, you could debug boot sector, 16 bit code, and perhaps debug 32 bit code all in the same debugging session. We actually use this bochs loader to debug other bochs loaders!

The second, idb loader, is a 32bit mode loader that allow you to debug anything within the database. The database will be your input file, thus whatever segments exist in the database, will be loaded and mapped into bochs' virtual memory. The idb loader understands and catches raw cpu exceptions and allows you to specify the startup stack segment's size.

Finally comes the pe loader, which is a specialized bochs loader, that will read your PE file and create a virtual environment similar to windows environment, trying to mimic basic demands for a PE file (import resolution, SEH, api emulation backed by IDC scripts).

This plugin is still under development, however we put a small video demonstrating the IDB loader.

Here's a small video:

]]>

Background Intelligent Transfer Service (BITS) is a component of modern Microsoft Windows operating systems that facilitates prioritized, throttled, and asynchronous transfer of files between machines using idle network bandwidth.

The web page ends with a list of third-party applications that use BITS. However, as any technical method, it can be used for evil purposes as well. Eric Landuyt analyzed a malware that exploits it for bad:

http://www.datarescue.com/laboratory/trojan2008/index.html

I liked the "proof of concept" WinDbg script that runs the malware in a controlled manner. Breakpoints with actions are very powerful, indeed.

Nice work, Eric!

This is not the first book about IDA Pro. However, this is the first book I recommend to anyone using IDA Pro because of the following points:

• Comprehensive: it describes all major IDA features by starting at the beginning and going all the way to the end. Experienced users may be tempted to skip the first few chapters; resist this temptation and you will discover something new (I did :)
• Accurate: it is very difficult to be detailed and precise when describing such a complex product. Chris does it excellently well.
• Real: handles real world malware, packers, and obfuscated code
• No fillers: it is direct and concise
• Profound: this is not just a collection of recipes or tricks, but will give you a better understanding of the IDA architecture, thus saving you from unnecessary frustration. Knowing the limitations of your tool is just as important as knowing its capabilities.

If you want to use IDA efficiently, get your copy from No Starch Press!

UPD for numerologists: the book has exactly 640 pages, no less, no more! ]]>