Final method of loading several files into a database

D. Use Steve's plugin

There might be other methods of loading PE files into the database but since the presented methods cover almost all cases and needs, I'd prefer to switch to other things. One more method deserves mentioning before we move on, though.

It is about a clever hack by Steve Micallef. In his excellent IDA Plugin Writing Tutorial Steve gives a sample of a fairly short plugin which loads the PE loader module and asks it to load an additional file into the database. You see, the idea is simple and the implementation is quite short yet it solves the problem.

You can download the plugin source here. It can be modified to work with other file types since it is not limited to PE file format in any way: just replace the "pe" string in the source code with any other loader name.

Since it is a hack (completely unforeseen by me - file loaders were not supposed to be used this way) there will be some strange things (for example, the import table will not be updated), but they can be solved one way or another. The main idea remains simple: when something can not be done trough the user interface or IDC, consider writing a plugin.

Posted by Ilfak Guilfanov at 06:25 PM | Permalink

I promised to tell you about the TLS callbacks.
Here is the discussion.

When you launch the debugger and set a breakpoint at the program entry point, one may think that the application will be suspended as soon as the process is created. In reality, this is not the case and the application can easily run out of control.

Let's take this sample program.
If we load the executable into IDA, we will see the following:

It seems to be an empty C program. However, the author might have used a modified version of the C startup. To avoid any problems because of the C startup, we will put a breakpoint at the program entry point by pressing F2 (the entry point is always named start by IDA):

Now, if we launch the debugger by pressing F9...

Hold on.. see.. the application runs!

The application managed to run beyond the entry point despite of our breakpoint!

In reality the mistery is explained easily: a TLS (thread-local-storage) callback is used by the application. Such a callback is called by the system before the application entry point, that's why our breakpoint was not triggered.

IDA Pro knows about TLS callbacks since long ago. If you open the entry point list (Ctrl-E), you will see that there is a function called TlsCallback_0 and the function is nicely disassembled.

What could you do to avoid such bad surprises? Here are some recommendations:

• Run the application in a confined virtual environment
  
• Look for TLS callbacks in the entry point list before launching the debugger
  
• Configure the debugger to suspend as soon as the process is created (see Debugger, Debugger options, Stop on options)

Posted by Ilfak Guilfanov at 11:56 PM | Permalink | Comments (2)

The third method to create a database with several PE files.

C. Change segment attributes in the debugger

We start the debugger and suspend the application as in the previous method. This time instead of using the 'take memory snapshot' command we will play with the segment attributes. Let's open the program segmentation by using View, Open subviews, Segments (the window might be hidden by other 'on top' windows, please make it 'on top' as well):

Please note the D and L columns. They mean 'debugger' and 'loader' respectively.

A debugger segment is a segment created by the debugger.

A loader segment is a segment created by the input file loader.

When the debugger is inactive, the database has only 'loader' segments. The debugger adds 'debugger' segments to the database. These segments are ephemeral and will be destroyed when the debugger stops. So, in order to preserve them in the database, all we have to do is just to change these bits: remove D and add L.

We can do it from the 'edit segment' dialog box. If we try to edit a segment, IDA displays a warning:

We will hide this message and proceed.

We remove the D attribute and add the L attribute.

This method is very flexible and allows us to save any DLL or segment in the database. To save a DLL, we need to change the attributes of all segments with the DLL's name (in other words, to save ADVAPI32.DLL we have to save all segments named advapi32.dll). The only inconvenience is that you have to edit each segment separately.
You can do it easily from the segmentation window by pressing Ctrl-E to edit the attributes of the selected line.

The segment attributes method can also be used to save an unpacked copy of a packed program to the database and continue the analysis without the debugger. Malware writers often choose to pack their creations, so this method can be used to analyze malware as well. We will discuss this topic more deeply in the near future...

Posted by Ilfak Guilfanov at 10:35 PM | Permalink

The second method to create a database with several PE files.

B. Use the debugger and take the memory snapshot

The built-in debugger in IDA has many interesting features - even the standard stuff like breakpoints can be used creatively (will cover them in the future). Today we will use the 'take memory snapshot' command to create a big IDB file.

Suppose you have a database with the .exe file loaded into it.

The first thing to do is to launch the debugger and to suspend the application as soon as the DLLs interesting us are loaded into the memory. The application entry point is a good candidate to stop at: position the cursor on the entry point and press F4 (this is the hotkey for the Run to cursor command). The debugger will create a breakpoint at the cursor position, start the application and let it run until the breakpoint.

The second step is to use the "Debugger, Take memory snapshot" command and take the snapshot of all segments. Do not forget to save the database to the disk (Ctrl-W) since the "take memory snapshot" command does not save it.

The method B is better than the method A but it has the following drawbacks:

• the database contains too much information (segments like ntdll.dll, kernel32.dll are always the same) and its size is around several hundred megabytes.
  
• there are stealth techniques to bypass the application entry point (like thread local storage callbacks), therefore stopping at the entry point does not guarantee that the application will not run out of control.
  

We will address these issues the next time.

Posted by Ilfak Guilfanov at 08:38 PM | Permalink

IDA Pro can load one PE file into a database and analyze it. Some users assume this is the maximum. Let's take a closer look at the situation...

Strictly speaking, this is true. The user interface allows you to load and analyze one file. There is the 'load additional binary file' command but it does not help much, especially if you are interested in loading formatted files, like Windows PE executables. Also IDA has only one 'exports' window, only one 'imports' window and many file attributes are stored as simple scalars (like the file size, initial stack pointer, etc). Imagine a situation when several DLLs refer to the same imported function - the function pointers in their .idata segments would need to have the same name but global names must be unique in the program. What will happen when you will ask to jump to an imported function, say, CreateFile - should IDA use the first function pointer in an .idata segment of the first DLL or the second pointer from the second .idata segment? Even nagivating in the disassembly would be more difficult - you would end up having several .text and .data segments, several copies of standard functions like printf(), memcpy(). There will be other problems as well but I think the above is enough to demonstrate that loading several files into a database has some problems which can be quite annoying.

However, if you are ready to live with these limitations and inconveniences, then you can create a database with several files in it. There are even several different methods if we talk about PE files.

A. The easiest method: load all referenced dlls at once.
Go to the user contributions page

http://www.datarescue.com/idabase/idadown.htm

and download the PE utilities by Atli Mar Gudmundsson. Unzip them to IDA\IDC subdirectory. With these scripts (fine IDC programming sample, by the way!), creating a database with all DLLs is a child's play:

• Create a database from the executable (.exe) file
  
• Run the pe_dlls.idc

BTW, these scripts can do many other things, just open them in a text editor and take some time to explore them.

Next time we will talk about the second method.

upd: fixed broken url

Posted by Ilfak Guilfanov at 12:18 AM | Permalink | Comments (2) | TrackBacks (1)

This is the first entry in the blog.

This is the message body. Seems to be easy.

Posted by Ilfak Guilfanov at 11:43 PM | Permalink | Comments (2)