Syncing Ghidra with Gdb over QEMU about ret-sync

[bootleg]

Hi @iburres o/

I used a similar setup some times ago and it should be functional.

Do you have a full exception log ? Which version of gdb are you using ?

Regarding the issue /proc/mappings, in the setup the gdb plugin can not access to a map of the modules thus you should specify a fake/virtual context though the .sync configuration file as described in the README to help the syncing. Is that what you did ? If so could you post the context you're using ?

Also, using Ghidra/IDA should not have an impact regarding this.

[iburres]

We are using gdb 7.11. I read the README concerning the two .sync files, but there is no way for me to place a .sync file in QEMU while it is emulating the ISA, at least not that I know of. I changed Ghidra's server settings to use localhost:1234 and the settings in sync.py, but that also did not work. The sync function seems to be working, since it does recognize the PID (unless I am completely misunderstanding this), so I think the issue is with Python or QEMU.

I am not certain this is the best place to address my issue, so if you would rather me communicate via some other means please let me know. I do appreciate your time though.

[bootleg]

Sorry I was not clear, no need to put the .sync file within the QEMU system itself. When you attach gdb to the gdb stub you're exposing (it is the one returning pid 42000 by default right?), gdb will look for for a .sync file on you host system, in your home and near the sync.py plugin file itself, see:

ret-sync/ext_gdb/sync.py

Line 1019 in c3f2de1

locations = [os.path.abspath(os.path.dirname(__file__)),

I would imagine you're loading a raw module containing assembled instructions in your QEMU to debug the ISA ? (if not please correct me).

If so, what you could do is to use this kind of config in the .sync file loaded by your gdb:

    [INIT]
    context = {
          "pid": 200,
          "mappings": [ [0x10000, 0x18000, 0x8000, "raw_binary_isa"] ]
      }

I believe that is no need to configure Ghidra server (at least with regards to ret-sync behaviour). Just to make sure a file named "raw_binary_isa" is loaded in Ghidra to be matched thanks to the "fake" name of the module we put in the configuration file.

If you want there is a mail in the credits file otherwise here is fine for me.

[iburres]

Ok, I will try that out and let you know the outcome. And yes, we are loading a raw binary hex file in QEMU; however, QEMU is also emulating a very complicated platform that we have built, so it’s not your run of the mill VM. Ian Burres Associate Professor Palm Beach State College Department of Computer Science & Mathematics 3000 St Lucie Avenue S. Boca Raton, FL 33431 W: 561-393-7222

…

On Jul 17, 2020, at 9:54 AM, bootleg ***@***.***> wrote:  Sorry I was not clear, no need to put the .sync file within the QEMU system itself. When you attach gdb to the gdb stub you're exposing (it is the one returning pid 42000 by default right?), gdb will look for for a .sync file on you host system, in your home and near the sync.py plugin file itself, see: https://github.com/bootleg/ret-sync/blob/c3f2de1201f2a3ca51b7b9455f539e809835a4c9/ext_gdb/sync.py#L1019 I would imagine you're loading a raw module containing assembled instructions in your QEMU to debug the ISA ? (if not please correct me). If so, what you could do is to use this kind of config in the .sync file loaded by your gdb: [INIT] context = { "pid": 200, "mappings": [ [0x10000, 0x18000, 0x8000, "raw_binary_isa"] ] } I believe that is no need to configure Ghidra server (at least with regards to ret-sync behaviour). Just to make sure a file named "raw_binary_isa" is loaded in Ghidra to be matched thanks to the "fake" name of the module we put in the configuration file. If you want there is a mail in the credits file otherwise here is fine for me. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[Matheus-Garbelini]

UPDATE

Hi @bootleg Thanks for the help. It works with the remapping but there is a rebase issue when the plugin is rebasing from remote base to local base. For example, when going to 0x400d18c8 (base 400d0018), the plugin uses a wrong base (3f400020) and goes to 3f4018d0 instead. This happens because ghidra plugin always uses the local base which sometimes is not the same as the remote base. It seems the plugin only register a single local base, whereas the sync.py correctly uses multiple bases.
I've added some logs inside the function

The issue happens inside rebaseLocal(Address loc).
I've fixed it on the plugin side by forcibly changing the image base by calling rsplugin.setLocalBase(base); (new function)

public boolean parse(JSONObject sync) {
            boolean bExit = false;
            type = sync.getString("type");
            base = sync.optLong("base");
            offset = sync.optLong("offset");

            switch (type) {
            // location request, update program's listing/graph view
            case "loc":
                if (rsplugin.program == null) {
                    break;
                }

                rsplugin.clrs.cbColorPre();

                // rsplugin.program (from mod request) is set as current program
                if (!rsplugin.program.equals(rsplugin.pm.getCurrentProgram())) {
                    rsplugin.pm.setCurrentProgram(rsplugin.program);
                }
                rsplugin.cs.println(String.format("base:%d, offset:%d",base,offset));
                rsplugin.setLocalBase(base); // New function added, a proper fix would have this being called by a new command
                rsplugin.gotoLoc(base, offset);
                rsplugin.clrs.cbColorPost();
                break;

Here's my .sync for example:

[INIT]
context = {"pid": 200, "mappings": [ [0x3f400020, 0x3f40cebf, 0xcea0, "firmware.elf"], [0x400d0018, 0x4012c7f3, 0x5c7dc, "firmware.elf"], [0x3ffadb5c, 0x3ffadfff, 0x4a4, "esp32_rom.elf"], [0x40010000, 0x40064ef1, 0x54ef2, "esp32_rom.elf"] ]}

This means that a single local base cannot be applied when multiple maps are defined (code regions for flash and ROM). For this reason, I suggest adding a new command ("cbase") to the RequestHandler.java which sets the LocalBase. This would solve the mismatch between ghidra and gdb for embedded devices.

[iburres]

I managed to solve the issue I was having by using netstat to see which host and port Ghidra was operating on, which happened to be different from the information used in sync.py. It is working as intended now. Very nice program bootleg.

[bootleg]

Hi Matheus, Ian,

sorry for the delay I was out these last weeks. Thank you for all the feedbacks, I'll look into the rebasing issue it asap.

which happened to be different from the information used in sync.py

Could you tell me more about this ? By default all plugins should listen on port 9100; except if another port has been specified in a .sync configuration file

[Matheus-Garbelini]

Hi @bootleg the port was not changed on .sync. The only issue is that the ghidra plugin never changes its local base, so it was not possible to debug firmware with multiple code regions as defined in sync.py. In contrast, the script sync.py always sends the correct address and base to ghidra retsync plugin.
My .sync file is described below.

[INIT]
context = {"pid": 200, "mappings": [ [0x3f400020, 0x3f40cebf, 0xcea0, "firmware.elf"], [0x400d0018, 0x4012c7f3, 0x5c7dc, "firmware.elf"], [0x40080000, 0x400803ff, 0x3ff, "firmware.elf"], [0x40080400, 0x4009396c, 0x1356c, "firmware.elf"], [0x3ffadb5c, 0x3ffadfff, 0x4a4, "esp32_rom.elf"], [0x40010000, 0x40064ef1, 0x54ef2, "esp32_rom.elf"] ]}

For your convenience, I'm attaching the compiled plugin for ghidra 9.1.2:
ghidra_9.1.2_PUBLIC_20200827_retsync.zip

Also find attached, my source code changes. Sorry I didn't have time to prepare a repo to show a commit.
changes_retsync.zip

Steps I did for the forced workaround:

1. Define new function setLocalBase:

    void setLocalBase(long cbase) {
        imageBaseLocal = imageBaseLocal.getNewAddress(cbase);
    }

2. Forcebly and baddly call setLocalBase on the protocol handling for "loc" command:

case "loc":
          ... More code here ...
          rsplugin.cs.println(String.format("base:%d, offset:%d",base,offset));
          rsplugin.setLocalBase(base); // < ---- A proper fix would have this being called by a new command
          rsplugin.gotoLoc(base, offset);
          rsplugin.clrs.cbColorPost();
          break;

[bootleg]

Hi Matheus,

thanks again for the complement of information. I must admit I am bothered by this case.

By design, a debugger client should not care about the memory mapping of the program in a disassembler (IDA/Ghidra). That is kind of the initial idea behind ret-sync. I'd prefer not to break this assumption.

The issue here is that the debugger and the disassembler do not share the same view of the "program". Are firmware.elf and
esp32_rom.elf mapped into the same Ghidra program/memory map ? If so, would something like this be acceptable ? :

[INIT]
context = {"pid": 200, "mappings": [ 
 [0x3f400020, 0x40130000, 0xd2ffe0, "firmware.elf"]
 ]}

(I've rounded the top address to 0x40130000) The full range of addresses (over-approximation of your actual mapping) is redirected to the Ghidra program whose name is firmware.elf and each address is rebased with respect to the address of the first "segment/section"-like i.e. 0x3f400020.

Regarding the Ghidra port, could you tell me which one it was found listening on ?

Cheers,

[iburres]

Issue solved by determining the host and port associated with Ghidra's Java interface, which can easily be identified with netstat.