SGXv2 support [SOLVED: UMIP] about sgx-step

[jovanbulck]

Hi StanPlatinum,

Thanks for your interest in our work and thanks for reporting! This looks like a bug that shouldn't happen indeed.. Not sure about SGXv2, as I don't have immediate access to such a machine at the moment. Not sure what's going wrong exactly and if it's related to SGXv2 somehow.

A full log would help, but from the lines above it looks like something goes wrong in the unprotected application, even before the enclave is called. Since the aep_cb_func doesn't output things and hence is never called? What goes wrong I think is that there's a SEGFAULT for some reason in the unprotected application and then the handler is called which thinks the enclave faulted and tries to resume after restoring the PTE. This would trigger an infite loop where the handler is called again and again, until it terminates through the ASSERT.

I suggest the following to try and pinpoint the cause for the bug: can you improve the output of the handler to print at which address the fault occurs? That should be fairly straightforward eg as follows:

https://github.com/jovanbulck/sgx-tutorial-space18/blob/master/common/pf.c

Once you get the si->si_addr address, we can see what code is causing the fault. It would also be useful to provide an objdump of the relevant program addresses around that location to see what goes wrong.

I can try to make some time next week then to dive a bit deeper if necessary ^^

[StanPlatinum]

Hi jovanbulck,

Thanks for the reply!

I disabled ASLR.

Following your instructions, here is the added fault handler wrapper:

void fault_handler_wrapper(int signo, siginfo_t *si, void *ctx)
{
    void *base_adrs;
    ucontext_t *uc = (ucontext_t *)ctx;

    info("fault_handler_wrapper addr: %x\n", fault_handler_wrapper);
    switch (signo)
    {
    case SIGSEGV:
        base_adrs = si->si_addr;
        info("Caught page fault (base address=%p)", base_adrs);
        break;

    default:
        info("Caught unknown signal '%d'", signo);
        abort();
    }

    /* Mask lower PFN bits to simulate clearing by SGX hardware when executing
     the unprotected programs */
    //base_adrs = GET_PFN(base_adrs);

    //if (__fault_handler_cb)    __fault_handler_cb(base_adrs);
    *pte_encl = MARK_NOT_EXECUTE_DISABLE(*pte_encl);
    ASSERT(fault_cnt++ < 10);
}

Here is some output:

sgxstep@xxxxxx:~/sgx-step/app/bench$ sudo ./app 

--------------------------------------------------------------------------------
[main.c] Creating enclave...
--------------------------------------------------------------------------------

[sched.c] continuing on CPU 1
==== System Settings ====
    Pstate max perf pct: 100
    Pstate min perf pct: 53
    Turbo Boost:         0
    cpu pinning:         1
    Designated cpu:      1
    Running on cpu:      1
[main.c] WARNING: interactive terminal detected; known to cause 
[main.c] unstable timer intervals! Use stdout file redirection for 
[main.c] precise single-stepping results...
[pt.c] /dev/sgx-step opened!
==== Victim Enclave ====
    Base:   0x7ffff6000000
    Size:   8388608
    Limit:  0x7ffff6800000
    TCS:    0x7ffff6471000
    SSA:    0x7ffff6472f48
    AEP:    0x5555555586c7
    EDBGRD: debug
[main.c] enclave string adrs at 0x7ffff6210000
[pt.c] /dev/mem opened!
[main.c] enclave trigger code adrs at 0x7ffff6002000


--------------------------------------------------------------------------------
[main.c] Establishing user space APIC/IDT mappings
--------------------------------------------------------------------------------

[main.c] fault_handler_wrapper addr: 55555531

[main.c] Caught page fault (base address=0xd68)
[main.c] fault_handler_wrapper addr: 55555531

It seems the program had run into a weird/invalid place (0xd68)...
0xd68 is beyond the address space of the program.

[jovanbulck]

Thanks for the additional info. In the meantime, I managed to reproduce this behavior on a recent i5-1035G1 machine. I found the problem is not related to SGXv2 but to recent CPU support for User-Mode Instruction Prevention (UMIP) which disallows certain instructions like SIDT, used by libsgxstep, from user-space.

The fix should be fairly simple by adding the clearcpuid=514 kernel boot parameter to disable UMIP altogether. I updated the README with instructions. On my machine, bench idt and cpl all work as expected now ^^

[jovanbulck]

Hope this fixes your problem, let me know if you run into more troubles!

[StanPlatinum]

Thanks!

Sorry for the delay. We have disabled the UMIP successfully according your instructions. Then all bench idt cpl can work well.

FYI, lvi gets frozen after

[main.c] enclave_page_b at 0x7fc0dda0e000 w PTE
+-------------------------------------------------------------------------------------------+
| XD | PK | IGN | RSVD | PHYS ADRS      | IGN | G | PAT | D | A | PCD | PWT | U/S | R/W | P | 
| 0  | x  | x   | 0    | 0x0000703ef000 | x   | x | x   | 1 | 1 | x   | x   | 1   | 1   | 1 | 
+-------------------------------------------------------------------------------------------+

, and foreshadow goes with a very low accuracy since we cannot use TSX (inside SGX) on our machine.

BTW, we are curious how do you find this problem that is not related to SGX2 but with this UMIP feature? Thanks!
And we @ya0guang have listened to your presentation at the Intel Webinar today. Great talk!

[jovanbulck]

Thanks! Glad to hear the fix worked for you :) The UMIP problem became visible when running the idt app and then inspecting the instruction at faulting instruction pointer. Also dmesg outputted a warning. Generally, it's good to first try to run app/idt and app/cpl and app/aep-redirect to check that all basic SGX-Step features work accordingly ^^

Regarding LVI: when the program doesn't output anything after that line, that probably means your processor is immune to the LVI variant (LVI-SB) exploited in the lvi demo. (This also holds for the foreshadow app btw, if your processor has silicon and/or microcode mitigations, the test program won't work of course.) For reference, you can find a list of affected LVI processors here:

https://software.intel.com/security-software-guidance/insights/processors-affected-load-value-injection