jovanbulck / sgx-step Goto Github PK

Hello

I was trying to reproduce the project, however, I failed to build the project

I am using Ubuntu 18.04 and gcc7.3.0

This is the error message I got

foreshadow $ make
[===] Enclave [===]
[GEN] sgx_edger8r encl.edl
[CC]  encl_t.c (trusted edge)
[CC]  encl.c (core)
[LD]   encl.o encl_t.o -lsgx_trts encl.unsigned.so
[SGN] encl.unsigned.so
[CC]  encl_u.c (untrusted edge)
[AR]   libencl_proxy.a
[===] ../../libsgxstep [===]
[CC]  apic.c
[CC]  cpu.c
[CC]  debug.c
[CC]  enclave.c
[CC]  file.c
[CC]  foreshadow.c
[CC]  gdt.c
[CC]  idt.c
[CC]  pt.c
[CC]  sched.c
[CC]  spy.c
[AS]  aep_trampoline.S
[AS]  irq_entry.S
[AS]  rtm.S
[AS]  transient.S
[AR]  libsgx-step.a
[CC]  main.c
[LD] main.o -o app
/usr/bin/ld: ../../libsgxstep/libsgx-step.a(aep_trampoline.o): relocation R_X86_64_32S against symbol `sgx_step_tcs' can not be used when making a PIE object; recompile with -fPIC
/usr/bin/ld: final link failed: Nonrepresentable section on output
collect2: error: ld returned 1 exit status
Makefile:56: recipe for target 'app' failed
make: *** [app] Error 1

Have you experienced similar issues before?

Thanks!

Hi, I'm learning single-step technology, but I found that I sometimes(with high probability) get screen freeze once I run the attack( project bench), and thus I have to manually restart the machine. So far, I've tried on three different laptops which are all Ubuntu with 5.3.0-28-generic Kernel and they all have this problem.

Besides, once I load the sgx-step kernel, System always warn me that System program problem detected.

And in the case that I could successfully execute the program, I could achieve single-step perfectly.

Do you have any clue on this problem? Is it an universal issue or it might because of my kernel version or something else?

Thanks for your help !

Below is my cpu information:( Here I disabled 2 physical cores and disbaled the hyper-threading)

Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 2
On-line CPU(s) list: 0,1
Thread(s) per core: 1
Core(s) per socket: 2
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 142
Model name: Intel(R) Core(TM) i5-8265U CPU @ 1.60GHz
Stepping: 11
CPU MHz: 1391.847
CPU max MHz: 3900.0000
CPU min MHz: 400.0000
BogoMIPS: 3600.00
Virtualisation: VT-x
L1d cache: 32K
L1i cache: 32K
L2 cache: 256K
L3 cache: 6144K
NUMA node0 CPU(s): 0,1

In README.md:
Building and running

1. Build and load /dev/sgx-step
   approach: https://www.libcrack.so/2012/09/02/bypassing-devmem_is_allowed-with-kprobes/

This hyperlink shows:
404: Page Not Found

So how to solve the issue of preventing /dev/mem?

Hey! I enabled bios while booting my machine but when I run ./install_SGX_driver.sh , it shows the following error:
make -C /lib/modules/5.4.0-53-generic/build M=/home/shalabh/nice/sgx-step/kernel/linux-sgx-driver modules
make[1]: Entering directory '/usr/src/linux-headers-5.4.0-53-generic'
Building modules, stage 2.
MODPOST 1 modules
make[1]: Leaving directory '/usr/src/linux-headers-5.4.0-53-generic'
modprobe: ERROR: could not insert 'isgx': Operation not permitted

I checked in 'dev/' but could not find isgx. What could be the case? I checked again and SGX was enabled.
Thanks

I have been trying to learn and understand SGX step and Foreshadow and while I was experimenting with it, I commented out unmap_alias and sim_reload from the program and found it didn't really change much. The exploit still worked. Could I get more context as to why those functions are in the program?

dmesg warnings

[   45.321866] [sgx-step] listening on /dev/sgx-step
[   56.452386] x86/PAT: app:2512 map pfn RAM range req uncached-minus for [mem 0x481488000-0x481488fff], got write-back
[   56.452489] x86/PAT: app:2512 map pfn RAM range req uncached-minus for [mem 0x2810e6000-0x2810e6fff], got write-back
[   78.424127] x86/PAT: app:2672 map pfn RAM range req uncached-minus for [mem 0x481488000-0x481488fff], got write-back
[   78.424228] x86/PAT: app:2672 map pfn RAM range req uncached-minus for [mem 0x2810e6000-0x2810e6fff], got write-back

HW: NUC7JYJ
SW: Ubuntu Desktop 20.04

Reproducible steps:

1. install sgx-step following the wiki Building and Running - 0, 1, check by $ls /dev/sgx-step, OK
2. reboot os
3. check by $ls /dev/sgx-step again, NO

Is this normal?

Hi,

I'm having some trouble configuring the APIC timer interval and was hoping you could help.

lscpu output:

Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              12
On-line CPU(s) list: 0-11
Thread(s) per core:  2
Core(s) per socket:  6
Socket(s):           1
NUMA node(s):        1
Vendor ID:           GenuineIntel
CPU family:          6
Model:               158
Model name:          Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
Stepping:            10
CPU MHz:             2508.222
CPU max MHz:         4500.0000
CPU min MHz:         800.0000
BogoMIPS:            5199.98
Virtualization:      VT-x
L1d cache:           32K
L1i cache:           32K
L2 cache:            256K
L3 cache:            12288K
NUMA node0 CPU(s):   0-11
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb invpcid_single pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 bmi2 erms invpcid rdseed adx clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp md_clear flush_l1d

I'm running app/bench as follows:

cd app/bench
NUM=100 make parse

The contents of out.txt are:

--------------------------------------------------------------------------------
[main.c] Creating enclave...
--------------------------------------------------------------------------------

[sched.c] continuing on CPU 1
==== System Settings ====
    Pstate max perf pct: 100
    Pstate min perf pct: 57
    Turbo Boost:         0
    cpu pinning:         1
    Designated cpu:      1
    Running on cpu:      1
[pt.c] /dev/sgx-step opened!
==== Victim Enclave ====
    Base:   0x7fa4a3000000
    Size:   8388608
    Limit:  0x7fa4a3800000
    TCS:    0x7fa4a3471000
    SSA:    0x7fa4a3472f48
    AEP:    0x55b0d19c73d1
    EDBGRD: debug
[main.c] enclave trigger code adrs at 0x7fa4a3005000

[pt.c] /dev/mem opened!

--------------------------------------------------------------------------------
[main.c] Establishing user space APIC/IDT mappings
--------------------------------------------------------------------------------

[idt.c] DTR.base=0xfffffe0000000000/size=4095 (256 entries)
[idt.c] established user space IDT mapping at 0x7fa4a5226000
[idt.c] installed ring3 IRQ handler with target_rip=0x55b0d19c4450
[idt.c] IDT[ 45] @0x7fa4a52262d0 = 0x55b0d19c741a (seg sel 0x33); p=1; dpl=3; type=15; ist=0
[file.c] reading buffer from '/dev/cpu/1/msr' (size=8)
[apic.c] established local memory mapping for APIC_BASE=0xfee00000 at 0x7fa4a5225000
[apic.c] APIC_ID=2000000; LVTT=400ec; TDCR=0
[apic.c] APIC timer one-shot mode with division 2 (lvtt=2d/tdcr=0)

--------------------------------------------------------------------------------
[main.c] Triggering user space software interrupts
--------------------------------------------------------------------------------

[main.c] calling enclave: attack=1; num_runs=100; timer=20
[main.c] Caught fault 11! Restoring enclave page permissions..
[main.c] ^^ enclave RIP=0x5500; ACCESSED=1
[main.c] ^^ enclave RIP=0x5500; ACCESSED=0
[main.c] ^^ enclave RIP=0x5500; ACCESSED=0
[main.c] ^^ enclave RIP=0x5500; ACCESSED=0
[main.c] ^^ enclave RIP=0x5500; ACCESSED=0
[main.c] ^^ enclave RIP=0x5500; ACCESSED=0
[main.c] ^^ enclave RIP=0x5500; ACCESSED=0
[main.c] ^^ enclave RIP=0x5500; ACCESSED=0
[main.c] ^^ enclave RIP=0x5500; ACCESSED=0
[main.c] ^^ enclave RIP=0x5500; ACCESSED=0
[main.c] ^^ enclave RIP=0x5500; ACCESSED=0
[main.c] ^^ enclave RIP=0x5500; ACCESSED=0
[main.c] ^^ enclave RIP=0x5500; ACCESSED=0
...
[main.c] ^^ enclave RIP=0x5500; ACCESSED=0
[main.c] ^^ enclave RIP=0x5500; ACCESSED=0
[main.c] ^^ enclave RIP=0x5500; ACCESSED=0
[main.c] ^^ enclave RIP=0x5500; ACCESSED=0
[apic.c] Restored APIC_LVTT=400ec/TDCR=0)
[file.c] writing buffer to '/dev/cpu/1/msr' (size=8)

--------------------------------------------------------------------------------
[main.c] all done; counted 651 IRQs
--------------------------------------------------------------------------------

while the output of parse_nop.py is

parse_nop.py: found instruction slide at 0x5000 (length=100)
parse_nop.py counted tot= 0  one= 0  zero= 0  plus= 0

If my understanding is correct, at least one value in the line should be non-zero (assuming an interrupt landed while the enclave was running). I also find it strange that RIP value stays constant. Inspecting the python script, I found out that this value was always greater than INST_SLIDE_END. Also, tampering with the timer interval value didn't help much.

Any ideas?

When run cpl/idt/memcmp, os would always hang after a few minutes, and need to hardware-reboot by manual.
Is that normal?

May I know how can I execute the main code file here? Any help

To prevent system crashes, we should consider gracefully restoring the APIC timer to its default configuration when it was modified and the ASSERT macro subsequently terminates the application.

eg Travis-CI to at least make sure everything compiles on an up-to-date system. This means we should also patch, build, and install the SGX-SDK.

We can also consider running at least some tests from the app directory in the CI environment.

When I try to install SGX_driver, I get the following compilation errors.
This part should be changed to automatically choose whether to use vm_insert_pfn or vmf_insert_pfn, depending on your version of Linux Kernel.

error: implicit declaration of function ‘vm_insert_pfn’; did you mean ‘vmf_insert_pfn’? [-Werror=implicit-function-declaration]
rc = vm_insert_pfn(vma, entry->addr, PFN_DOWN(entry->epc_page->pa));
^~~~~~~~~~~~~
vmf_insert_pfn

error: implicit declaration of function ‘vm_insert_pfn’; did you mean ‘vmf_insert_pfn’? [-Werror=implicit-function-declaration]
ret = vm_insert_pfn(vma, encl_page->addr, PFN_DOWN(epc_page->pa));
^~~~~~~~~~~~~
vmf_insert_pfn

Starting with Kernel 4.20, vm_insert_pfn transitions to vmf_insert_pfn breaking backward compatibility.

http://web.ist.utl.pt/joao.leao.guerreiro/2019/04/setting-up-a-sgx-development-environment/

Thanks

I am trying to run app/memcmp and it tells me that the assertion that pwd_len < MAX_LEN is false. I have gotten foreshadow, aep-redirect, and bench to work. I can't figure out whats the issue here. I tested it unmodified and with Debug set to 1.

I did all the necessary steps present at [https://github.com//issues/24] for running the foreshadow app.

1. When i set 'LD_LIBRARY_PATH=/opt/intel/sgxsdk/lib64/ ldd ./app' to this i am getting this error when executing ./app

2. When i set 'LD_LIBRARY_PATH=/usr/lib/ ldd ./app' to this i am getting this error when executing ./app :

In both the cases i am not able to run ./app. What should i do to resolve this issue ?
Also i ran Sample codes present in this : sdk/intel-sdk/linux-sgx/SampleCode/ and i am able to run it.

Hey, i'm currently trying to build the 'foreshadow' app but when i run the makefile i get this error:

[===] Enclave [===]
[GEN] sgx_edger8r encl.edl
make[1]: sgx_edger8r: Command not found
Makefile:84: recipe for target 'edger' failed
make[1]: *** [edger] Error 127
Makefile:72: recipe for target 'build-Enclave' failed
make: *** [build-Enclave] Error 2

I edited the makefile inside the 'Enclave' folder by:

• adding the line:
  SGX_SDK ?= /opt/intel/sgxsdk
  (when i tried to print the value of SGX_SDK through "echo" i got blank output).

• changing the EDGER variable to be
  EDGER = $(SGX_SDK)/bin/x64/sgx_edger8r

the error above seemed to be solved but then i encountered a new one:
[===] Enclave [===]
[GEN] /opt/intel/sgxsdk/bin/x64/sgx_edger8r encl.edl
[CC] encl_t.c (trusted edge)
[CC] encl.c (core)
[LD] encl.o encl_t.o -lsgx_trts encl.unsigned.so
Makefile:51: recipe for target 'encl.so' failed
make[1]: *** [encl.so] Error 127
Makefile:72: recipe for target 'build-Enclave' failed
make: *** [build-Enclave] Error 2

i followed the installation instructions inside the README, would appreciate some help.

Thanks,
Bar.

the app directory should contain simple, self-contained example programs for all of the features of the framework

I'm starting to tweak Foreshadow and when I change fs_reload_threshold to t1-t2 or even t1-t2+60, I get varying results. What was the thought process behind making fs_reload_threshold = t1-t2-60?

Hi Researchers,

I am doing experiments using sgx-step. It is really strong. I follow the steps on the top README.md. But I encountered a strange problem related to file.c.

Environment

Cloud Vendor: Alibaba Cloud ECS Security-enhanced family (SGX-capable)
OS: Ubuntu 18.04 LTS
model name: Intel(R) Xeon(R) Platinum 8369B CPU @ 2.70GHz
bugs: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit

I just follow the steps in README to set up.

aep-redirect
I use the same command provided in README and everything is normal.

bench
I use the same command provided in README but I am getting this error:

# NUM=100 STRLEN=1 make parse
[===] Enclave [===]
[RM] encl.o asm.o asm_nop.o encl.unsigned.so encl.so libencl_proxy.a
[RM] encl_t.o encl_u.o encl_t.h encl_t.c encl_u.h encl_u.c
[===] ../../libsgxstep [===]
[RM] apic.o cpu.o debug.o enclave.o file.o foreshadow.o gdt.o idt.o pt.o sched.o spy.o aep_trampoline.o irq_entry.o rtm.o transient.o libsgx-step.a
[RM] main.o app
[===] Enclave [===]
[GEN] sgx_edger8r encl.edl
[CC]  encl_t.c (trusted edge)
[CC]  encl.c (core)
[AS]  asm.S (core)
[AS]  asm_nop.S (core)
[LD]   encl.o asm.o asm_nop.o encl_t.o -lsgx_trts encl.unsigned.so
[SGN] encl.unsigned.so
[CC]  encl_u.c (untrusted edge)
[AR]   libencl_proxy.a
[===] ../../libsgxstep [===]
[CC]  apic.c
[CC]  cpu.c
[CC]  debug.c
[CC]  enclave.c
[CC]  file.c
[CC]  foreshadow.c
[CC]  gdt.c
[CC]  idt.c
[CC]  pt.c
[CC]  sched.c
[CC]  spy.c
[AS]  aep_trampoline.S
[AS]  irq_entry.S
[AS]  rtm.S
[AS]  transient.S
[AR]  libsgx-step.a
[CC]  main.c
[LD] main.o -o app
[file.c] assertion '(f = fopen(path, "w"))' failed: No such file or directory
Aborted
Makefile:64: recipe for target 'run' failed
make: *** [run] Error 134

The same error reports when I try to run foreshadow and lvi as well. However, I never encounter it when running aep-redirect. What could be the problem and the solution?

Thank you very much.

Best regards

Ideally libsgxstep should be less tightly coupled to the Intel SGX-SDK and rely on a clean and stable interface that can be easily ported to other SDKs or libOSs, eg at least the following:

• set AEP by hooking EENTER to allow for custom AEP stub for single-stepping
• get TCS for current enclave to allow to retrieve metadata (eg base address) via the driver. This should be similarly possible by intercepting EENTER
• refactor build system to more easily link libsgxstep to other SDKs

At least the user-space SDK components should be easily pluggable. Integration with the kernel space isgx driver may be more tight, but that should be less of an issue as most SDKs rely on the isgx driver anyway..

Currently, the install script will always install to a system-wide location:

https://github.com/jovanbulck/sgx-step/blob/master/install_SGX_SDK.sh#L27

It would be nice to be able to specify a non-standard path e.g. through a cmdline param to the script.

Hi,

I just tried to run bench project with your latest version of sgx-step on my new laptop. And when I compile the bench project, I got following warning:
aep_trampoline.S: Assembler messages:
aep_trampoline.S:60 Warning: no instruction mnemonic suffix given and no register operands; using default for 'add'

And I also got same warning from irq_entry.S:36 .
Then my laptop was frozen and I had to restart my laptop.(This happens whenever I run the bench project)

So far, I have tried with it on two machines which had skylake and whisky model. Both systems are ubuntu 20.04.

Would it be the problem related to ubuntu 20 ? Could you gave me some suggestions on how to fix it ?

Thanks in advance.

Regards

Hey, we're currently trying to build the 'foreshadow' app but when we run the makefile we get this error:

[main.c] Creating enclave...
Error calling enclave at main.c:153 (rv=0x2006)
Makefile:48: recipe for target 'run' failed
make: *** [run] Aborted

We followed the instructions from the README.md of sgx-step and then executed make run in app/foreshadow.
This is a slight time-critical issue, @jovanbulck we'd appreciate it a lot if you could help us out. Thanks!

while performing make run in app/aep_redirect it shows sgx_get_aep undefined.
Tried the solution in issue #24 but error still exits.

Custom AEP trampoline seems to cause sgx-gdb to crash with a segmentation fault. Full output for the aep-redirect sample program:

GNU gdb (Ubuntu 7.11.1-0ubuntu1~16.5) 7.11.1
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Source directories searched: /opt/intel/sgxsdk/lib64/gdb-sgx-plugin:$cdir:$cwd
Setting environment variable "LD_PRELOAD" to null value.
Reading symbols from ./app...done.
(gdb) r
Starting program: /home/jo/write_buffer_writeup/code/sgx-step/app/aep-redirect/app 
detect urts is loaded, initializing
Function "random_stack_notify_gdb" not defined.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[main.c] Creating enclave...
add-symbol-file '/home/jo/write_buffer_writeup/code/sgx-step/app/aep-redirect/Enclave/encl.so' 0x7ffff5c00bd0 -readnow -s .interp 0x7ffff5c00238  -s .note.gnu.build-id 0x7ffff5c00254  -s .gnu.hash 0x7ffff5c00278  -s .dynsym 0x7ffff5c003d8  -s .dynstr 0x7ffff5c007c8  -s .rela.dyn 0x7ffff5c00a88  -s .plt 0x7ffff5c00bb0  -s .plt.got 0x7ffff5c00bc0  -s .nipx 0x7ffff5c12ca0  -s .rodata 0x7ffff5c134d0  -s .niprod 0x7ffff5c13c80  -s .eh_frame_hdr 0x7ffff5c144c0  -s .eh_frame 0x7ffff5c148a8  -s .fini_array 0x7ffff5e16e20  -s .data.rel.ro 0x7ffff5e16e40  -s .dynamic 0x7ffff5e16ea0  -s .got 0x7ffff5e16fe0  -s .got.plt 0x7ffff5e17000  -s .data 0x7ffff5e17020  -s .nipd 0x7ffff5e17904  -s .bss 0x7ffff5e18000 
==== Victim Enclave ====
[pt.c] /dev/sgx-step opened!
    Base:   0x7ffff5c00000
    Size:   4194304
    Limit:  0x7ffff6000000
    TCS:    0x7ffff5f7a000
    SSA:    0x7ffff5f7bf48
    AEP:    0x402c82
    EDBGRD: debug
[main.c] revoking a access rights..

Program received signal SIGSEGV, Segmentation fault.
0x00007fffffffd9c5 in ?? ()
(gdb) c
Continuing.

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff79a6269 in raise (sig=11) at ../sysdeps/unix/sysv/linux/pt-raise.c:35
35	../sysdeps/unix/sysv/linux/pt-raise.c: No such file or directory.
(gdb) c
Continuing.

Program terminated with signal SIGSEGV, Segmentation fault.
The program no longer exists.
(gdb) quit

After some digging it seems this is caused by the is_eresume() function of upstream SDK, which assumes the AEP is always a single ENCLU instruction. As such, sgx-gdb compatibility will probably require another SDK patch...

https://github.com/intel/linux-sgx/blob/master/sdk/debugger_interface/linux/se_ptrace.c#L368

static int is_eresume(pid_t pid, struct user_regs_struct *regs)
{
    unsigned int instr;

    if(!se_read_process_mem(pid, (void *)regs->REG(ip), (char *)&instr, sizeof(instr), NULL))
        return FALSE;
    if((ENCLU == (instr & 0xffffff))
            && (SE_ERESUME == regs->REG(ax)))
        return TRUE;
    return FALSE;
}
```

https://github.com/jovanbulck/0xbadc0de/blob/master/intel-sgx-sdk/sgx-strlen/main.c#L58

http://windows-internals.com/cet-on-windows/

Hi,

Currently sgx-step can only run with SGX oot-driver, does it have the capability to run on SGX in-kernel or dcap driver since those are the major drivers of SGX?
I tried sgx-step with SGX in-kernel/dcap driver, it failed to run...

1. Building error

For app/foreshadow/, run $make,
[===] Enclave [===]
[GEN] sgx_edger8r encl.edl
[CC] encl_t.c (trusted edge)
[CC] encl.c (core)
[LD] encl.o encl_t.o -lsgx_trts encl.unsigned.so
[SGN] encl.unsigned.so
make[1]: *** [Makefile:53: encl.so] Error 255
make: *** [Makefile:73: build-Enclave] Error 2

For app/lvi/, run $make,
[===] Enclave [===]
[GEN] sgx_edger8r encl.edl
[CC] encl_t.c (trusted edge)
[CC] encl.c (core)
[AS] asm.S (core)
[LD] encl.o asm.o encl_t.o -lsgx_trts encl.unsigned.so
[SGN] encl.unsigned.so
make[1]: *** [Makefile:53: encl.so] Error 255
make: *** [Makefile:66: build-Enclave] Error 2

2. Running error

For memcmp:
[main.c] assertion 'pwd_len < MAX_LEN' failed: Invalid argument
Aborted

hello, thank you for the answer on my previous issue.
i'm now facing a different one when running the Foreshadow app, the results that the attack reads are mostly 0, and on some runs it provides random values that are mostly incorrect.

is there anything i missed?
thanks,
Bar.

Hi, I am trying to run the single-step bench and sometimes encounter kernel bug especially when step over 1000s times. Here is an example from the kernel log:

[ 132.182650] BUG: unable to handle kernel paging request at 000055bb86c8b000
[ 132.182657] IP: 0x55bb86c8b000
[ 132.182658] PGD 80000007b65d0067 P4D 80000007b65d0067 PUD 7ad9f5067 PMD 7f8e31067 PTE 7bed47025
[ 132.182661] Oops: 0011 [#1] SMP PTI
[ 132.182663] Modules linked in: sgx_step(OE) msr thunderbolt rfcomm cmac snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic bnep intel_wmi_thunderbolt wmi_bmof arc4 intel_rapl iwlmvm x86_pkg_temp_thermal intel_powerclamp coretemp crct10dif_pclmul crc32_pclmul ghash_clmulni_intel mac80211 pcbc aesni_intel rtsx_pci_ms aes_x86_64 crypto_simd iwlwifi glue_helper memstick cryptd intel_cstate intel_rapl_perf btusb btrtl cfg80211 btbcm btintel joydev input_leds bluetooth ecdh_generic snd_hda_intel ir_rc6_decoder snd_hda_codec snd_hda_core snd_hwdep rc_rc6_mce snd_pcm ir_lirc_codec snd_seq_midi lirc_dev snd_seq_midi_event i915 snd_rawmidi ite_cir rc_core drm_kms_helper snd_seq video drm snd_seq_device snd_timer i2c_algo_bit fb_sys_fops syscopyarea acpi_pad sysfillrect mei_me snd sysimgblt wmi
[ 132.182690] mei mac_hid soundcore intel_pch_thermal sch_fq_codel binfmt_misc kvm_intel kvm isgx(OE) parport_pc ppdev lp parport ip_tables x_tables autofs4 hid_generic usbhid hid rtsx_pci_sdmmc ahci e1000e rtsx_pci libahci
[ 132.182699] CPU: 1 PID: 3739 Comm: app Tainted: G OE 4.15.18+ #3
[ 132.182700] Hardware name: Intel Corporation NUC7i7BNH/NUC7i7BNB, BIOS BNKBL357.86A.0062.2018.0222.1644 02/22/2018
[ 132.182701] RIP: 0010:0x55bb86c8b000
[ 132.182702] RSP: 0000:ffffaac644e87ee8 EFLAGS: 00010002
[ 132.182703] RAX: 0000000000000008 RBX: 0000000000000008 RCX: 0000000000000000
[ 132.182704] RDX: ffff932c01c80000 RSI: 0000000000000008 RDI: ffffaac644e87f58
[ 132.182704] RBP: ffffaac644e87f28 R08: 0000000000000000 R09: 0000000000000000
[ 132.182705] R10: 0000000000000000 R11: 0000000000000000 R12: ffffaac644e87f58
[ 132.182706] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 132.182707] FS: 00007f34f50e4b80(0000) GS:ffff932c01c80000(0000) knlGS:0000000000000000
[ 132.182708] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 132.182709] CR2: 000055bb86c8b000 CR3: 00000007acf9a001 CR4: 00000000000606e0
[ 132.182709] Call Trace:
[ 132.182713] ? exit_to_usermode_loop+0x4f/0xd0
[ 132.182715] prepare_exit_to_usermode+0x83/0x90
[ 132.182718] retint_user+0x8/0x8
[ 132.182719] RIP: 0033:0x55bb86c8a2fd
[ 132.182720] RSP: 002b:00007ffd60331b60 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff02
[ 132.182721] RAX: 0000000000000003 RBX: 00007f34f3a76000 RCX: 000055bb86c8a2fd
[ 132.182721] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000
[ 132.182722] RBP: 00007ffd60332050 R08: 0000000000000000 R09: 0000000000000000
[ 132.182723] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
[ 132.182723] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 132.182724] Code: Bad RIP value.
[ 132.182726] RIP: 0x55bb86c8b000 RSP: ffffaac644e87ee8
[ 132.182726] CR2: 000055bb86c8b000
[ 132.182728] ---[ end trace cad0a7670dc9a000 ]---
[ 132.182829] mm/pgtable-generic.c:40: bad pmd 00000000b3c05ac0(00000007b2884047)

Some info that may help to reproduce the bug:
commands: cd app/bench && NUM=10000 STRLEN=1 make run
kernel version: Ubuntu-4.15.0-135.139 (git://kernel.ubuntu.com/ubuntu/ubuntu-bionic.git)
cpu model: Intel(R) Core(TM) i7-7567U
kernel parameters: GRUB_CMDLINE_LINUX_DEFAULT="quiet splash nox2apic iomem=relaxed no_timer_check nosmep nosmap clearcpuid=514 isolcpus=1 nmi_watchdog=0"

Can you help to check and advise what is the potential causes of this? Thanks so much

Hi,
How could sgx-step support APIC periodic mode?

Shujie

SGX_STEP_PLATFORM and SGX_STEP_TIMER_INTERVAL and actual timer configuration AEP code should better be encapsulated in libsgxstep plus a common Makefile.config, such that the actual application AEP callback code can focus exclusively on gathering side-channel information.

Are there any plans on this front? (Readme)

A dynamic ELF parser library could easily return addresses (relative to the enclave base) for required enclave symbols (functions, data locations) etc. Together with get_enclave_base(), this way one does not have to either (i) get addresses manually using objdump or (ii) add special ecalls to get addresses at runtime for demo purposes.

Current libsgxstep implicitly assumes a single victim enclave in the host process. Functions in enclave.h should better be parameterized with a unique victim enclave identifier.

SGX driver has finally been upstreamed into recent Linux kernels, meaning the dependence on linux-sgx-driver (/dev/isgx) could eventually go, see:

https://lwn.net/Articles/840129/
https://www.kernel.org/doc/html/latest/x86/sgx.html

Should check how dev/sgx-step relies on the legacy /dev/isgx driver and ensure compatibility with the new upstreamed /dev/sgx_enclave driver (probably want to support both drivers for now).

In order to let user know what those test applicaions mainly do(test point)?

Hi there,
I installed SGX-step by following the steps, and tried to test the samples in /app, but I always get the following errors. How could I solve it? Thanks.
./app: symbol lookup error: ./app: undefined symbol: sgx_get_aep

Currently, page-fault attack code is rather unclean and has a lot of code duplication to setup fault handlers and revoke enclave page permissions through either mprotect calls and/or PTE remapping + editing..

Goal: Offer a simple and intuitive abstraction in libsgxstep/pf.c to mount page-fault controlled-channel attacks on enclaves so applications need only to focus on the actual attack logic (state machines) to interpret page-fault sequences.

I got the same error message " general protection fault: 0031 [#1] SMP PTI......." , as shown in #GP issue #6, when testing app/idt.
The original app/idt is fine. The error message shows when using periodical APIC timer mode, or when I move the "apic_timer_irq" within the for-loop from "main" to the "hello_world" handler.
The respective solution (247d921) for app/bench seems not applicable for app/idt.
Do you have any idea how to solve this issue for app/idt?

You might be curious why I did that.
I attempt to attack SGX-LKL with sgx-step. But SGX-LKL is not based on SGX SDK, and for some reason SGX-LKL doesn't support AEP stub registration freely.
So I cannot trigger interrupts in AEX handler as done in app/bench.
Instead, my idea is to set periodic interrupts, or trigger interrupts in userspace handler, e.g. "hello_world" in app/idt.
Do you have better ideas?

Currently, the Intel SGX-SDK patch and aep_trampoline.S use global variables to store the current TCS. This may not work in a multi-threaded environment (e.g., Gramine).

Best would be better to properly refactor to avoid the usage of a stored tcs variable in AEP interception in aep_trampoline.S and debug functionality in enclave.c.

After SGX-Step using the latest version v2.14, step 1 and 2 succeeded,
but step 3.Build and run test applications has an error:

$ cd app/aep-redirect
$ make run

...
: note: this is the location of the previous definition
[AS] aep_trampoline.S
[AS] irq_entry.S
[AS] rtm.S
[AS] transient.S
[AR] libsgx-step.a
[CC] main.c
[LD] main.o -o app
./app: symbol lookup error: ./app: undefined symbol: sgx_get_aep
make: *** [Makefile:45: run] Error 127

And also an error:
$ cd app/bench
$ NUM=100 STRLEN=1 make parse

...
: note: this is the location of the previous definition
[AS] aep_trampoline.S
[AS] irq_entry.S
[AS] rtm.S
[AS] transient.S
[AR] libsgx-step.a
[CC] main.c
[LD] main.o -o app
./app: symbol lookup error: ./app: undefined symbol: sgx_get_aep
make: *** [Makefile:64: run] Error 127

Current code only configures local APIC timer in TSC deadline mode, without actually scheduling an IRQ event. As such, kernel will never regain control of the CPU (unless maybe when /dev/sgx-step is released and the wrapped handler is explicitly called). See

$ watch -n 1 cat /proc/interrupts

Kernel should be put back in control of the local APIC timer by explicitly scheduling an IRQ with wrmsr IA32_TSC_DEADLINE_MSR via an IOCTL call to the /dev/sgx-step kernel driver.

Greetings,

Big fan of your work!

Just want to report that when I run app/bench with some SGXv2 machines, interesting things occurred. It said the following lines,

[main.c] Caught fault 11! Restoring enclave page permissions..
[main.c] assertion 'fault_cnt++ < 10'

However, this won't happen on machines that only support SGXv1.
Any thoughts?

FYI, all the experiments were conducted on a Ubuntu 18.04 with a Linux 4.15.0 kernel, with and without Turbo Boost enabled.

Thanks.

Keeping here for future reference (low priority): Current Makefiles contain a lot of unnecessary duplicated variables and config.

Better include a top-level Makefile.include file or so.

or even consider using cmake

hello
when i run app/aep-redirect, the error happen

idt.c] locking IRQ handler pages 0x558324224000/0x55832422a000 [main.c] Creating enclave... [main.c] Dry run to allocate pages [main.c] revoking data page access rights.. [main.c] data at 0x7f200d811190 with PTE: [pt.c] /dev/sgx-step opened! [pt.c] /dev/mem opened! +-------------------------------------------------------------------------------------------+ | XD | PK | IGN | RSVD | PHYS ADRS | IGN | G | PAT | D | A | PCD | PWT | U/S | R/W | P | | 1 | x | x | 0 | 0x00008097f000 | x | x | x | 0 | 1 | x | x | 1 | 1 | 1 | +-------------------------------------------------------------------------------------------+ +-------------------------------------------------------------------------------------------+ | XD | PK | IGN | RSVD | PHYS ADRS | IGN | G | PAT | D | A | PCD | PWT | U/S | R/W | P | | 0 | x | x | 1 | 0x007f7f680000 | x | x | x | 0 | 1 | x | x | 0 | 0 | 0 | +-------------------------------------------------------------------------------------------+ [main.c] revoking code page access rights.. [main.c] code at 0x7f200d803000 with PTE: +-------------------------------------------------------------------------------------------+ | XD | PK | IGN | RSVD | PHYS ADRS | IGN | G | PAT | D | A | PCD | PWT | U/S | R/W | P | | 0 | x | x | 0 | 0x000080971000 | x | x | x | 0 | 1 | x | x | 1 | 0 | 1 | +-------------------------------------------------------------------------------------------+ +-------------------------------------------------------------------------------------------+ | XD | PK | IGN | RSVD | PHYS ADRS | IGN | G | PAT | D | A | PCD | PWT | U/S | R/W | P | | 0 | x | x | 1 | 0x007f7f68e000 | x | x | x | 0 | 1 | x | x | 0 | 0 | 0 | +-------------------------------------------------------------------------------------------+ ==== Victim Enclave ==== [enclave.c] assertion 'victim.base && "no enclave found in /proc/self/maps"' failed: No such file or directory Aborted

How could I solve it? Thanks.

Hi,

I have been using sgx-step for half a year, and this tool could really help us understand how sgx and related attacks work. And I'm very curious about why we have to redirect our output to an extra file ?

For example, when I was running bench, I got good result when redirecting the output to out.txt but sometimes got a bad result(miss some string accesses) if I print the result directly to the terminal.

I know that printing things to the terminal will reduce the accuracy of attack result, but why ? Is this because printing things to terminal cost more cycles then writing things to a file and the longer cycles influence the accuracy of single-step ?

BTW, I remembered that you mentioned this phenomenon somewhere, but now I can't find it. If you still remember where you mentioned it, it will be very helpful.

Thanks for your help.

regards
Neo

A fully automated single-stepping timer configuration benchmark program would repeatedly interrupt an enclaved nop instruction slide, each time adjusting the current timer interval guess -- in a binary search fashion based on the previous in-enclave instruction pointer (zero-step vs. multi-step).

Trying to run ./install_SGX_SDK.sh but it says "E: Unable to locate package python2". I checked and I do have python 2.7 installed in "/usr/bin/python2". I removed "python2" from the installing prequisites part of ./install_SGX_SDK.sh and it ran, even saying that the SDK was successfully installed, but testing it out with the commands "cd app/idt" and then "make run", it gives the error " assertion '(fd_step = open("/dev/sgx-step", O_RDWR)) >= 0' failed: No such file or directory"