can not build TaLoS using clang,need help about talos

[plaublin]

Hi.

Please copy/paste the error instead of taking a screenshot, as we cannot fully see the error message.

It looks like the compiler is trying to link with the standard C library (you can find the /usr/include/x86_64-linux-gnu/bits/stdio2.h line). This cannot work as enclaveshim_ocalls.c is part of the enclave, for which the libc is not entirely defined (and in particular printf). I do not know if this is due to the use of clang or if you have modified some other code that could trigger this error.

[jmp0x7c00]

Thanks for your quick reply!!!❤️

• my source files are same with repo

test@ubuntu16:~/TaLos$ git status
On branch master
Your branch is up-to-date with 'origin/master'.
nothing to commit, working directory clean

• install patch

test@ubuntu16:~/TaLos$ cd ${PROJECT_ROOT}/src/talos
test@ubuntu16:~/TaLos/src/talos$ ./patch_libressl.sh
patching file ./crypto/bio/bio_lib.c
patching file ./include/openssl/asn1.h
patching file ./crypto/engine/eng_all.c
patching file ./crypto/ecdsa/ecs_vrf.c
patching file ./include/openssl/bio.h
patching file ./crypto/evp/m_sha1.c
patching file ./crypto/evp/evp_key.c
patching file ./crypto/x509v3/v3_utl.c
patching file ./crypto/pem/pem_info.c
patching file ./crypto/objects/obj_dat.c
patching file ./crypto/pem/pem_pk8.c
patching file ./crypto/pkcs12/p12_crpt.c
patching file ./crypto/hmac/hm_ameth.c
patching file ./crypto/sha/sha256.c
patching file ./crypto/pem/pvkfmt.c
patching file ./crypto/objects/obj_lib.c
patching file ./crypto/ripemd/rmd_one.c
patching file ./crypto/ec/ec_lib.c
patching file ./ssl/s3_lib.c
patching file ./crypto/bn/bn_lib.c
patching file ./include/openssl/dh.h
patching file ./crypto/x509/by_dir.c
patching file ./crypto/dso/dso_dlfcn.c
patching file ./crypto/pkcs12/p12_key.c
patching file ./crypto/asn1/a_int.c
patching file ./crypto/mem_clr.c
patching file ./crypto/compat/getentropy_osx.c
patching file ./crypto/bio/bss_file.c
patching file ./crypto/md5/md5_one.c
patching file ./crypto/pem/pem_all.c
patching file ./crypto/pem/pem_seal.c
patching file ./crypto/gost/streebog.c
patching file ./crypto/pkcs12/p12_decr.c
patching file ./include/openssl/x509v3.h
patching file ./crypto/asn1/asn1_lib.c
patching file ./crypto/x509/x_all.c
patching file ./ssl/ssl_locl.h
patching file ./crypto/pem/pem_x509.c
patching file ./crypto/asn1/a_time_tm.c
patching file ./crypto/asn1/ameth_lib.c
patching file ./ssl/s3_srvr.c
patching file ./crypto/asn1/n_pkey.c
patching file ./crypto/ecdh/ech_lib.c
patching file ./crypto/stack/stack.c
patching file ./crypto/conf/conf_def.c
patching file ./crypto/bn/bn_rand.c
patching file ./ssl/s3_pkt.c
patching file ./crypto/evp/e_idea.c
patching file ./crypto/x509v3/v3_enum.c
patching file ./crypto/evp/p_open.c
patching file ./crypto/hmac/hm_pmeth.c
patching file ./include/openssl/conf.h
patching file ./crypto/x509/x509_trs.c
patching file ./crypto/asn1/x_x509a.c
patching file ./crypto/rand/randfile.c
patching file ./crypto/asn1/p8_pkey.c
patching file ./crypto/conf/conf_mod.c
patching file ./crypto/rsa/rsa_eay.c
patching file ./ssl/ssl_lib.c
patching file ./crypto/ui/ui_openssl.c
patching file ./ssl/bs_cbs.c
patching file ./crypto/buffer/buffer.c
patching file ./include/openssl/objects.h
patching file ./crypto/compat/getentropy_linux.c
patching file ./crypto/evp/bio_enc.c
patching file ./crypto/evp/p5_crpt2.c
patching file ./include/openssl/bn.h
patching file ./crypto/gost/gost2814789.c
patching file ./crypto/ocsp/ocsp_lib.c
patching file ./crypto/conf/conf_sap.c
patching file ./include/openssl/stack.h
patching file ./crypto/asn1/x_name.c
patching file ./ssl/t1_meth.c
patching file ./crypto/compat/getentropy_aix.c
patching file ./crypto/pem/pem_pkey.c
patching file ./crypto/engine/eng_lib.c
patching file ./crypto/ecdsa/ecs_lib.c
patching file ./include/openssl/tls1.h
patching file ./include/openssl/err.h
patching file ./ssl/d1_lib.c
patching file ./crypto/asn1/tasn_fre.c
patching file ./crypto/x509v3/v3_genn.c
patching file ./crypto/compat/arc4random.h
patching file ./crypto/conf/conf_mall.c
patching file ./crypto/x509v3/v3_bcons.c
patching file ./ssl/ssl_stat.c
patching file ./crypto/err/err.c
patching file ./include/openssl/ssl.h
patching file ./ssl/ssl_err2.c
patching file ./crypto/evp/p_lib.c
patching file ./crypto/bio/b_sock.c
patching file ./ssl/t1_srvr.c
patching file ./crypto/asn1/a_object.c
patching file ./crypto/ui/ui_util.c
patching file ./ssl/t1_lib.c
patching file ./crypto/err/err_all.c
patching file ./crypto/evp/p5_crpt.c
patching file ./crypto/x509/x509_set.c
patching file ./crypto/bio/bss_conn.c
patching file ./crypto/asn1/a_bitstr.c
patching file ./crypto/modes/gcm128.c
patching file ./crypto/sha/sha512.c
patching file ./crypto/malloc-wrapper.c
patching file ./crypto/x509/x509_cmp.c
patching file ./crypto/x509v3/v3_purp.c
patching file ./ssl/s3_clnt.c
patching file ./ssl/t1_enc.c
patching file ./crypto/evp/e_aes.c
patching file ./include/openssl/pem.h
patching file ./crypto/ts/ts_conf.c
patching file ./crypto/buffer/buf_str.c
patching file ./crypto/evp/e_aes_cbc_hmac_sha1.c
patching file ./ssl/t1_clnt.c
patching file ./crypto/asn1/a_verify.c
patching file ./crypto/evp/evp_lib.c
patching file ./crypto/pem/pem_xaux.c
patching file ./crypto/asn1/asn_mime.c
patching file ./include/openssl/x509.h
patching file ./crypto/bio/bss_dgram.c
patching file ./crypto/ui/ui_lib.c
patching file ./include/openssl/evp.h
patching file ./crypto/evp/evp_enc.c
patching file ./crypto/evp/digest.c
patching file ./crypto/ui/ui_openssl_win.c
patching file ./crypto/bn/bn_print.c
patching file ./crypto/x509/x509_ext.c
patching file ./crypto/x509/x509name.c
patching file ./crypto/engine/eng_openssl.c
patching file ./crypto/txt_db/txt_db.c
patching file ./crypto/md4/md4_one.c
patching file ./ssl/ssl_sess.c
patching file ./crypto/x509/x509_obj.c
patching file ./crypto/dsa/dsa_asn1.c
patching file ./include/openssl/md5.h
patching file ./crypto/evp/c_all.c
patching file ./crypto/cversion.c
patching file ./crypto/x509/x509_vpm.c
patching file ./crypto/x509/x509_vfy.c
patching file ./crypto/gost/gostr341001_key.c
patching file ./ssl/ssl_algs.c
patching file ./crypto/dh/dh_lib.c
patching file ./crypto/compat/getentropy_hpux.c
patching file ./crypto/bn/bn_asm.c
patching file ./crypto/des/str2key.c
patching file ./crypto/rsa/rsa_sign.c
patching file ./crypto/compat/arc4random.c
patching file ./crypto/evp/names.c
patching file ./crypto/evp/e_chacha20poly1305.c
patching file ./crypto/pem/pem_lib.c
patching file ./crypto/md5/md5_dgst.c
patching file ./crypto/sha/sha1_one.c
patching file ./crypto/pkcs7/pk7_doit.c
patching file ./crypto/gost/gostr341194.c
patching file ./crypto/rsa/rsa_saos.c
patching file ./crypto/ec/ec_mult.c
patching file ./crypto/asn1/x_x509.c
patching file ./crypto/md32_common.h
patching file ./ssl/ssl_rsa.c
patching file ./include/openssl/ec.h
patching file ./crypto/aes/aes_wrap.c
patching file ./ssl/ssl_asn1.c
patching file ./crypto/bio/bss_acpt.c
patching file ./ssl/ssl_ciph.c
patching file ./include/openssl/engine.h
patching file ./crypto/compat/getentropy_solaris.c
patching file ./crypto/cmac/cmac.c
patching file ./crypto/dso/dso_lib.c
patching file ./ssl/ssl_cert.c
patching file ./crypto/asn1/a_sign.c
patching file ./crypto/bn/bn_exp.c
patching file ./crypto/bio/b_print.c
patching file ./include/openssl/crypto.h
patching file ./crypto/cryptlib.c
patching file ./crypto/ec/ec_key.c
test@ubuntu16:~/TaLos/src/talos$ cd ${PROJECT_ROOT}/src/libressl-2.4.1/crypto
test@ubuntu16:~/TaLos/src/libressl-2.4.1/crypto$ cp Makefile.sgx Makefile.sgx.bk

• modify Makefile.sgx
  * change gcc to clang-8
  * remove -Werror[if not, compilation will terminate due to unused -I flags warning when compiling .i files to .o files ]

test@ubuntu16:~/TaLos/src/libressl-2.4.1/crypto$ cp Makefile.sgx  Makefile.sgx.bk
test@ubuntu16:~/TaLos/src/libressl-2.4.1/crypto$ vim Makefile.sgx
test@ubuntu16:~/TaLos/src/libressl-2.4.1/crypto$ diff Makefile.sgx Makefile.sgx.bk
15,17c15,17
< CC=clang-8
< CCASFLAGS=-g -O2 -Wall  -std=gnu99 -fno-strict-aliasing -fno-strict-overflow -D_FORTIFY_SOURCE=2 -fstack-protector-all -DHAVE_GNU_STACK -Wno-implicit-function-declaration
< CFLAGS=-g -O2 -Wall  -std=gnu99 -fno-strict-aliasing -fno-strict-overflow -D_FORTIFY_SOURCE=2 -fstack-protector-all -DHAVE_GNU_STACK -Wno-pointer-sign
---
> CC=gcc
> CCASFLAGS=-g -O2 -Wall -Werror -std=gnu99 -fno-strict-aliasing -fno-strict-overflow -D_FORTIFY_SOURCE=2 -fstack-protector-all -DHAVE_GNU_STACK -Wno-implicit-function-declaration
> CFLAGS=-g -O2 -Wall -Werror -std=gnu99 -fno-strict-aliasing -fno-strict-overflow -D_FORTIFY_SOURCE=2 -fstack-protector-all -DHAVE_GNU_STACK -Wno-pointer-sign
115c115
< 	@$(CC) -g -O2 -I../include -fPIC -DPIC  -c $< -o $@
---
> 	@$(CC) -g -O2 -I../include -fPIC -DPIC -Werror -c $< -o $@

• start building

test@ubuntu16:~/TaLos/src/libressl-2.4.1/crypto$ make -f Makefile.sgx
GEN  =>  enclave_u.c
CC   <=  enclave_u.c
CC   <=  hashmap.c
clang-8 -m64 -g -O2 -fPIC -Wno-attributes -Wno-implicit-function-declaration  -DNDEBUG -UEDEBUG -UDEBUG -UCOMPILE_WITH_INTEL_SGX -fPIC -c -o ecall_queue-nosgx.o ecall_queue.c
CC   <=  ecall_queue.c
CC   <=  enclaveshim_ecalls.c
CC   <=  ocalls.c
ar cru libenclave.a enclave_u.o hashmap-nosgx.o ecall_queue-nosgx.o enclaveshim_ecalls.o ocalls.o cpuid-elf-x86_64-ocall.o
ar: `u' modifier ignored since `D' is the default (see `U')
clang-8 -fPIC -shared -o libenclave.so enclave_u.o hashmap-nosgx.o ecall_queue-nosgx.o enclaveshim_ecalls.o ocalls.o cpuid-elf-x86_64-ocall.o -L/home/test/Downloads/sgxsdk/lib64 -lsgx_urts_sim 
GEN  =>  enclave_t.c
CC   <=  enclave_t.c
clang: warning: argument unused during compilation: '-I ../include' [-Wunused-command-line-argument]
clang: warning: argument unused during compilation: '-I .' [-Wunused-command-line-argument]
clang: warning: argument unused during compilation: '-I ../include' [-Wunused-command-line-argument]
clang: warning: argument unused during compilation: '-I /home/test/Downloads/sgxsdk/include' [-Wunused-command-line-argument]
clang: warning: argument unused during compilation: '-I /home/test/Downloads/sgxsdk/include/tlibc' [-Wunused-command-line-argument]
clang: warning: argument unused during compilation: '-I /home/test/Downloads/sgxsdk/include/libcxx' [-Wunused-command-line-argument]
clang: warning: argument unused during compilation: '-I /home/test/Downloads/sgxsdk/include' [-Wunused-command-line-argument]
In file included from enclaveshim_ocalls.c:45:
In file included from ./enclave_t.h:10:
./openssl_types.h:254:34: warning: redefinition of typedef 'CRYPTO_EX_DATA' is a C11 feature [-Wtypedef-redefinition]
typedef struct crypto_ex_data_st CRYPTO_EX_DATA;
                                 ^
../include/openssl/ossl_typ.h:177:34: note: previous definition is here
typedef struct crypto_ex_data_st CRYPTO_EX_DATA;
                                 ^
In file included from enclaveshim_ocalls.c:45:
In file included from ./enclave_t.h:10:
./openssl_types.h:277:26: warning: redefinition of typedef 'BIGNUM' is a C11 feature [-Wtypedef-redefinition]
typedef struct bignum_st BIGNUM;
                         ^
../include/openssl/ossl_typ.h:102:26: note: previous definition is here
typedef struct bignum_st BIGNUM;
                         ^
In file included from enclaveshim_ocalls.c:45:
In file included from ./enclave_t.h:10:
./openssl_types.h:595:13: warning: redefinition of typedef 'ASN1_BOOLEAN' is a C11 feature [-Wtypedef-redefinition]
typedef int ASN1_BOOLEAN;
            ^
../include/openssl/ossl_typ.h:77:13: note: previous definition is here
typedef int ASN1_BOOLEAN;
            ^
In file included from enclaveshim_ocalls.c:45:
In file included from ./enclave_t.h:10:
./openssl_types.h:695:30: warning: redefinition of typedef 'EVP_MD_CTX' is a C11 feature [-Wtypedef-redefinition]
typedef struct env_md_ctx_st EVP_MD_CTX;
                             ^
../include/openssl/ossl_typ.h:114:30: note: previous definition is here
typedef struct env_md_ctx_st EVP_MD_CTX;
                             ^
enclaveshim_ocalls.c:668:19: error: expected parameter declarator
int __printf_chk (2 - 1, const char* format, ...) {
                  ^
enclaveshim_ocalls.c:668:19: error: expected ')'
enclaveshim_ocalls.c:668:18: note: to match this '('
int __printf_chk (2 - 1, const char* format, ...) {
                 ^
enclaveshim_ocalls.c:668:5: error: conflicting types for '__printf_chk'
int __printf_chk (2 - 1, const char* format, ...) {
    ^
/usr/include/x86_64-linux-gnu/bits/stdio2.h:87:12: note: previous declaration is here
extern int __printf_chk (int __flag, const char *__restrict __format, ...);
           ^
enclaveshim_ocalls.c:671:25: error: use of undeclared identifier 'format'
 __builtin_va_start(ap, format);
                        ^
enclaveshim_ocalls.c:672:31: error: use of undeclared identifier 'format'
 int r = vsnprintf(buf, 8192, format, ap);
                              ^
enclaveshim_ocalls.c:709:34: error: expected parameter declarator
int __asprintf_chk (char **strp, 2 - 1, const char *fmt, ...) {
                                 ^
enclaveshim_ocalls.c:709:34: error: expected ')'
enclaveshim_ocalls.c:709:20: note: to match this '('
int __asprintf_chk (char **strp, 2 - 1, const char *fmt, ...) {
                   ^
enclaveshim_ocalls.c:709:5: error: conflicting types for '__asprintf_chk'
int __asprintf_chk (char **strp, 2 - 1, const char *fmt, ...) {
    ^
/usr/include/x86_64-linux-gnu/bits/stdio2.h:158:12: note: previous declaration is here
extern int __asprintf_chk (char **__restrict __ptr, int __flag,
           ^
enclaveshim_ocalls.c:711:25: error: use of undeclared identifier 'fmt'
 __builtin_va_start(ap, fmt);
                        ^
enclaveshim_ocalls.c:712:29: error: use of undeclared identifier 'fmt'
 int r = my_vasprintf(strp, fmt, ap);
                            ^
enclaveshim_ocalls.c:717:41: error: expected parameter declarator
int __builtin___sprintf_chk (char *str, 2 - 1, __builtin_object_size (char *str, 2 > 1), const char *format, ...) {
                                        ^
enclaveshim_ocalls.c:717:41: error: expected ')'
enclaveshim_ocalls.c:717:29: note: to match this '('
int __builtin___sprintf_chk (char *str, 2 - 1, __builtin_object_size (char *str, 2 > 1), const char *format, ...) {
                            ^
enclaveshim_ocalls.c:717:5: error: conflicting types for '__builtin___sprintf_chk'
int __builtin___sprintf_chk (char *str, 2 - 1, __builtin_object_size (char *str, 2 > 1), const char *format, ...) {
    ^
enclaveshim_ocalls.c:717:5: note: '__builtin___sprintf_chk' is a builtin with type 'int (char *, int, unsigned long, const char *, ...)'
enclaveshim_ocalls.c:717:5: error: definition of builtin function '__builtin___sprintf_chk'
int __builtin___sprintf_chk (char *str, 2 - 1, __builtin_object_size (char *str, 2 > 1), const char *format, ...) {
    ^
enclaveshim_ocalls.c:720:25: error: use of undeclared identifier 'format'
 __builtin_va_start(ap, format);
                        ^
enclaveshim_ocalls.c:721:23: error: use of undeclared identifier 'format'
 vsnprintf(buf, 8192, format, ap);
                      ^
enclaveshim_ocalls.c:1059:34: error: expected parameter declarator
int __fprintf_chk (FILE *stream, 2 - 1, const char *format, ...) {
                                 ^
enclaveshim_ocalls.c:1059:34: error: expected ')'
enclaveshim_ocalls.c:1059:19: note: to match this '('
int __fprintf_chk (FILE *stream, 2 - 1, const char *format, ...) {
                  ^
enclaveshim_ocalls.c:1059:5: error: conflicting types for '__fprintf_chk'
int __fprintf_chk (FILE *stream, 2 - 1, const char *format, ...) {
    ^
/usr/include/x86_64-linux-gnu/bits/stdio2.h:85:12: note: previous declaration is here
extern int __fprintf_chk (FILE *__restrict __stream, int __flag,
           ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
4 warnings and 20 errors generated.
Makefile.sgx:782: recipe for target 'enclaveshim_ocalls.o' failed
make: *** [enclaveshim_ocalls.o] Error 1

[plaublin]

These lines don't make sense: why would the compiler ignore the SGX includes when compiling an SGX application?
Does it compile with gcc?

clang: warning: argument unused during compilation: '-I ../include' [-Wunused-command-line-argument]
clang: warning: argument unused during compilation: '-I .' [-Wunused-command-line-argument]
clang: warning: argument unused during compilation: '-I ../include' [-Wunused-command-line-argument]
clang: warning: argument unused during compilation: '-I /home/test/Downloads/sgxsdk/include' [-Wunused-command-line-argument]
clang: warning: argument unused during compilation: '-I /home/test/Downloads/sgxsdk/include/tlibc' [-Wunused-command-line-argument]
clang: warning: argument unused during compilation: '-I /home/test/Downloads/sgxsdk/include/libcxx' [-Wunused-command-line-argument]
clang: warning: argument unused during compilation: '-I /home/test/Downloads/sgxsdk/include' [-Wunused-command-line-argument]

[jmp0x7c00]

These warnings are reported when generating .o files from .i files, I think the -I option may only make sense during the precompile phase (ie *.c to *.i)

Compile normally under gcc

[jmp0x7c00]

when I change compiler back to gcc, compilation is OK

test@ubuntu16:~/TaLos/src/libressl-2.4.1/crypto$ make clean -f Makefile.
make: Makefile.: No such file or directory
make: *** No rule to make target 'Makefile.'.  Stop.
test@ubuntu16:~/TaLos/src/libressl-2.4.1/crypto$ make clean -f Makefile.sgx
rm -rf \.libs
rm -rf *\.la
find . -iname "enclave_u.?" -delete
find . -iname "enclave_t.?" -delete
find . -iname "enclave.so" -delete
find . -iname "enclave.signed.so" -delete
find . -iname "*\.dirstamp" -delete
find . -iname "*\.o" -delete
find ../ssl -iname "*\.o" -delete
find . -iname "*\.i" -delete
find ../ssl -iname "*\.i" -delete
find . -iname "*\.deps" -delete
find . -iname "*\.libs" -delete
find . -iname "libenclave.a" -delete
find . -iname "libenclave.so" -delete
test@ubuntu16:~/TaLos/src/libressl-2.4.1/crypto$ make
make: *** No targets specified and no makefile found.  Stop.
test@ubuntu16:~/TaLos/src/libressl-2.4.1/crypto$ make -f Makefile.sgx
GEN  =>  enclave_u.c
CC   <=  enclave_u.c
CC   <=  hashmap.c
gcc -m64 -g -O2 -fPIC -Wno-attributes -Wno-implicit-function-declaration  -DNDEBUG -UEDEBUG -UDEBUG -UCOMPILE_WITH_INTEL_SGX -fPIC -c -o ecall_queue-nosgx.o ecall_queue.c
CC   <=  ecall_queue.c
CC   <=  enclaveshim_ecalls.c
CC   <=  ocalls.c
ar cru libenclave.a enclave_u.o hashmap-nosgx.o ecall_queue-nosgx.o enclaveshim_ecalls.o ocalls.o cpuid-elf-x86_64-ocall.o
ar: `u' modifier ignored since `D' is the default (see `U')
gcc -fPIC -shared -o libenclave.so enclave_u.o hashmap-nosgx.o ecall_queue-nosgx.o enclaveshim_ecalls.o ocalls.o cpuid-elf-x86_64-ocall.o -L/home/test/Downloads/sgxsdk/lib64 -lsgx_urts_sim 
GEN  =>  enclave_t.c
CC   <=  enclave_t.c
CC  <=  enclaveshim_ocalls.c
CC  <=  tls_processing_interface.c
CC  <=  ecall_queue.c
CC  <=  mpmc_queue.c
CC  <=  lthread.c
CC  <=  lthread_sched.c
CC  <=  mempool.c
LINK =>  enclave.so
<!-- Please refer to User's Guide for the explanation of each field -->
<EnclaveConfiguration>
    <ProdID>0</ProdID>
    <ISVSVN>0</ISVSVN>
    <StackMaxSize>0x40000</StackMaxSize>
    <HeapMaxSize>0x4600000</HeapMaxSize>
    <TCSNum>50</TCSNum>
    <TCSPolicy>1</TCSPolicy>
    <DisableDebug>0</DisableDebug>
    <MiscSelect>0</MiscSelect>
    <MiscMask>0xFFFFFFFF</MiscMask>
</EnclaveConfiguration>
tcs_num 50, tcs_max_num 50, tcs_min_pool 1
The required memory is 89571328B.
Succeed.
SIGN =>  enclave.signed.so

[jmp0x7c00]

OK, maybe there are some subtle differences in precompile phase between gcc and clang.

I only used gcc in the preprocessing of enclaveshim_ocalls.c, and clang for the rest, and finally passed the compilation successfully.

CC=clang-8
...

enclaveshim_ocalls.o: enclaveshim_ocalls.c enclave_t.c enclaveshim_ocalls.h
	$(eval srcbasefile=$(basename $@))
        gcc $(CFLAGS) -I../include -I$(SGX_SDK)/include -E -o $(srcbasefile).i $<
	@$(CC) $(CFLAGS) -I../include $(Enclave_C_Flags) -I$(SGX_SDK)/include -fPIC -DPIC -c -o $@ $(srcbasefile).i
	@echo "CC  <=  $<"

...