bug: Blight journal is empty about mate

[F1o0T]

Thanks for your time a lot, that was really helpful. Since it is midnight here already; I will run now, and tomorrow I will check why cmake is not generating Makefiles with $(CC). I will keep you posted here!, enjoy your rest of the day :)

[thinkmoore]

By default, compilation takes place in a docker image that does not have access to your local environment and cmake is not present. Two workarounds are:

1. Edit the makefile to install the necessary dependencies before the step that breaks
2. Use the more detailed mate-cli workflow to specify a container to use for the build (https://galoisinc.github.io/MATE/cli-overview.html#a-basic-workflow-with-mate-cli, https://galoisinc.github.io/MATE/cli.html#mate-cli-compile-create)

Unfortunately, the documentation for the latter is currently lacking...

[F1o0T]

Hi @thinkmoore,

Thanks for your reply, I have modified the Makefile to install cmake, and now it is able to see cmake, but I think command mate-cli is also still not mounting the sourcr directory into the container.

Now I get the error:

STDERR:
CMake Error: The source directory "/home/user/projects/testMate/exampleProject" does not exist.
Specify --help for usage, or press the help button on the CMake GUI.
make: *** [Makefile:4933: cmake_check_build_system] Error 1

May I ask if you can give some hints about how to use the more detailed mate-cli workflow to specify a container to use for the build, since as you said, the link you shared doesn't have enough information about that.

[F1o0T]

I have fixed the error of not being able to find the source directoy by mouning that directory into the container using the docker-compose.yml file by adding an entry to the executer under the volume section.

Now, I am facing another issue, my project is compiling successfully, but the build command is not showing any output. I understand that the build command is building the CPG graph. However, it is not showing anything.

Is it maybe because I am compiling for riscv not x86?

Thanks in advance for your support.

[thinkmoore]

0 builds completed successfully makes me wonder if something went wrong with the build interposition and we're not seeing any bitcode files. Can you run the first command here https://galoisinc.github.io/MATE/debugging-builds.html#debugging-a-build and see if there are any bitcode_artifact entries?

You might also check docker service logs mate_executor_1 for errors.

[F1o0T]

Here is the output of docker container logs mate-executer-one

 [2023-07-02 11:12:37,969: INFO/MainProcess] Task mate.tasks.compile.compile_artifact[8449e833-5b2c-4549-942b-54dab6120080] received
 [2023-07-02 11:12:37,970: DEBUG/MainProcess] TaskPool: Apply <function fast_trace_task at 0x7fb71ade7700> (args:('mate.tasks.compile.compile_artifact', '8449e833-5b2c-4549-942b-54dab6120080', {'lang': 'py', 'task': 'mate.tasks.compile.compile_artifact', 'id': '8449e833-5b2c-4549-942b-54dab6120080', 'shadow': None, 'eta': None, 'expires': None, 'group': None, 'group_index': None, 'retries': 0, 'timelimit': [None, None], 'root_id': '8449e833-5b2c-4549-942b-54dab6120080', 'parent_id': None, 'argsrepr': "('d0a1495933d7400a88cab6310bec5c96', CompileOptions(testbed=None, containerized=False, experimental_embed_bitcode=False, docker_image=None, containerized_infer_build=True, make_targets=None, extra_compiler_flags=[]))", 'kwargsrepr': "{'compilation_id': 'fa0403f986a74d4984d9949918e245b9'}", 'origin': 'gen29@6fb04f90469d', 'ignore_result': False, 'properties': {'content_type': 'application/x-python-serialize', 'content_encoding': 'binary', 'application_headers': {'lang': 'py', 'task': 'mate.tasks.compile.compile_artifact', 'id': '8449e833-5b2c-4549-942b-54dab6120080', 'shadow': None, 'eta': None, 'expires': None, 'group':... kwargs:{})
 [2023-07-02 11:12:37,994: DEBUG/ForkPoolWorker-8] http://storage:9000 "GET /artifacts/d0a1495933d7400a88cab6310bec5c96 HTTP/1.1" 200 272442946
 [2023-07-02 11:12:42,450: DEBUG/ForkPoolWorker-8] starting at /tmp/tmpobg741j0/s400; searching ["I have intentially deleted file paths here since I am using it on proprietary codebase"]
 [2023-07-02 11:12:42,450: DEBUG/ForkPoolWorker-8] /tmp/tmpobg741j0/exampleProject does not contain a Makefile; checking the next layer
 [2023-07-02 11:12:42,451: DEBUG/ForkPoolWorker-8] found a makefile in /tmp/tmpobg741j0/exampleProject/build
 [2023-07-02 11:12:42,451: DEBUG/ForkPoolWorker-8] running args=['blight-exec', '--swizzle-path', '--guess-wrapped', '--', 'make']
 [2023-07-02 11:12:42,616: DEBUG/ForkPoolWorker-8] http://storage:9000 "PUT /artifacts/084ed03d58a5421199e5737ef10402a6 HTTP/1.1" 200 0
 [2023-07-02 11:12:42,617: ERROR/ForkPoolWorker-8]
 You have encountered a bug in MATE's implementation. Please report it.

 Additional information:
 artifact: given an empty fileobj to upload self.uuid='e601f9e652194a53872c44820731f976' self.kind=<ArtifactKind.BlightJournal: 'blight:journal'>
 [2023-07-02 11:12:42,621: DEBUG/ForkPoolWorker-8] http://storage:9000 "PUT /artifacts/e601f9e652194a53872c44820731f976 HTTP/1.1" 200 0
 [2023-07-02 11:12:42,892: INFO/ForkPoolWorker-8] Task mate.tasks.compile.compile_artifact[8449e833-5b2c-4549-942b-54dab6120080] succeeded in 4.921525783999641s: None

When I compare that log output with the log output of the notes.c provided example, it seems that the get-bc command is not called since an error occured while executing blight-exec.

[langston-barrett]

Here is that assertion.

[F1o0T]

Hi @langston-barrett,

I am still not sure how to work-around that issue, I have prepared a very simple cmake/make based project with the notes.c source file only. It seems that Mate/blight is not able to intercept the build commands so that it creates bitcode.

[woodruffw]

@xreinheitx could you check which version of blight you're running with? It's unlikely, but this may have been a regression on my part (blight is maintained externally, and has undergone changes since MATE was last released).

[F1o0T]

@woodruffw , I think Mate is using blight which is shipped with the container ghcr.io/galoisinc/mate-dist:main, I am not sure how can I get the version of that one.

[thinkmoore]

I wonder if the problem is that CMAKE is hardcoding a path for cc, etc. and so blight's PATH-based interposition isn't working...

[matteasu]

@xreinheitx you can run a shell on that specific container and then check the version using pip list
https://kodekloud.com/blog/docker-exec/

[F1o0T]

Hi @matteasu,

Thanks for the hint, the version is 0.0.41

root@7f4a8e195da9:~# pip list | grep blight
blight                 0.0.41

[F1o0T]

@thinkmoore, FYI, I grepped grep -r '$(CC)' . the build dir that CMake used to store the Makefiles, and I couldn't see that the CC is used!

[woodruffw]

Thanks for the hint, the version is 0.0.41

Thanks for checking -- that version is pretty old, which is good in this case: it's one that was known to work with MATE 🙂

Per what @thinkmoore said: this suggests something weird about the build itself, since CMake should respect CC, CXX, etc. in the blight-instrumented environment. I don't remember the invocation off the top of my head, but I seem to recall there being a mate-cli invocation that'll dumb the logs for each compilation attempt, including any environment variables. That might be a good place to check next.

[F1o0T]

@woodruffw

Do you mean a mate-cli compile option?

mate-cli compile --help
usage: mate-cli compile [-h] {create,get} ...

positional arguments:
  {create,get}
    create      create a compilation
    get         list compilations

options:
  -h, --help    show this help message and exit

And again here is the log that I got using docker container logs mate-executer-1

 [2023-07-10 16:07:28,683: INFO/MainProcess] Task mate.tasks.compile.compile_artifact[01499b65-5038-4c56-abf6-ec3c1fc492dc] received
 [2023-07-10 16:07:28,684: DEBUG/MainProcess] TaskPool: Apply <function fast_trace_task at 0x7fcb0bad0700> (args:('mate.tasks.compile.compile_artifact', '01499b65-5038-4c56-abf6-ec3c1fc492dc', {'lang': 'py', 'task': 'mate.tasks.compile.compile_artifact', 'id': '01499b65-5038-4c56-abf6-ec3c1fc492dc', 'shadow': None, 'eta': None, 'expires': None, 'group': None, 'group_index': None, 'retries': 0, 'timelimit': [None, None], 'root_id': '01499b65-5038-4c56-abf6-ec3c1fc492dc', 'parent_id': None, 'argsrepr': "('aaf0467e840e47af9b3e591f5dfe268b', CompileOptions(testbed=None, containerized=False, experimental_embed_bitcode=False, docker_image=None, containerized_infer_build=True, make_targets=None, extra_compiler_flags=[]))", 'kwargsrepr': "{'compilation_id': '270bcb6c99ac41f78133945ee879050e'}", 'origin': 'gen28@c312ff637c87', 'ignore_result': False, 'properties': {'content_type': 'application/x-python-serialize', 'content_encoding': 'binary', 'application_headers': {'lang': 'py', 'task': 'mate.tasks.compile.compile_artifact', 'id': '01499b65-5038-4c56-abf6-ec3c1fc492dc', 'shadow': None, 'eta': None, 'expires': None, 'group':... kwargs:{})
 [2023-07-10 16:07:28,748: DEBUG/ForkPoolWorker-8] http://storage:9000 "GET /artifacts/aaf0467e840e47af9b3e591f5dfe268b HTTP/1.1" 200 33992
 [2023-07-10 16:07:28,786: DEBUG/ForkPoolWorker-8] starting at /tmp/tmpdcsmn_kn/mateExampleCode; searching [PosixPath('/tmp/tmpdcsmn_kn/mateExampleCode/CMakeLists.txt'), PosixPath('/tmp/tmpdcsmn_kn/mateExampleCode/build'), PosixPath('/tmp/tmpdcsmn_kn/mateExampleCode/notes.c')]
 [2023-07-10 16:07:28,786: DEBUG/ForkPoolWorker-8] /tmp/tmpdcsmn_kn/mateExampleCode does not contain a Makefile; checking the next layer
 [2023-07-10 16:07:28,787: DEBUG/ForkPoolWorker-8] found a makefile in /tmp/tmpdcsmn_kn/mateExampleCode/build
 [2023-07-10 16:07:28,788: DEBUG/ForkPoolWorker-8] running args=['blight-exec', '--swizzle-path', '--guess-wrapped', '--', 'make']
 [2023-07-10 16:07:31,678: DEBUG/MainProcess] heartbeat_tick : for connection 7f8209c98a5a4af9983be79384c1cd0c
 [2023-07-10 16:07:31,678: DEBUG/MainProcess] heartbeat_tick : Prev sent/recv: 313/33461, now - 314/33494, monotonic - 80052.570986458, last_heartbeat_sent - 80052.570985486, heartbeat int. - 60 for connection 7f8209c98a5a4af9983be79384c1cd0c
 [2023-07-10 16:07:31,800: DEBUG/ForkPoolWorker-8] http://storage:9000 "PUT /artifacts/d3c6ed0bc0f341edbab6c0e777628cca HTTP/1.1" 200 0
 [2023-07-10 16:07:31,801: ERROR/ForkPoolWorker-8]
 You have encountered a bug in MATE's implementation. Please report it.

 Additional information:
 artifact: given an empty fileobj to upload self.uuid='5c221f594d554389bc9aa5b843249db1' self.kind=<ArtifactKind.BlightJournal: 'blight:journal'>

[woodruffw]

Do you mean a mate-cli compile option?

I believe there's a mate-cli artifact subcommand that you can use to interact with different outputs produced by the compilation, including the dedicated blight logs.

[F1o0T]

@woodruffw I will search for that option.

@thinkmoore, I have tested Mate against an only make simple project (not a cmake-based) with one source file and a Makefile, the Makefile in this case is using $(CC) explicitly. I still see the same issue.

[F1o0T]

@woodruffw, I am trying to figure out which option could help me to get more insights with mate-cli artifact, it is not clear. I have usedmate-cli artifact get <ID>but then I got no useful information about what went wrong.

mate-cli artifact --help
usage: mate-cli artifact [-h] {create,dump,get} ...

positional arguments:
  {create,dump,get}
    create           create an artifact
    dump             dump the contents of an artifact
    get              list artifacts

options:
  -h, --help         show this help message and exit

mate-cli artifact get 779dc3b34343471fbfb7a932f12e1b2f
[
    {
        "artifact_id": "779dc3b34343471fbfb7a932f12e1b2f",
        "attributes": {
            "filename": "tmprot_7u_x.tar.gz"
        },
        "build_ids": [],
        "compilation_ids": [
            "d116ac61577d44cf801ee2bbfa77af3b"
        ],
        "has_object": true,
        "kind": "compile-target:tarball"
    }
]

[woodruffw]

@xreinheitx try this:

mate-cli artifact get --kind compile-output:compile-log

that should dump one or more artifact IDs, which you can then do this work:

mate-cli artifact dump SOME-ID-HERE

[F1o0T]

Hi @woodruffw, thanks a lot for the hint.

Here is the ouput of the dump command for the artifact, I still don't see where is the problem!

mate-cli artifact dump 6e8849874e994aba8779262817201195

=======================
EXITED WITH: 0
=======================

=======================
STDOUT:
blight-cc test1.c -o test1.c.o

=======================

=======================
STDERR:
INFO:Entering CC [test1.c -o test1.c.o -g3 -grecord-gcc-switches]
DEBUG:Compile using parsed arguments:
InputList:         [test1.c -o test1.c.o -g3 -grecord-gcc-switches]
InputFiles:        [test1.c]
ObjectFiles:       []
OutputFilename:    test1.c.o
CompileArgs:       [-g3 -grecord-gcc-switches]
LinkArgs:          []
ForbiddenFlags:    []
IsVerbose:         false
IsDependencyOnly:  false
IsPreprocessOnly:  false
IsAssembleOnly:    false
IsAssembly:        false
IsCompileOnly:     false
IsEmitLLVM:        false
IsLTO:             false
IsPrintOnly:       false

DEBUG:buildObjectFile: [-g3 -grecord-gcc-switches test1.c -c -o .test1.c.o]
DEBUG:Calling execCmd(clang, [test1.c -o test1.c.o -g3 -grecord-gcc-switches])
DEBUG:execCmd: clang [-g3 -grecord-gcc-switches test1.c -c -o .test1.c.o] had exitCode 0
DEBUG:execCmd: clang [-g3 -grecord-gcc-switches -emit-llvm -c test1.c -o .test1.c.o.bc] had exitCode 0
DEBUG:execCmd: clang [test1.c -o test1.c.o -g3 -grecord-gcc-switches] had exitCode 0
DEBUG:attachBitcodePathToObject recognized .o as something it can inject into.
DEBUG:execCmd: objcopy [--add-section .llvm_bc=/tmp/gllvm232942747 .test1.c.o] had exitCode 0
DEBUG:execCmd: clang [.test1.c.o -o test1.c.o] had exitCode 0
INFO:LINKING: clang [.test1.c.o -o test1.c.o]
DEBUG:Calling [/opt/mate/libexec/gclang test1.c -o test1.c.o -g3 -grecord-gcc-switches] returned 0

=======================

[woodruffw]

Hmm -- that looks pretty reasonable, although I don't actually see where the final object file link step occurs: it produces test1.c.o, but doesn't appear to detect test1? Maybe the linker is being invoked manually (rather than through the C compiler), and that's confusing the instrumentation?

[F1o0T]

but doesn't appear to detect test1

I think I confused you because during this troubleshooting, I modified the Makefile to be very simple and to have an explicit CC. Let me list here what I had for the last output.

Here is the source file test1.c
int main() { return 0; }

Here is the current Makefile

all:
	$(CC) test1.c -o test1.c.o

Here is the directory content

>_  ls simpleMakeExample/
Makefile  test1.c

Here is mate-cli executed command
mate-cli oneshot simpleMakeExample

And I got compile-log using your provided instructions above.

mate-cli artifact dump 6e8849874e994aba8779262817201195

=======================
EXITED WITH: 0
=======================

=======================
STDOUT:
blight-cc test1.c -o test1.c.o

=======================

=======================
STDERR:
INFO:Entering CC [test1.c -o test1.c.o -g3 -grecord-gcc-switches]
DEBUG:Compile using parsed arguments:
InputList:         [test1.c -o test1.c.o -g3 -grecord-gcc-switches]
InputFiles:        [test1.c]
ObjectFiles:       []
OutputFilename:    test1.c.o
CompileArgs:       [-g3 -grecord-gcc-switches]
LinkArgs:          []
ForbiddenFlags:    []
IsVerbose:         false
IsDependencyOnly:  false
IsPreprocessOnly:  false
IsAssembleOnly:    false
IsAssembly:        false
IsCompileOnly:     false
IsEmitLLVM:        false
IsLTO:             false
IsPrintOnly:       false

DEBUG:buildObjectFile: [-g3 -grecord-gcc-switches test1.c -c -o .test1.c.o]
DEBUG:Calling execCmd(clang, [test1.c -o test1.c.o -g3 -grecord-gcc-switches])
DEBUG:execCmd: clang [-g3 -grecord-gcc-switches test1.c -c -o .test1.c.o] had exitCode 0
DEBUG:execCmd: clang [-g3 -grecord-gcc-switches -emit-llvm -c test1.c -o .test1.c.o.bc] had exitCode 0
DEBUG:execCmd: clang [test1.c -o test1.c.o -g3 -grecord-gcc-switches] had exitCode 0
DEBUG:attachBitcodePathToObject recognized .o as something it can inject into.
DEBUG:execCmd: objcopy [--add-section .llvm_bc=/tmp/gllvm232942747 .test1.c.o] had exitCode 0
DEBUG:execCmd: clang [.test1.c.o -o test1.c.o] had exitCode 0
INFO:LINKING: clang [.test1.c.o -o test1.c.o]
DEBUG:Calling [/opt/mate/libexec/gclang test1.c -o test1.c.o -g3 -grecord-gcc-switches] returned 0

=======================

[F1o0T]

@woodruffw With that, I still get 0 build completed successfully.

 mate-cli oneshot simpleMakeExample                                                                                                                                                    
✔ 6825f6b64c5a40d8b7fbab9f78a99c75: compiled
✔ 0 builds completed successfully

[woodruffw]

Okay, so you're intentionally not producing an executable, right? Just an object file?

In that case, this doesn't work by design: MATE expects a compilation step to produce one or more executables, since the CPG build process fundamentally requires a main or similar entrypoint.

That being said, it's strange that it fails when you do produce a full executable. My only guess is that the linker is somehow being invoked separately, in a way that blight/GLLVM can't detect.

[F1o0T]

Hi @woodruffw,

In the last example in my last message the make file I used is actually generating an executable with an entry point, it is just the name is test1.c.o, but it is an executable which has an entry point main. And still Mate is unable to build the CPG.

all:
	$(CC) test1.c -o test1.c.o

[woodruffw]

Ah, got it. Just confirming: you see the same behavior when you generate the output file as test1 without the .o suffix, right?

[F1o0T]

Interesting enought to say that when I removed .o from the executable name, it worked!

all:
	$(CC) test1.c -o test1

[F1o0T]

Thenaks @woodruffw for your time! Now we know it worked with that example make project when I removed the .o suffix.

Now, when I get back to the simple Cmake-based project, as you know cmake's job is to automatically generate Makefiles that can build the targets mentioned in the CMakeLists.txt file.

Now the problem is that when I checked the cmake generated Makefiles, I found that cmake is always adding .o at the end of the generated object file before finally doing the linking.

/usr/bin/cc $(C_DEFINES) $(C_INCLUDES) $(C_FLAGS) -MD -MT CMakeFiles/mateExampleCode.elf.dir/notes.c.o -MF CMakeFiles/mateExampleCode.elf.dir/notes.c.o.d -o CMakeFiles/mateExampleCode.elf.dir/notes.c.o -c /home/abdelaziz/projects/testMate/mateExampleCode/notes.c

Here is how the linking is done.

/usr/bin/cc CMakeFiles/mateExampleCode.elf.dir/notes.c.o -o mateExampleCode.elf
With that I still have the 0 build with the simple CMake project.

mate-cli oneshot mateCMakeExampleCode
✔ 8315dbfc5c054b43993f85079605e6f2: compiled
✔ 0 builds completed successfully
[]

I have modified the Makefiles so that it doesn't have .o suffix, and I run mate-cli again, still I got 0 builds

 mate-cli oneshot mateCMakeExampleCode
✔ 8784bedd6c334623811baef6ebfdebc5: compiled
✔ 0 builds completed successfully
[]

And here is the compile-log output.

mate-cli artifact dump efc16c943e70420e96d57fde3c0d854f                            

=======================
EXITED WITH: 0
=======================

=======================
STDOUT:
[ 50%] Building C object CMakeFiles/mateExampleCode.elf.dir/notes
[100%] Linking C executable mateExampleCode.elf
[100%] Built target mateExampleCode.elf

=======================

=======================
STDERR:

=======================

[woodruffw]

No problem! Glad to see we're making progress 🙂

The .elf suffix might also be confusing the instrumentation -- are you able to tweak the build to remove that as well?

[F1o0T]

Unfortuantly, this didn't help.

Dropped .elf suffix from the final executable as followes:

/usr/bin/cc -rdynamic CMakeFiles/mateExampleCode.elf.dir/notes -o mateExampleCode 

And still it didn't work, here is the compile-log.

mate-cli artifact dump feaace818597482cb159a081af265101

=======================
EXITED WITH: 0
=======================

=======================
STDOUT:
[ 50%] Building C object CMakeFiles/mateExampleCode.elf.dir/notes
[100%] Linking C executable mateExampleCode
[100%] Built target mateExampleCode

=======================

=======================
STDERR:

=======================

And again here is the docker container log mate-executer-1

mate-executor-1            | [2023-07-11 21:10:55,797: INFO/MainProcess] Task mate.tasks.compile.compile_artifact[f233333f-320d-46cf-b3a8-08c1ef3c0426] received
mate-executor-1            | [2023-07-11 21:10:55,798: DEBUG/MainProcess] TaskPool: Apply <function fast_trace_task at 0x7fcb0bad0700> (args:('mate.tasks.compile.compile_artifact', 'f233333f-320d-46cf-b3a8-08c1ef3c0426', {'lang': 'py', 'task': 'mate.tasks.compile.compile_artifact', 'id': 'f233333f-320d-46cf-b3a8-08c1ef3c0426', 'shadow': None, 'eta': None, 'expires': None, 'group': None, 'group_index': None, 'retries': 0, 'timelimit': [None, None], 'root_id': 'f233333f-320d-46cf-b3a8-08c1ef3c0426', 'parent_id': None, 'argsrepr': "('f5b0c6f7cbf541d8800fd2d3078aaef9', CompileOptions(testbed=None, containerized=False, experimental_embed_bitcode=False, docker_image=None, containerized_infer_build=True, make_targets=None, extra_compiler_flags=[]))", 'kwargsrepr': "{'compilation_id': 'd726950941e44049b38cc92283ecba3c'}", 'origin': 'gen28@c312ff637c87', 'ignore_result': False, 'properties': {'content_type': 'application/x-python-serialize', 'content_encoding': 'binary', 'application_headers': {'lang': 'py', 'task': 'mate.tasks.compile.compile_artifact', 'id': 'f233333f-320d-46cf-b3a8-08c1ef3c0426', 'shadow': None, 'eta': None, 'expires': None, 'group':... kwargs:{})
mate-server-1              | INFO:     172.18.0.1:50838 - "POST /api/v1/compilations HTTP/1.1" 201 Created
mate-executor-1            | [2023-07-11 21:10:55,856: DEBUG/ForkPoolWorker-8] http://storage:9000 "GET /artifacts/f5b0c6f7cbf541d8800fd2d3078aaef9 HTTP/1.1" 200 44950
mate-executor-1            | [2023-07-11 21:10:55,899: DEBUG/ForkPoolWorker-8] starting at /tmp/tmpiv9vuzq8/mateCMakeExampleCode; searching [PosixPath('/tmp/tmpiv9vuzq8/mateCMakeExampleCode/CMakeLists.txt'), PosixPath('/tmp/tmpiv9vuzq8/mateCMakeExampleCode/build'), PosixPath('/tmp/tmpiv9vuzq8/mateCMakeExampleCode/notes.c')]
mate-executor-1            | [2023-07-11 21:10:55,899: DEBUG/ForkPoolWorker-8] /tmp/tmpiv9vuzq8/mateCMakeExampleCode does not contain a Makefile; checking the next layer
mate-executor-1            | [2023-07-11 21:10:55,900: DEBUG/ForkPoolWorker-8] found a makefile in /tmp/tmpiv9vuzq8/mateCMakeExampleCode/build
mate-executor-1            | [2023-07-11 21:10:55,901: DEBUG/ForkPoolWorker-8] running args=['blight-exec', '--swizzle-path', '--guess-wrapped', '--', 'make']
mate-executor-1            | [2023-07-11 21:10:56,264: DEBUG/ForkPoolWorker-8] http://storage:9000 "PUT /artifacts/feaace818597482cb159a081af265101 HTTP/1.1" 200 0
mate-executor-1            | [2023-07-11 21:10:56,265: ERROR/ForkPoolWorker-8]
mate-executor-1            | You have encountered a bug in MATE's implementation. Please report it.
mate-executor-1            |
mate-executor-1            | Additional information:
mate-executor-1            | artifact: given an empty fileobj to upload self.uuid='adbb5a5bb64d42cfb47a9eabb12bc137' self.kind=<ArtifactKind.BlightJournal: 'blight:journal'>

[woodruffw]

Oh well.

Is CMake calling /usr/bin/cc explicitly? That means it isn't honoring CC, which is strange.

[F1o0T]

I have replaced every /usr/bin/cc with $(CC) and it worked!, thanks a lot :)
But I have no idea why CMake is not doing that itself! why is it using /usr/bin/cc instead of $(CC)!

BTW, I have added the suffix .o and .elf back now while having $(CC) instead of /usr/bin/cc, and it worked. But Mate wasn't really telling where exactly the issue untill we went through this process of renaming.

[woodruffw]

I have replaced every /usr/bin/cc with $(CC) and it worked!, thanks a lot :)

Glad to hear it!

But I have no idea why CMake is not doing that itself! why is it using /usr/bin/cc instead of $(CC)!

Yeah, me neither. It could be an environmental step I missed, some default or assumption that's changed, etc. We unfortunately didn't make this part of MATE super easy to debug 😅