sourcegraph / lsif-cpp

Language Server Index Format (LSIF) generator for C++

C/C++ LSIF indexer

See sourcegraph/lsif-clang.

lsif-cpp's Issues

Update Dependencies (Renovate Bot)

This master issue contains a list of Renovate updates and their statuses.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• Pin dependencies (@types/glob, @types/jest, @types/lodash, @types/node, @types/parsimmon, @types/source-map-support, @types/verror)
• Update ubuntu:14.04.3 Docker digest to 2febbce
• Update dependency @types/lodash to v4.14.140
• Update dependency @types/node to v12.7.8
• Update dependency ts-jest to ^24.1.0
• Update dependency tslint to ^5.20.0
• Update dependency typescript to ^3.6.3
• Update ubuntu Docker tag to v14.04.5
• Update dependency tsc-watch to v4
• Check this option to rebase all the above open PRs at once

---

Advanced

• Check this box to trigger a request for Renovate to run again on this repository

Incorrect package IDs

Upon uploading a dump for https://github.com/michaelforney/samurai

Job failed to complete: Unknown packageInformation 'package:unknown'.

Reported in https://sourcegraph.slack.com/archives/CHXHX7XAS/p1574142326447700

Add ENV override to specify clang version

generate-csv should have a way to override the clang command in case a different version is required. This occurred on a system with LLVM 10 as the default, so LLVM 9 was specified by overriding the LLVM_CONFIG when building via make. When running generate-csv, the system defaulted to clang 10, causing a version conflict when loading the module.

This is likely an uncommon setup, so ok to keep this on the backlog.

lsif-cpp and android kernel

Hi,

Following up from sourcegraph/sourcegraph#9, trying to get useful lsif data from the android kernel.

I did the following:

1. I cloned the wahoo-pie kernel build tree from android.googlesource.com, using repo:

mkdir wahoo-pie
cd wahoo-pie
repo init -u https://android.golsoglesource.com/kernel/manifest -b android-msm-wahoo-4.4-pie-qpr2
repo sync

2. I cloned the latest android clang prebuilts from android.googlesource.com:

git clone https://android.googlesource.com/platform/prebuilts/clang/host/linux-x86 aosp-clang

3. I patched the build.config file (in the wahoo-pie root) to point it to my aosp-clang toolchain:

diff --git a/build.config b/build.config
index 471f62b386f4..2e9c813b1972 100644
--- a/build.config
+++ b/build.config
@@ -8,7 +8,7 @@ DEFCONFIG=wahoo_defconfig
 EXTRA_CMDS=''
 KERNEL_DIR=private/msm-google
 POST_DEFCONFIG_CMDS='check_defconfig'
-CLANG_PREBUILT_BIN=prebuilts-master/clang/host/linux-x86/clang-4053586/bin/
+CLANG_PREBUILT_BIN=aosp-clang/clang-r365631c/bin/
 LINUX_GCC_CROSS_COMPILE_PREBUILTS_BIN=prebuilts/gcc/linux-x86/aarch64/aarch64-linux-android-4.9/bin
 LINUX_GCC_CROSS_COMPILE_ARM32_PREBUILTS_BIN=prebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.9/bin
 LZ4_PREBUILTS_BIN=prebuilts-master/misc/linux-x86/lz4

4. I patched the kernel root makefile (in private/msm-google) to match CC containing 'clang' instead of exactly equal to 'clang':

diff --git a/Makefile b/Makefile
index 7d990fa2352a..241764ed7591 100644
--- a/Makefile
+++ b/Makefile
@@ -375,7 +375,7 @@ CFLAGS_GCOV = -fprofile-arcs -ftest-coverage -fno-tree-loop-im
 CFLAGS_KCOV    = -fsanitize-coverage=trace-pc


-ifeq ($(cc-name),clang)
+ifeq ($(findstring clang,$(cc-name)),clang)
 ifneq ($(CROSS_COMPILE),)
 CLANG_TRIPLE   ?= $(CROSS_COMPILE)
 CLANG_TARGET   := --target=$(notdir $(CLANG_TRIPLE:%-=%))

5. I added symlinks to 'clang' to the target-specific gcc toolchain bin directories, to stop clang from using /usr/bin/as. (This is according to https://blog.printk.io/2015/03/cross-compile-linux-for-arm-using-llvm-clang-on-arch-linux/):

pushd prebuilts/gcc/linux-x86/aarch64/aarch64-linux-android-4.9/bin
ln -s ../../../../../../aosp-clang/clang-r365631c/bin/clang.real clangp
popd

pushd prebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.9/bin
ln -s ../../../../../../aosp-clang/clang-r365631c/bin/clang.real clang
popd

6. Patched build/build.sh to wrap the kernel build 'make' with generate-csv:

diff --git a/build.sh b/build.sh
index 73eb3fc..f72cc57 100755
--- a/build.sh
+++ b/build.sh
@@ -268,7 +268,7 @@ echo "========================================================"
 echo " Building kernel"

 set -x
-(cd ${OUT_DIR} && make O=${OUT_DIR} "${TOOL_ARGS[@]}" ${MAKE_ARGS})
+(cd ${OUT_DIR} && env CLEAN=true ABSOUTDIR=/tmp/lsifout ABSROOTDIR=`pwd` ~/src/lsif-cpp/generate-csv make O=${OUT_DIR} ${MAKE_ARGS})
 set +x

 if [ -n "${POST_KERNEL_BUILD_CMDS}" ]; then

7. Patched generate-csv in lsif-cpp, in order to make it use the FULL command, instead of just the first word of the command, and to make it pass the CC, HOSTCC to the command:

╰─[✘]$ CXX=clang++ PATH=~/src/aosp_kernel/wahoo-pie/aosp-clang/clang-r365631c/bin:$PATH ./build
++ dirname ./build
+ root=.
+ cd .
+ yarn -s
+ yarn run tsc
yarn run v1.16.0

-cmd="$1"
+cmd="$@"

 FLAGS="-Xclang -load -Xclang $(realpath "$(dirname "${BASH_SOURCE[0]}")")/clang/libclang-index-plugin.so -Xclang -add-plugin -Xclang dxr-index -Xclang -plugin-arg-dxr-index -Xclang ."
 export CXX="clang++ $FLAGS"
@@ -37,7 +37,7 @@ echo "Running... $cmd"
         CXX="$CXX" \
         CC="$CC" \
         DXR_CXX_CLANG_TEMP_FOLDER="$ABSOUTDIR" \
-        bash -c "$cmd"
+        bash -c "$cmd CC=\"$CC\" HOSTCC=clang"
 )

 echo

8. Built lsif-cpp using the same clang as I'm using for the kernel, so that the clang plugin is compatible.

pushd ~/src/lsif-cpp
PATH=~/src/aosp_kernel/wahoo-pie/aosp-clang/clang-r365631c/bin:$PATH CXX=clang++ ./build 
popd

9. Ran the build:

./build/build.sh

I've attached the output dump.lsif.zip. Note that it doesn't include the words 'read' or 'vfork', which should definitely be parts of symbols in the kernel.

I'd appreciate any direction or help you could give me...

Thanks,
s1341

Update README to explain parameters

While going through setting up this LSIF indexer on a C project, there were a few things that weren't clear:

• What does ABSROOTDIR and ABSOUTDIR represent? What are they used for and where should they point to?
• When running lsif-cpp, it assumes that the C compiler used is linked to the LLVM. For this particular setup, they were not, so it would be nice to be able to pass this information
• Similarly for the conversion of the CSV files to LSIF, what are the parameters supposed to be pointing at?

We were able to work through and successfully generate an LSIF file, but when uploading it, we encountered two different errors on try 1 and try 2:

1. 404 Not Found Unknown commit.
2. stream error: stream ID 1; INTERNAL_ERROR

Not sure if this is an error with the uploader, or with the LSIF indexer.

Incompatible build with LLVM 10 (unreleased)

We should add some additional specifications to docs indicating which LLVM is supported (works with 9).

Supporting msbuild on Windows

Do you have any plan to support msbuild on Windows? Or are there altanative method to generate dump.lsif?

I know that this project support on Linux and macOS with Clang now.
I want to use this project with msbuild.

Thanks.