Nighly feedstock build failed about conda-forge-nightly-controller

[KiterLuc]

Kicked off a rerun for tiledbfeedstock_CI.

[KiterLuc]

@jdblischak we have plenty of time before we ship so I’m ok waiting a day.

[KiterLuc]

@DimitrisStaratzis FYI.

[teo-tsirpanis]

This is the change in 3.29.1 that looks the most relevant. I will run a local build to see if there is anything suspicious with the WebP files.

[teo-tsirpanis]

OK, I will further investigate next week.

[teo-tsirpanis]

See TileDB-Inc/TileDB#4857 (comment)

[github-actions]

Nightly feedstock build failure for TileDB-Py%20Feedstock%20Testing at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=5&_a=summary

[jdblischak]

The second time the osx-64 build failed to download vcpkg-macos. Restarted again

[jdblischak]

Failed again at the exact same point. Restarted (this is Attempt 4)

[KiterLuc]

@teo-tsirpanis could this be caused by the vcpkg change?

[teo-tsirpanis]

Doubt, Conda has been using vcpkg for quite some time. This is the error:

CMake Error at build/vcpkg_installed/x64-osx/share/WebP/WebPConfig.cmake:15 (message):
  File or directory
  /Users/runner/miniforge3/conda-bld/tiledb_1712243674461/work/build/vcpkg_installed/include
  referenced by variable WebP_INCLUDE_DIR does not exist !

WebP has not recently changed.

[jdblischak]

This is the error:

Oh, right, I forgot about your function print_logs. That explains why I thought it was always failing to download vcpkg itself

WebP has not recently changed.

How should we proceed? It's already failed 4 times in a row today. I'm inclined to just let it be and see what happens tonight

[github-actions]

Nightly feedstock build failure for tiledbfeedstock_CI at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=4&_a=summary

[github-actions]

Nightly feedstock build failure for TileDB-Py%20Feedstock%20Testing at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=5&_a=summary

[jdblischak]

Well now it is no longer only the osx-64 build. Now all the builds are failing with the Webp configuration error:

-- Found Threads: TRUE
-- Found ZLIB: $PREFIX/lib/libz.so (found version "1.2.13")
CMake Error at build/vcpkg_installed/arm64-linux/share/WebP/WebPConfig.cmake:15 (message):
  File or directory
  /home/conda/feedstock_root/build_artifacts/tiledb_1712280430428/work/build/vcpkg_installed/include
  referenced by variable WebP_INCLUDE_DIR does not exist !
Call Stack (most recent call first):
  build/vcpkg_installed/arm64-linux/share/WebP/WebPConfig.cmake:38 (set_and_check)
  build/_deps/vcpkg-src/scripts/buildsystems/vcpkg.cmake:859 (_find_package)
  cmake/Modules/FindWebp_EP.cmake:31 (find_package)
  cmake/TileDB-Superbuild.cmake:108 (include)
  CMakeLists.txt:151 (include)


-- Configuring incomplete, errors occurred!

[ihnorton]

From discussion w/ @jdblischak this may be breakage from cmake version 3.29.0 -> 3.29.1

[jdblischak]

this may be breakage from cmake version 3.29.0 -> 3.29.1

Yes. @ihnorton suspected this could be due to an update in CMake. Sure enough, 3.29.1 was merged on April 4th at 4:27 AM ET.

The key evidence is the job from 2 nights ago (Wednesday night). At first the osx-64 build failed simply due to a spurious connection error when downloading the macOS SDK. However, when we restarted it later that day on April 4th, it installed CMake 3.29.1, and thus repeatedly failed with the Webp error we see now.

And then last night, all the builds installed CMake 3.29.1, and thus are all failing.

I'm going to push cmake < 3.29.1 to the nightly-build branch to confirm this fixes it.

[jdblischak]

Re-running the nightly builds with CMake 3.29.0 🤞

TileDB-Inc/tiledb-feedstock@26d2b39
https://dev.azure.com/TileDB-Inc/CI/_build/results?buildId=38844&view=results

[teo-tsirpanis]

CI is succeeding. Should we open an issue on CMake?

[jdblischak]

Should we open an issue on CMake?

Yes, it would be great if someone could open an Issue upstream. Would it be possible to put together a minimal, reproducible example to isolate this Webp error?

[teo-tsirpanis]

Cannot reproduce locally by reverting the pinning to <3.29.1 and running conda build ..

Never mind, I am getting the exact same error.

[teo-tsirpanis]

This is the difference between WebPConfig.cmake:

diff --git "a/C:\\Users\\teo\\code\\TileDB\\build\\Default\\vcpkg_installed\\x64-windows\\share\\WebP\\WebPConfig.cmake" "b/C:\\Users\\teo\\miniconda3\\conda-bld\\tiledb_1712362107853\\work\\build\\vcpkg_installed\\x64-windows\\share\\WebP\\WebPConfig.cmake"
index ecf40bf..f6c6c2d 100644
--- "a/C:\\Users\\teo\\code\\TileDB\\build\\Default\\vcpkg_installed\\x64-windows\\share\\WebP\\WebPConfig.cmake"
+++ "b/C:\\Users\\teo\\miniconda3\\conda-bld\\tiledb_1712362107853\\work\\build\\vcpkg_installed\\x64-windows\\share\\WebP\\WebPConfig.cmake"
@@ -7,7 +7,7 @@ set(WEBP_VERSION ${WebP_VERSION})
 ####### Any changes to this file will be overwritten by the next CMake run ####
 ####### The input file was WebPConfig.cmake.in                            ########

-get_filename_component(PACKAGE_PREFIX_DIR "${CMAKE_CURRENT_LIST_DIR}/../../" ABSOLUTE)
+get_filename_component(PACKAGE_${CMAKE_FIND_PACKAGE_NAME}_COUNTER_1 "${CMAKE_CURRENT_LIST_DIR}/../../../" ABSOLUTE)

 macro(set_and_check _var _file)
   set(${_var} "${_file}")
@@ -35,7 +35,7 @@ endif()

 include("${CMAKE_CURRENT_LIST_DIR}/WebPTargets.cmake")

-set_and_check(WebP_INCLUDE_DIR "${PACKAGE_PREFIX_DIR}/include")
+set_and_check(WebP_INCLUDE_DIR "${PACKAGE_${CMAKE_FIND_PACKAGE_NAME}_COUNTER_1}/include")
 set(WEBP_INCLUDE_DIRS ${WebP_INCLUDE_DIR})
 set(WebP_LIBRARIES "")
 include(SelectLibraryConfigurations)

The new config files have an extra ../, which causes the failures.

[teo-tsirpanis]

That CMake PR might be a red herring. The number of ../s depends on the CMAKE_INSTALL_PREFIX; I don't think it has changed. Or maybe something else in CMake 3.29.1 has changed. 🤔

Might have to do something with the superbuild which configures the actual project one directory inside build/ but again, why it started failing now?

[github-actions]

Nightly feedstock build failure for tiledbfeedstock_CI at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=4&_a=summary

[github-actions]

Nightly feedstock build failure for TileDB-Py%20Feedstock%20Testing at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=5&_a=summary

[KiterLuc]

Passing on rerun.

[jdblischak]

The nightlies were fixed by conda-forge/tiledb-feedstock@6f55d29 to install CMake 3.29.0.

We still need to investigate the issue with CMake 3.29.1. @teo-tsirpanis can we assign this to you?