jeremyd2019 / package-grokker Goto Github PK

a package could potentially be using delayimp

The default is 0x2000, but this is exceeded by openblas at least (discovered in msys2/MINGW-packages#16467). This limit is accepted as a parameter to the PE constructor, so should probably be set to 0x10000 (since there's a 16-bit limit inherent to the PE format)

In some cases packages build for example Python modules but don't define Python as a runtime dependency because it's only reachable when installing Python (some systems call this peer dependencies).

So, to catch all potential DLL users we should also look at the makedepends of the given package. And then follow normal transitive dependencies from there on as before.

the pefile module has a concept of 'warnings', which are not fatal and are not output by default. it provides get_warnings and show_warnings methods on the PE class.

One such warning is "Export directory contains more than {} symbol entries. Assuming corrupt.", which is triggered on msys2/MINGW-packages#16467

If used to check python packages, a good number of them would not have any EXE/DLL/PYD files, and would be a waste of time and bandwidth to download them to find that out. The script could load the .files db, and see if the files list contain any potentially matching files to only download those that might match.

Traceback (most recent call last):
  File "/home/runner/work/_actions/jeremyd2019/package-grokker/main/grokkermod.py", line 71, in __call__
    for entry in pe.DIRECTORY_ENTRY_IMPORT:
AttributeError: 'PE' object has no attribute 'DIRECTORY_ENTRY_IMPORT'

https://github.com/msys2/MINGW-packages/actions/runs/6988233339/job/19015883798?pr=19193

We currently get this a lot:

Warnings for ucrt64/lib/python2.7/site-packages/gtk-2.0/gtk/_gtk.pyd:
> Excessive number of imports 8193 (>8192)
Warnings for ucrt64/bin/libqgis_app.dll:
> Excessive number of imports 8193 (>8192)
> Excessive number of imports 8193 (>8192)
> Excessive number of imports 8193 (>8192)
> Excessive number of imports 8193 (>8192)
> Damaged Import Table information. ILT and/or IAT appear to be broken. OriginalFirstThunk: 0xbe86b8 FirstThunk: 0xbf80c0
> Excessive number of imports 8193 (>8192)
> Excessive number of imports 8193 (>8192)
> Damaged Import Table information. ILT and/or IAT appear to be broken. OriginalFirstThunk: 0xbebd88 FirstThunk: 0xbfb790
Warnings for ucrt64/bin/liboctinterp-11.dll:
> Excessive number of imports 8193 (>8192)
Warnings for ucrt64/bin/libqgis_gui.dll:
> Excessive number of imports 8193 (>8192)
> Excessive number of imports 8193 (>8192)
> Excessive number of imports 8193 (>8192)

I wonder if that " Damaged Import Table information" part is just there because it stops parsing when it reaches the limit.

And what is the real limit of import symbols, and can we set pefile.MAX_IMPORT_SYMBOLS to that, to avoid those warnings.

and then possibly plug those dlls and symbols into package-grokker to find references to them.

msys2/MINGW-packages#12987. Seems to be because ruby module has a .so extension, .so should be added to grokkermod.PE_FILE_EXTENSIONS

If more than one package is updated in a PR, the downloading and checking of packages should be combined somehow, to avoid downloading the same packages multiple times, once for each changed package it depends on.

Also, take the updated packages into account instead of downloading and checking the published versions of them, in case the PR already rebuilt dependencies.