Debian packaging about imageio

[ghisvail]

Following the ANN for v1.1, I have refreshed the packaging repository on Alioth and attempted to rebuild the package. Few issues have come up but I'll see those I can fix downstream before bothering you here for upstream changes.

[almarklein]

Thanks @ghisvail. Let me know if changes are needed. We can aim for a 1.1.1 on short notice if necessary.

[ghisvail]

@almarklein, what is the purpose of installing all the stuff in data_files:

• in tests/ the full test suite,
• in docs/ the sphinx sources and config script,
• in make/, not sure what this is,
• in ./ the licensing, contributors and readme files,

?

EDIT: formatting

[almarklein]

Ah, I did that for you :) I understood that this would make running tests and deploying docs easier. The make/ is just a few dev tools. You can ignore it.

[ghisvail]

Perhaps I should have been more explicit about that.

First, I only needed the test suite and docs to be included in the source release (which is now the case), but not deployed as part of the installation. IMO, you should remove this data_files entry from your setup.py metadata as it will pollute other people's venv with superfluous files.

Then, I need to be able to generate the html documentation with sphinx and run the test suite without the requirement of an internet connection to fetch additional resources. Is that possible with v1.1 ?

Once again, I apologize for the confusion.

[ghisvail]

Ok so the core module package builds fine for both Python 2 and Python 3, without tests and docs. I have added a patch that comments out the data_files entry of setup.py for now. Best would be for you to use a MANIFEST file and drop data_files for the next release.

Now I installed the package and tested it. On first import, I noticed imageio automatically downloaded a binary of libfreeimage over the internet and installed it in $HOME/.imageio. Can imageio use a system-installed version of libfreeimage instead ?

[almarklein]

mmm... If I don't specify data_files, the docs and tests won't be included in the source release. Do you know a good way to only have them in the source release? Scanning sys.argv should work, but feels a bit hacky.

Imageio will try to load the imageio of the system, the version in imageio-dir/resources and ~/.imageio. If it does not find the lib in any of these, it attempts a download. Note that it requires the version of freeimage to be >= 3.15. You can also do python setup.py build_with_fi to download the fi lib at build time (and place in imageio-dir/resources/freeimage.

[ghisvail]

Do you know a good way to only have them in the source release?

A MANIFEST.in file ?

https://docs.python.org/2/distutils/sourcedist.html#the-manifest-in-template

[almarklein]

Ah, I always thought that was an auto-generated thing and never dared to touch it :) Ok, I'll apply the necessary changes soon.

What about the freeimage lib, did you find the problem? Is your system lib version >= 3.15?

[ghisvail]

I always thought that was an auto-generated thing

Yes it is. python setup.py sbuild is supposed to read the content of MANIFEST.in, generate the embedded MANIFEST file and include or prune the files of the source tree according to the rules written in MANIFEST.in.

What about the freeimage lib, did you find the problem?

I have not tried it yet, I'll have a look today. Stay tune :)

[ghisvail]

Ok, so Debian's libfreeimage is properly recognized and used by imageio instead of downloading it online. That allows the test suite to be run and the different binary packages for Python 2 and 3 to be built.

However, the avbin and ffmpeg plugins still download remote files despite the Debian libs / executables being installed. These would need to be addressed as well.

One option could be to simply use ctypes.utils.find_library, which should pick whatever installed library available system-wide. On my system:

In [1]: from ctypes.util import find_library
In [2]: find_library('freeimage')
Out[2]: 'libfreeimage.so.3'
In [3]: find_library('avbin')
Out[3]: 'libavbin.so.0'

What do you think ?

[almarklein]

Until now I've only focused on using system-level freeimage. Seems reasonable to do the same for avbin. For ffmpeg I am not sure. When I run ffmpeg on Ubuntu I get a deprecation warning. And IIRC there were some issues with that version (will need to look into that though).

[ghisvail]

Yes the situation with ffmpeg is not ideal since the forking to libav. I'll let you google the rationales of the fork or details regarding their core differences today if are interested. The consequence of the fork though was that ffmpeg got kicked out of the Debian tree (decision which has been reverted since) and Ubuntu naturally followed.

One solution would be to check whether your test suite works with Libav's "avconv" executable, provided by the libav-tools package. In that case, just make a test to check whether ffmpeg is available on the system and fallback to Libav's avconv in case it is not.

[almarklein]

Closing this. I made new issues for using system-level ffmpeg and avbin.

[ghisvail]

This is technically not resolved though. We should close it once a Debian package worthy of submission is made.

I'd probably leave it open as a meta-issue. Your call.

[almarklein]

Closing in favor of #151