build fails on Solaris 11 about python-subprocess32

[gpshead]

python 2 is EOL, no further updates to this project.

[jdemeyer]

More generally, the .c files in subprocess32 contains lots of checks of the form #ifdef HAVE_FOO. However, those cannot be expected to work properly because this backport is missing the configure script from Python 3.x which sets all those macros. Some of those macros also appear in Python 2.7, so those are safe. But others like HAVE_DIRFD do not exist in Python 2.7.

So I think that you should also use the configure script from Python 3.x in subprocess32.

[gpshead]

I'd rather not require autoconf for this project, especially given the small number of defines not available in 2.7. Could we put specific platform+version checks in setup.py that set the necessary extra CFLAGS for the extension?

[dimpase]

Well, gcc does not have anything useful for this purpose defined on Solaris 11. In Python you can do

In [6]: os.uname()
Out[6]: ('SunOS', 'xenos-sagetest', '5.11', '11.3', 'sun4v')

and so you'd need to check 1st entry for SunOS, and then check for 5.11 (3rd entry) to generate -DHAVE_DIRFD in your compiler flags.

[jdemeyer]

Could we put specific platform+version checks in setup.py that set the necessary extra CFLAGS for the extension?

The problem is that you will end up with a bunch a special cases for specific platforms but no general solution.

[jdemeyer]

For the record, the macros missing from Python 2.7.14 are

HAVE_DIRFD
HAVE_PIPE2
HAVE_SYS_SYSCALL_H

[gpshead]

Please give pull request #48 a try on your systems and report back.

[gpshead]

I just put up a 3.5.0rc3, please give it a try.

[gpshead]

3.5.0 should fix this but i have no ability to test. report and issue if not.

[dimpase]

well, no, there is a bug with recognising working C compiler:

[subprocess32-3.5.0] Installing collected packages: subprocess32
[subprocess32-3.5.0]   Running setup.py install for subprocess32: started
[subprocess32-3.5.0]     Running command /datapool/dima/Sage/sagetrac-mirror/local/bin/python2 -u -c "import setuptools, tokenize;__file__='/tmp/pip-1WyU_b-build/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" --no-user-cfg install --record /tmp/pip-vPZNo1-record/install-record.txt --single-version-externally-managed --compile
[subprocess32-3.5.0]     running install
[subprocess32-3.5.0]     running build
[subprocess32-3.5.0]     running build_py
[subprocess32-3.5.0]     creating build
[subprocess32-3.5.0]     creating build/lib.solaris-2.11-sun4v.64bit-2.7
[subprocess32-3.5.0]     copying subprocess32.py -> build/lib.solaris-2.11-sun4v.64bit-2.7
[subprocess32-3.5.0]     running build_ext
[subprocess32-3.5.0]     running build_configure
[subprocess32-3.5.0]     checking for gcc... gcc
[subprocess32-3.5.0]     checking whether the C compiler works... yes
[subprocess32-3.5.0]     checking for C compiler default output file name... a.out
[subprocess32-3.5.0]     checking for suffix of executables...
[subprocess32-3.5.0]     checking whether we are cross compiling... configure: error: in `/tmp/pip-1WyU_b-build':
[subprocess32-3.5.0]     configure: error: cannot run C compiled programs.
[subprocess32-3.5.0]     If you meant to cross compile, use `--host'.

[dimpase]

here is the relevant part of config.log

## Core tests. ##
## ----------- ##

configure:2341: checking for gcc
configure:2368: result: gcc
configure:2597: checking for C compiler version
configure:2606: gcc --version >&5
gcc (GCC) 7.2.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

configure:2617: $? = 0
configure:2606: gcc -v >&5
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/datapool/dima/libexec/gcc/sparcv9-sun-solaris2.11/7.2.0/lto-wrapper
Target: sparcv9-sun-solaris2.11
Configured with: ./configure --prefix=/datapool/dima --with-gmp-include=/usr/include/gmp --with-mpfr-include=/usr/include/mpfr --with-mpc=/usr --without-gnu-ld --with-ld=/usr/bin/ld --with-gnu-as --enable-languages=c,c++,fortr
an --enable-shared --enable-initfini-array --with-system-zlib --with-as=/usr/gnu/bin/as --build=sparcv9-sun-solaris2.11
Thread model: posix
gcc version 7.2.0 (GCC) 
configure:2617: $? = 0
configure:2606: gcc -V >&5
gcc: error: unrecognized command line option '-V'
gcc: fatal error: no input files
compilation terminated.
configure:2617: $? = 1
configure:2606: gcc -qversion >&5
gcc: error: unrecognized command line option '-qversion'; did you mean '--version'?
gcc: fatal error: no input files
compilation terminated.
configure:2617: $? = 1
configure:2637: checking whether the C compiler works
configure:2659: gcc   -L/datapool/dima/Sage/sagetrac-mirror/local/lib -Wl,-rpath,/datapool/dima/Sage/sagetrac-mirror/local/lib  conftest.c  >&5
configure:2663: $? = 0
configure:2711: result: yes
configure:2714: checking for C compiler default output file name
configure:2716: result: a.out
configure:2722: checking for suffix of executables
configure:2729: gcc -o conftest   -L/datapool/dima/Sage/sagetrac-mirror/local/lib -Wl,-rpath,/datapool/dima/Sage/sagetrac-mirror/local/lib  conftest.c  >&5
configure:2733: $? = 0
configure:2755: result: 
configure:2777: checking whether we are cross compiling
configure:2785: gcc -o conftest   -L/datapool/dima/Sage/sagetrac-mirror/local/lib -Wl,-rpath,/datapool/dima/Sage/sagetrac-mirror/local/lib  conftest.c  >&5
In file included from /datapool/dima/lib/gcc/sparcv9-sun-solaris2.11/7.2.0/include-fixed/stdio.h:24:0,
                 from conftest.c:17:
/datapool/dima/lib/gcc/sparcv9-sun-solaris2.11/7.2.0/include-fixed/sys/feature_tests.h:363:2: error: #error "Compiler or options invalid for pre-UNIX 03 X/Open applications    and pre-2001 POSIX applications"
 #error "Compiler or options invalid for pre-UNIX 03 X/Open applications \
  ^~~~~
configure:2789: $? = 1
configure:2796: ./conftest
./configure[2798]: eval[1]: ./conftest: not found [No such file or directory]
configure:2800: $? = 127
configure:2807: error: in `/datapool/dima/Sage/sagetrac-mirror/local/var/tmp/sage/build/subprocess32-3.5.0/src':
configure:2809: error: cannot run C compiled programs.
If you meant to cross compile, use `--host'.

[dimpase]

I suspect you're doing something outdated for Solaris in configure.ac, as we are perfectly able to use autoconf with the setup I'm testing this on.

[dimpase]

You probably need to call AC_PROG_CC, too. If I put it right after AC_INIT then something more meaningful happens, even though no headers are recognised:

checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking how to run the C preprocessor... gcc -E -traditional-cpp
checking for grep that handles long lines and -e... /opt/csw/gnu/grep
checking for egrep... /opt/csw/gnu/grep -E
checking for ANSI C header files... no
checking for sys/types.h... no
checking for sys/stat.h... no
checking for stdlib.h... no
checking for string.h... no
checking for memory.h... no
checking for strings.h... no
checking for inttypes.h... no
checking for stdint.h... no
checking for unistd.h... no
checking for unistd.h... (cached) no
checking fcntl.h usability... no
checking fcntl.h presence... no
checking for fcntl.h... no
checking sys/cdefs.h usability... no
checking sys/cdefs.h presence... no
checking for sys/cdefs.h... no
checking for sys/types.h... (cached) no
checking for sys/stat.h... (cached) no
checking sys/syscall.h usability... no
checking sys/syscall.h presence... yes
configure: WARNING: sys/syscall.h: present but cannot be compiled
configure: WARNING: sys/syscall.h:     check for missing prerequisite headers?
configure: WARNING: sys/syscall.h: see the Autoconf documentation
configure: WARNING: sys/syscall.h:     section "Present But Cannot Be Compiled"
configure: WARNING: sys/syscall.h: proceeding with the compiler's result
configure: WARNING:     ## ------------------------------------------------------------- ##
configure: WARNING:     ## Report this to https://github.com/google/python-subprocess32/ ##
configure: WARNING:     ## ------------------------------------------------------------- ##
checking for sys/syscall.h... no
checking for dirent.h that defines DIR... no
checking for sys/ndir.h that defines DIR... no
checking for sys/dir.h that defines DIR... no
checking for ndir.h that defines DIR... no
checking for library containing opendir... none required
checking for pipe2... no
checking for setsid... no
checking whether dirfd is declared... no
configure: creating ./config.status
config.status: creating _posixsubprocess_config.h

[gpshead]

odd environment. Python setuptools knows how to invoke the C compiler itself; I think figuring out how to get that from Python and plumb that through to the configure run (via CC= and potentially CFLAGS=) would make sense. even if configure finds a compiler toolchain on its own, there is no guarantee that it is the one used by Python.

[jdemeyer]

The compiler toolchain is not important. The checks you are doing involving OS features which shouldn't depend on the compiler. Of course, you do need a working compiler.