Thanks Elf, i've reposted my Python topic from IN to here, but i needed to shorten it. My post seems to be too long for the forum system, so i've split the post in three parts.
This is a post about how i managed to compile python 2.7.16 with MIPSPro 7.4.4. on my IRIX 6.5.22m O2.
My objective was to minimize the amount of failed unittests, and for the ones that do fail, understand why they fail.
Since Python 2.7 has reached the end of life as of 01-01-2020 there will be a final 2.7 release before april 2020 with version number 2.7.18, so instead of going to 2.7.17 i release fixes to 2.7.16 for now and update to 2.7.18 later this year, so that we don't need to wait for further versions and have an 'ultimate' version ready, which saves me from doing a lot of work.
I started out with the Nekoware python patches and build instructions from Rainer Canavan. He creaed an excellent patch file as well as including IndigoMagic icons for the Python program and scripts.
I then included patches from Daniel Hams at
https://github.com/danielhams/didbs/blob/master/packages/python2/python2_7_16.sgifixes.patch and at
https://github.com/danielhams/didbs/blob/master/packages/python2/python2.sgifixes.patch
The patch that i have extended from Rainer's and Daniel's patch files is big and fixes, among other things, shared library creation, ABI settings, semaphore handling and a few bugs in separate modules. I've started running the test suite and noted down each failed test, find the module which causes it and try to fix it.
This fixes the building of a shared libpython.so library:
--- ./Makefile.pre.in Thu Sep 12 21:18:17 2019
+++ ./Makefile.pre.in Fri Sep 13 18:34:55 2019
@@ -541,8 +541,8 @@
libpython$(VERSION).so: $(LIBRARY_OBJS)
if test $(INSTSONAME) != $(LDLIBRARY); then \
- $(BLDSHARED) -Wl,-h$(INSTSONAME) -o $(INSTSONAME) $(LIBRARY_OBJS) $(MODLIBS) $(SHLIBS) $(LIBC) $(LIBM) $(LDLAST); \
- $(LN) -f $(INSTSONAME) $@; \
+ $(BLDSHARED) -soname $(INSTSONAME) -set_version sgi1.0 -o $(INSTSONAME) $(LIBRARY_OBJS) $(MODLIBS) $(SHLIBS) $(LIBC) $(LIBM) $(LDLAST); \
+ $(LN) -sf $(INSTSONAME) $@; \
else \
$(BLDSHARED) -o $@ $(LIBRARY_OBJS) $(MODLIBS) $(SHLIBS) $(LIBC) $(LIBM) $(LDLAST); \
fi
--- ./configure Thu Sep 12 13:08:47 2019
+++ ./configure Sat Sep 14 15:30:37 2019
@@ -5492,6 +5492,12 @@
esac
INSTSONAME="$LDLIBRARY".$SOVERSION
;;
+ IRIX*)
+ LDLIBRARY='libpython$(VERSION).so'
+ BLDLIBRARY='-Wl,-rpath -Wl,$(LIBDIR) -L. -lpython$(VERSION)'
+ RUNSHARED=LD_LIBRARYN32_PATH=`pwd`${LD_LIBRARYN32_PATH:+:${LD_LIBRARYN32_PATH}}
+ INSTSONAME="$LDLIBRARY".$SOVERSION
+ ;;
hp*|HP*)
case `uname -m` in
ia64)
@@ -8747,10 +8753,11 @@
then CCSHARED="-fPIC"
else CCSHARED="-Kpic -belf"
fi;;
- IRIX*/6*) case $CC in
- *gcc*) CCSHARED="-shared";;
- *) CCSHARED="";;
- esac;;
+ IRIX*/6*)
+ if test "$GCC" = "yes"
+ then CCSHARED="-fPIC"
+ else CCSHARED="-KPIC"
+ fi;;
atheos*) CCSHARED="-fPIC";;
esac
fi
This fixes the ABI to n32, enters some sensible commands for shared library creation and preprocessors and sets the target to IRIX regardless of IRIX or IRIX64 platforms.
--- ./configure Thu Sep 12 13:08:47 2019
+++ ./configure Sat Sep 14 15:30:37 2019
@@ -3313,7 +3313,7 @@
cygwin*) MACHDEP="cygwin";;
darwin*) MACHDEP="darwin";;
atheos*) MACHDEP="atheos";;
- irix646) MACHDEP="irix6";;
+ irix6*) MACHDEP="irix6";;
'') MACHDEP="unknown";;
esac
fi
@@ -3459,7 +3459,7 @@
if test ! -z "$SGI_ABI"
then
- CC="cc $SGI_ABI"
+ CC="$CC $SGI_ABI"
LDFLAGS="$SGI_ABI $LDFLAGS"
MACHDEP=`echo "${MACHDEP}${SGI_ABI}" | sed 's/ *//g'`
fi
--- ./Lib/distutils/sysconfig.py Tue Sep 3 15:44:04 2019
+++ ./Lib/distutils/sysconfig.py Fri Sep 13 01:13:00 2019
@@ -192,16 +192,14 @@
if 'CPP' in os.environ:
cpp = os.environ['CPP']
else:
- cpp = cc + " -E" # not always
+ cpp = cc + " -P" # not always
if 'LDFLAGS' in os.environ:
ldshared = ldshared + ' ' + os.environ['LDFLAGS']
if 'CFLAGS' in os.environ:
cflags = opt + ' ' + os.environ['CFLAGS']
- ldshared = ldshared + ' ' + os.environ['CFLAGS']
if 'CPPFLAGS' in os.environ:
cpp = cpp + ' ' + os.environ['CPPFLAGS']
cflags = cflags + ' ' + os.environ['CPPFLAGS']
- ldshared = ldshared + ' ' + os.environ['CPPFLAGS']
if 'AR' in os.environ:
ar = os.environ['AR']
if 'ARFLAGS' in os.environ:
--- ./Lib/distutils/unixccompiler.py Tue Sep 3 15:53:47 2019
+++ ./Lib/distutils/unixccompiler.py Sun Jan 5 16:56:44 2020
@@ -57,10 +57,10 @@
# Python extensions).
executables = {'preprocessor' : None,
'compiler' : ["cc"],
- 'compiler_so' : ["cc"],
- 'compiler_cxx' : ["cc"],
- 'linker_so' : ["cc", "-shared"],
- 'linker_exe' : ["cc"],
+ 'compiler_so' : ["cc -KPIC"],
+ 'compiler_cxx' : ["CC"],
+ 'linker_so' : ["ld", "-shared"],
+ 'linker_exe' : ["ld"],
'archiver' : ["ar", "-cr"],
'ranlib' : None,
}
Unittests which fail(ed) are:
test_bsddb185
This fails on the byte check for a particular test database. I've checked this with db_dump185 in the db4 package and it turns out that SGI's bsddb 1.8.5 version in /usr/lib32/libdb.a is not compatible with the test185.db database in the Python test directory, but it does work for db files like /etc/mail/aliases.db. Nevertheless, bsddb185 is buggy and deprecated, and can be replaced entirely by a db4 implementation.
test_codecs
see test_locale
test_ctypes
There are at least two bugs: one bug when returning a NULL Python object pointer to callproc.c and its NULL reference in GetResult:callproc.c triggering a segfault. The latter can be fixed with a patch by checking for the NULL pointer in advance before a call to Py_DECREF() is made. This will cause an exception if the first bug isn't fixed, but that is much better than a segfault in the python module.
A separate bug concerns the qsort test. This test in ctypes incorrectly sorts a string: It looks like it has problems swapping bytes around.
For this to have a minimal amount of failures, you need a recent libffi with fixes from didbs or dual_mips3_mips4 nekoware. See also
https://tudl1910.home.xs4all.nl/nekoware/dual-mips3-mips4/relnotes_patches/libffi-3.2.1_irix.patch
--- ./Modules/_ctypes/_ctypes.c Thu Aug 29 03:04:27 2019
+++ ./Modules/_ctypes/_ctypes.c Tue Sep 3 11:04:21 2019
@@ -4092,7 +4092,7 @@
}
}
- result = _ctypes_callproc(pProc,
+ result = _ctypes_callproc((PPROC)pProc,
callargs,
#ifdef MS_WIN32
piunk,
--- ./Modules/_ctypes/callproc.c Tue Sep 3 10:43:04 2019
+++ ./Modules/_ctypes/callproc.c Thu Sep 19 16:32:17 2019
@@ -834,7 +834,7 @@
#endif
delta =
#endif
- ffi_call(&cif, (void *)pProc, resmem, avalues);
+ ffi_call(&cif, FFI_FN(pProc), resmem, avalues);
#ifdef MS_WIN32
#ifndef DONT_USE_SEH
}
@@ -934,7 +934,7 @@
O_get), we have to call Py_DECREF because O_get has already
called Py_INCREF.
*/
- if (dict->getfunc == _ctypes_get_fielddesc("O")->getfunc) {
+ if (retval && dict->getfunc == _ctypes_get_fielddesc("O")->getfunc) {
Py_DECREF(retval);
}
} else
test_distutils
This tests the various compiler tools on the python binary but fails on test_config_cmd.py in line 50 where preprocessing is tested with search_cpp from Lib/distutils/command/config.py . What happens is that an 'empty' c file with body /* xxx */ is sent throught the preprocessor stage. Unlike the first test, which succeeds, the second test assumes that the preprocessing output can be redirected with -o _configtest.i option and checks for that filename, but MIPSPro can't do that with cc -P. This is the same situation on AIX, see
https://bugs.python.org/issue11191 : it is there that they recommended skipping the test on AIX and so do we.
--- ./Lib/distutils/tests/test_config_cmd.py Fri Jan 3 17:11:02 2020
+++ ./Lib/distutils/tests/test_config_cmd.py Sun Jan 5 16:49:35 2020
@@ -38,6 +38,7 @@
self.assertEqual(len(self._logs), numlines+1)
@unittest.skipIf(sys.platform == 'win32', "can't test on Windows")
+ @unittest.skipIf(sys.platform.startswith('irix6'), 'issue #11191: IRIX also affected')
def test_search_cpp(self):
pkg_dir, dist = self.create_dist()
cmd = config(dist)