Friday, April 19, 2013

Compiling Matplotlib

Compile Qt4 and PyQt4.

We will only be using MinGW and Windows cmd.exe to compile Qt4 and PyQt4. No need to install Qt4.

qt-everywhere-opensource-src-4.8.4
http://download.qt-project.org/archive/qt/4.8/4.8.4/qt-everywhere-opensource-src-4.8.4.zip
patch: qt-4.8.4_mingw_static.diff

Update the environment;
set QTDIR=C:\Programs\Dev\qt-4.8.4
set QMAKESPEC=%QTDIR%\mkspecs\win32-g++
set PATH=%PATH%;C:\MinGW64\bin;%QTDIR%\bin
Patch the source;
cd %QTDIR%
patch -p1 <../qt-4.8.4_mingw_static.diff
Run configure;
configure.exe -confirm-license -release -static -opensource -qt-style-windowsxp -qt-style-windowsvista -opengl desktop -audio-backend -webkit -no-exceptions -no-s60 -no-phonon -no-qt3support -nomake tests -nomake examples -nomake demos -nomake translations -nomake docs
Compile;
mingw32-make

sip-4.15.2
http://sourceforge.net/projects/pyqt/files/sip/sip-4.15.2/sip-4.15.2.tar.gz/download
patch: sip-4.15.2_mingw-python.diff

Set the environment;
set QTDIR=C:\Programs\Dev\qt-4.8.4
set PATH=C:\MinGW64\bin;C:\Programs\Python27\bin;%QTDIR%\bin
Patch the source;
cd C:\Programs\Dev\sip-4.15.2
patch -p1 <../sip-4.15.2_mingw-python.diff
Run configure;
cd C:\Programs\Dev\sip-4.15.2
python configure.py -p win32-g++
Compile & install;
mingw32-make
mingw32-make install

PyQt-win-gpl-4.10.3
http://sourceforge.net/projects/pyqt/files/PyQt4/PyQt-4.10/PyQt-win-gpl-4.10.3.zip/download
patch: PyQt-win-gpl-4.10.3_mingw-python.diff

Set the environment;
set QTDIR=C:\Programs\Qt4
set PATH=%PATH%;C:\MinGW64\bin;C:\Programs\Python27\bin;%QTDIR%\bin
Patch the source;
cd C:\Programs\Dev\PyQt-win-gpl-4.10.3
patch -p1 <../PyQt-win-gpl-4.10.3_mingw-python.diff
 Run configure;
python configure.py --confirm-license -t qsvgicon -t qico -t qsvg -t qtga -g -w
Compile & install;
mingw32-make -j3
mingw32-make install
Test the compilation;
python
from PyQt4.Qt import *
from sip import *
print SIP_VERSION_STR, QT_VERSION_STR, PYQT_VERSION_STR

It should return something like;
4.15.2 4.8.4 4.10.3
Compile extra dependencies.

From this point forward, we will be using MSYS.

freetype-2.4.12
http://sourceforge.net/projects/freetype/files/freetype2/2.4.12/freetype-2.4.12.tar.bz2/download
cd freetype-2.4.12
./configure --prefix=c:/programs/dev/python64 --build=x86_64-w64-mingw32 --disable-shared
make -j2
make install

libpng-1.5.17
http://sourceforge.net/projects/libpng/files/libpng15/1.5.17/libpng-1.5.17.tar.bz2/download
cd libpng-1.5.17
./configure --prefix=c:/programs/dev/python64 --build=x86_64-w64-mingw32 --disable-shared
make -j2
make install

Compile python modules dependencies.

setuptools-1.3
https://pypi.python.org/packages/source/s/setuptools/setuptools-1.3.tar.gz
cd setuptools-1.3
python setup.py build install
pip-1.4.1
https://pypi.python.org/packages/source/p/pip/pip-1.4.1.tar.gz
cd pip-1.4.1
python setup.py build install
six-1.4.1
https://pypi.python.org/packages/source/s/six/six-1.4.1.tar.gz
cd six-1.4.1
python setup.py build install
python-dateutil-2.1
https://pypi.python.org/packages/source/p/python-dateutil/python-dateutil-2.1.tar.gz
cd python-dateutil-2.1
python setup.py build install
pyparsing-2.0.1
https://pypi.python.org/packages/source/p/pyparsing/pyparsing-2.0.1.tar.gz
cd pyparsing-2.0.1
python setup.py build install
pytz-2013.7
https://pypi.python.org/packages/source/p/pytz/pytz-2013.7.tar.gz
cd pytz-2013.7
python setup.py build install
unittest2-0.5.1
https://pypi.python.org/packages/source/u/unittest2/unittest2-0.5.1.tar.gz
cd unittest2-0.5.1
python setup.py build install
futures-2.1.4
https://pypi.python.org/packages/source/f/futures/futures-2.1.4.tar.gz
cd futures-2.1.4
python setup.py build install
pycares-0.5.0
https://pypi.python.org/packages/source/p/pycares/pycares-0.5.0.tar.gz
patch: add '#undef socklen_t' in 'src/pycares.h'
cd pycares-0.5.0
python setup.py build install
tornado-3.1.1
https://pypi.python.org/packages/source/t/tornado/tornado-3.1.1.tar.gz
cd tornado-3.1.1
python setup.py build install
numpy-1.8.0
scipy-0.13.0


** For NumPy and SciPy, refer to previous article. **


Compile Matplotlib.

matplotlib-1.3.1
http://sourceforge.net/projects/matplotlib/files/matplotlib/matplotlib-1.3.1/matplotlib-1.3.1.tar.gz/download
patch: matplotlib-1.3.1_mingw-python.diff
cd matplotlib-1.3.1
patch -p1 <../matplotlib-1.3.1_mingw-python.diff
cp setup.cfg.template setup.cfg
Edit setup.cfg;
change from '#basedirlist = /usr' to 'basedirlist = c:/programs/dev/python64'

Add extra CFLAGS to environment;
export CFLAGS="$CFLAGS -Ic:/programs/dev/python64/include/freetype2"
Compile & install;
python setup.py config build install
Test the compilation;
cd $HOME
python qt_mpl_bars.py


Tuesday, April 2, 2013

Compiling NumPy & SciPy

 ** Updated 11 August 2016 **

Continued from here.


* compiled using:
gcc-4.9.2 (dongsheng build) :
gcc-4.9-win32_4.9.2-20141029.7z
gcc-4.9-win64_4.9.2-20141029.7z

gcc-5.3.1 (dongsheng build) :
gcc-5.3-win32_5.3.1-20160601.7z
gcc-5.3-win64_5.3.1-20160601.7z

ATLAS-3.10.3
http://sourceforge.net/projects/math-atlas/files/Stable/3.10.3/atlas3.10.3.tar.bz2/download
http://www.netlib.org/lapack/lapack-3.6.1.tgz
patch: ATLAS-3.10.3_mingw.diff

** patch is based on vtjnash work.


Patch the source;
cd ATLAS
patch -p1 <../ATLAS-3.10.3_mingw.diff
Configure;

[ for i686]
cd ATLAS
mkdir -p _build32 && cd _build32
../configure --with-netlib-lapack-tarfile=../lapack-3.6.1.tgz -v 1 -A 30 -O 8 -V 960 -s 1 -b 32

make

[for x86_64]
cd ATLAS
mkdir -p _build64 && cd _build64
../configure --with-netlib-lapack-tarfile=../lapack-3.6.1.tgz -v 1 -A 30 -O 9 -V 960 -s 2 -b 64
make

Install;

[ for i686]
mkdir -p /dev/atlas-w32/{include/atlas,lib}
install -c -m 644 ../include/{cblas,clapack}.h /dev/atlas-w32/include
install -c -m 644 include/*.h /dev/atlas-w32/include/atlas
install -c -m 644 lib/lib*.a /dev/atlas-w32/lib

[for x86_64]
mkdir -p /dev/atlas-w64/{include/atlas,lib}
install -c -m 644 ../include/{cblas,clapack}.h /dev/atlas-w64/include
install -c -m 644 include/*.h /dev/atlas-w64/include/atlas
install -c -m 644 lib/lib*.a /dev/atlas-w64/lib


Run the testsuite;
make check
DONE BUILDING TESTERS, RUNNING:
SCOPING FOR FAILURES IN BIN TESTS:
fgrep -e fault -e FAULT -e error -e ERROR -e fail -e FAIL \
        bin/sanity.out
8 cases: 8 passed, 0 skipped, 0 failed
4 cases: 4 passed, 0 skipped, 0 failed
8 cases: 8 passed, 0 skipped, 0 failed
4 cases: 4 passed, 0 skipped, 0 failed
8 cases: 8 passed, 0 skipped, 0 failed
4 cases: 4 passed, 0 skipped, 0 failed
8 cases: 8 passed, 0 skipped, 0 failed
4 cases: 4 passed, 0 skipped, 0 failed
DONE
SCOPING FOR FAILURES IN CBLAS TESTS:
fgrep -e fault -e FAULT -e error -e ERROR -e fail -e FAIL \
        interfaces/blas/C/testing/sanity.out | \
                fgrep -v PASSED
make[1]: [sanity_test] Error 1 (ignored)
DONE
SCOPING FOR FAILURES IN F77BLAS TESTS:
fgrep -e fault -e FAULT -e error -e ERROR -e fail -e FAIL \
        interfaces/blas/F77/testing/sanity.out | \
                fgrep -v PASSED
make[1]: [sanity_test] Error 1 (ignored)
DONE

swig

swig : copy swig.exe from pywin32 to /dev/atlas-w{32,64}/bin

setuptools-5.8

cd setuptools-5.8
python setup.py build
python setup.py install

nose-1.3.7
cd nose-1.3.7
python setup.py config build
python setup.py install


* compiled using:
gcc-4.9.3 (mingw-builds) :
i686-4.9.3-release-win32-sjlj-rt_v4-rev1.7z
x86_64-4.9.3-release-win32-seh-rt_v4-rev1.7z
gcc-5.3.0 (mingw-builds) :
i686-5.3.0-release-win32-sjlj-rt_v4-rev0.7z
x86_64-5.3.0-release-win32-seh-rt_v4-rev0.7z

numpy-1.11.1
https://sourceforge.net/projects/numpy/files/NumPy/1.11.1/numpy-1.11.1.tar.gz/download
patch: numpy-1.11.1_mingw-python.diff

Patch the source;
cd numpy-1.11.1
patch -p1 <../numpy-1.11.1_mingw-python.diff
Add site.cfg;

cat > site.cfg << "EOF"
[DEFAULT]
library_dirs = c:\programs\dev\atlas-w64\lib
include_dirs = c:\programs\dev\atlas-w64\include
EOF
Build & install;
python setup.py config build --fcompiler=gnu95 install
Test the installed numpy;
cd $HOME
python -c "import numpy; numpy.test(verbose=3)"
======================================================================
FAIL: test_collections_hashable (test_multiarray.TestHashing)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\programs\python35\lib\python3.5\site-packages\numpy-1.11.1-py3.5-win-x86_64.egg\numpy\core\tests\test_multiarray.py",
line 6265, in test_collections_hashable
    self.assertFalse(isinstance(x, collections.Hashable))
AssertionError: True is not false

----------------------------------------------------------------------
Ran 6044 tests in 141.898s

FAILED (KNOWNFAIL=8, SKIP=10, failures=1)


scipy-0.18.0
http://sourceforge.net/projects/scipy/files/scipy/0.18.0/scipy-0.18.0.tar.gz/download
patch: scipy-0.18.0_mingw-python.diff

Patch the source;
cd scipy-0.18.0
patch -p1 <../scipy-0.18.0_mingw-python.diff
Add site.cfg;
cat > site.cfg << "EOF"
[DEFAULT]
library_dirs = c:\programs\dev\atlas-w64\lib
include_dirs = c:\programs\dev\atlas-w64\include
EOF
Build & install;
python setup.py config build --fcompiler=gnu95 install
Test the installed scipy;
cd $HOME
python -c "import scipy; scipy.test(verbose=3)"
======================================================================
ERROR: test_base.TestLIL.test_fancy_assignment_dtypes(dtype('float64'),)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\programs\python35\lib\python3.5\site-packages\nose-1.3.7-py3.5.egg\nose\case.py", line 198, in runTest
    self.test(*self.arg)
  File "c:\programs\python35\lib\python3.5\site-packages\scipy-0.18.0-py3.5-win-x86_64.egg\scipy\sparse\tests\test_base.py", line 2739, in
check
    A[[0,1],[0,1]] = dtype.type(1)
  File "c:\programs\python35\lib\python3.5\site-packages\scipy-0.18.0-py3.5-win-x86_64.egg\scipy\sparse\lil.py", line 377, in __setitem__
    i, j, x)
  File "scipy/sparse/_csparsetools.pyx", line 231, in scipy.sparse._csparsetools.lil_fancy_set (scipy\sparse\_csparsetools.c:4671)
  File "scipy/sparse/_csparsetools.pyx", line 368, in scipy.sparse._csparsetools._lil_fancy_set_int32_float64
(scipy\sparse\_csparsetools.c:6608)
ValueError: Buffer dtype mismatch, expected 'npy_float64' but got 'long double'

======================================================================
ERROR: test_base.TestLIL.test_fancy_assignment_dtypes(dtype('complex128'),)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\programs\python35\lib\python3.5\site-packages\nose-1.3.7-py3.5.egg\nose\case.py", line 198, in runTest
    self.test(*self.arg)
  File "c:\programs\python35\lib\python3.5\site-packages\scipy-0.18.0-py3.5-win-x86_64.egg\scipy\sparse\tests\test_base.py", line 2739, in
check
    A[[0,1],[0,1]] = dtype.type(1)
  File "c:\programs\python35\lib\python3.5\site-packages\scipy-0.18.0-py3.5-win-x86_64.egg\scipy\sparse\lil.py", line 377, in __setitem__
    i, j, x)
  File "scipy/sparse/_csparsetools.pyx", line 231, in scipy.sparse._csparsetools.lil_fancy_set (scipy\sparse\_csparsetools.c:4671)
  File "scipy/sparse/_csparsetools.pyx", line 432, in scipy.sparse._csparsetools._lil_fancy_set_int32_clongdouble
(scipy\sparse\_csparsetools.c:7510)
ValueError: Buffer dtype mismatch, expected 'long double complex' but got 'complex double'

======================================================================
ERROR: test_base.TestLIL.test_fancy_assignment_dtypes(dtype('complex128'),)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\programs\python35\lib\python3.5\site-packages\nose-1.3.7-py3.5.egg\nose\case.py", line 198, in runTest
    self.test(*self.arg)
  File "c:\programs\python35\lib\python3.5\site-packages\scipy-0.18.0-py3.5-win-x86_64.egg\scipy\sparse\tests\test_base.py", line 2739, in
check
    A[[0,1],[0,1]] = dtype.type(1)
  File "c:\programs\python35\lib\python3.5\site-packages\scipy-0.18.0-py3.5-win-x86_64.egg\scipy\sparse\lil.py", line 377, in __setitem__
    i, j, x)
  File "scipy/sparse/_csparsetools.pyx", line 231, in scipy.sparse._csparsetools.lil_fancy_set (scipy\sparse\_csparsetools.c:4671)
  File "scipy/sparse/_csparsetools.pyx", line 432, in scipy.sparse._csparsetools._lil_fancy_set_int32_clongdouble
(scipy\sparse\_csparsetools.c:7510)
ValueError: Item size of buffer (16 bytes) does not match size of 'long double complex' (32 bytes)

----------------------------------------------------------------------
Ran 23047 tests in 630.914s

FAILED (KNOWNFAIL=61, SKIP=1817, errors=3)
Running unit tests for scipy
NumPy version 1.11.1
NumPy relaxed strides checking option: False
NumPy is installed in c:\programs\python35\lib\python3.5\site-packages\numpy-1.11.1-py3.5-win-x86_64.egg\numpy
SciPy version 0.18.0
SciPy is installed in c:\programs\python35\lib\python3.5\site-packages\scipy-0.18.0-py3.5-win-x86_64.egg\scipy
Python version 3.5.2 (default, Jul 28 2016, 21:19:51) [GCC 5.3.0 (64-bit)]
nose version 1.3.7
0-th dimension must be fixed to 3 but got 15
0-th dimension must be fixed to 3 but got 5
0-th dimension must be fixed to 4 but got 2
0-th dimension must be fixed to 4 but got 2
0-th dimension must be fixed to 4 but got 2
0-th dimension must be fixed to 4 but got 2
0-th dimension must be fixed to 4 but got 2
0-th dimension must be fixed to 4 but got 2
0-th dimension must be fixed to 4 but got 2
0-th dimension must be fixed to 4 but got 2
0-th dimension must be fixed to 4 but got 2
0-th dimension must be fixed to 4 but got 2


Downloads:

i686 packages

x86_64 packages