Configure Theano to use Intel MKL on Windows

In order to use convolution layers and FFT in Theano, Theano requires either BLAS or CUDA to be present. On computers without an Nvidia graphics card, BLAS is the only choice.

There are 3 popular BLAS implementations, namely Intel MKL, OpenBLAS and ATLAS. A benchmark shows that ATLAS is the slowest, while MKL and OpenBLAS are on par. I could not get OpenBLAS to work, as I keep getting the error "DLL load failed: The specified module could not be found" despite ensuring that all the dependent DLLs are on my path.

Getting the MKL library

1. Before Intel would let you download the MKL library, you have to register at https://software.intel.com/en-us/intel-mkl
2. You will receive an email containing the link to the download page of various Intel libraries. Download the Math Kernel Library (MKL) installer
3. Install

 Configuration

1. By default, MKL gets installed into C:\Program Files (x86)\IntelSWTools. I could not figure out what escape sequence I need to enter to get spaces to work in gcc. Thus use Link Shell Extension to create a symbolic link from C:\tools\IntelSWTools to C:\Program Files (x86)\IntelSWTools.
2. Edit C:\Users\.theanorc and add the following
   [blas]
   ldflags = -LC:/tools/IntelSWTools/compilers_and_libraries/windows/mkl/lib/intel64_win -LC:/tools/IntelSWTools/compilers_and_libraries/windows/mkl/lib/intel64_win_mic  -lmkl_core -lmkl_intel_thread -lmkl_lapack95_lp64 -lmkl_blas95_lp64 -lmkl_rt
3. Add C:\tools\IntelSWTools\compilers_and_libraries\windows\redist\intel64_win\mkl to your system environmental variable PATH

Checking that it works

1. Open command line and cd to \misc\
2. python.exe check_blas.py
3. The output should have something like the following
   We executed 10 calls to gemm with a and b matrices of shapes (5000, 5000) and (5000, 5000).
   
   Total execution time: 18.00s on CPU (with direct Theano binding to blas).
   
   Try to run this script a few times. Experience shows that the first time is not as fast as followings calls. The difference is not big, but consistent.

Compiling aspell-python

While the Unofficial Windows Binaries for Python has aspell-python for Python 3.4, it does not have one for Python 3.5. Thus, we've got to compile it ourselves.

Since Python 3.5 requires Visual Studio 2015, we first have to modify C:\tools\python\tools\Lib\distutils\cygwinccompiler.py to recognize Visual Studio 2015
--- cygwinccompiler.py
+++ cygwinccompiler.py
@@ -82,7 +82,18 @@ def get_msvcr():
         elif msc_ver == '1600':
             # VS2010 / MSVC 10.0
             return ['msvcr100']
+        elif msc_ver == '1700':
+            # Visual Studio 2012 / Visual C++ 11.0
+            return ['msvcr110']
+        elif msc_ver == '1800':
+            # Visual Studio 2013 / Visual C++ 12.0
+            return ['msvcr120']
+        elif msc_ver == '1900':
+            # Visual Studio 2015 / Visual C++ 14.0
+            # "msvcr140.dll no longer exists" http://blogs.msdn.com/b/vcblog/archive/2014/06/03/visual-studio-14-ctp.aspx
+            return ['vcruntime140']
         else:
+            # to do: can we make this futureproof?
             raise ValueError("Unknown MS Compiler version %s " % msc_ver)
 


Next, download and unzip aspell-python from https://github.com/WojciechMula/aspell-python

 In setup.3.py, modify get_include_dirs() from
        return [] 
to

        return ['c:/tools/Aspell/include'] 

this is for aspell-python build scripts to find the aspell.h

In  aspell.c, modify static PyTypeObject aspell_AspellType from

    PyVarObject_HEAD_INIT(&PyType_Type, 0)

to

    PyVarObject_HEAD_INIT(NULL, 0)

and insert after
     speller_as_sequence.sq_contains = m_contains;

the following

    Py_TYPE(&aspell_AspellType) = &PyType_Type;

as per instructions on https://docs.python.org/3.5/c-api/typeobj.html#c.PyObject.ob_type

Copy C:\tools\Aspell\lib\libaspell.dll.a to the directory with aspell-python source code, and rename it to aspell.lib.

Then build it normally using

python setup.3.py build
python setup.3.py install