Configure Theano to use Intel MKL on Windows

In order to use convolution layers and FFT in Theano, Theano requires either BLAS or CUDA to be present. On computers without an Nvidia graphics card, BLAS is the only choice.

There are 3 popular BLAS implementations, namely Intel MKL, OpenBLAS and ATLAS. A benchmark shows that ATLAS is the slowest, while MKL and OpenBLAS are on par. I could not get OpenBLAS to work, as I keep getting the error "DLL load failed: The specified module could not be found" despite ensuring that all the dependent DLLs are on my path.

Getting the MKL library

1. Before Intel would let you download the MKL library, you have to register at https://software.intel.com/en-us/intel-mkl
2. You will receive an email containing the link to the download page of various Intel libraries. Download the Math Kernel Library (MKL) installer
3. Install

 Configuration

1. By default, MKL gets installed into C:\Program Files (x86)\IntelSWTools. I could not figure out what escape sequence I need to enter to get spaces to work in gcc. Thus use Link Shell Extension to create a symbolic link from C:\tools\IntelSWTools to C:\Program Files (x86)\IntelSWTools.
2. Edit C:\Users\.theanorc and add the following
   [blas]
   ldflags = -LC:/tools/IntelSWTools/compilers_and_libraries/windows/mkl/lib/intel64_win -LC:/tools/IntelSWTools/compilers_and_libraries/windows/mkl/lib/intel64_win_mic  -lmkl_core -lmkl_intel_thread -lmkl_lapack95_lp64 -lmkl_blas95_lp64 -lmkl_rt
3. Add C:\tools\IntelSWTools\compilers_and_libraries\windows\redist\intel64_win\mkl to your system environmental variable PATH

Checking that it works

1. Open command line and cd to \misc\
2. python.exe check_blas.py
3. The output should have something like the following
   We executed 10 calls to gemm with a and b matrices of shapes (5000, 5000) and (5000, 5000).
   
   Total execution time: 18.00s on CPU (with direct Theano binding to blas).
   
   Try to run this script a few times. Experience shows that the first time is not as fast as followings calls. The difference is not big, but consistent.

Compiling aspell-python

While the Unofficial Windows Binaries for Python has aspell-python for Python 3.4, it does not have one for Python 3.5. Thus, we've got to compile it ourselves.

Since Python 3.5 requires Visual Studio 2015, we first have to modify C:\tools\python\tools\Lib\distutils\cygwinccompiler.py to recognize Visual Studio 2015
--- cygwinccompiler.py
+++ cygwinccompiler.py
@@ -82,7 +82,18 @@ def get_msvcr():
         elif msc_ver == '1600':
             # VS2010 / MSVC 10.0
             return ['msvcr100']
+        elif msc_ver == '1700':
+            # Visual Studio 2012 / Visual C++ 11.0
+            return ['msvcr110']
+        elif msc_ver == '1800':
+            # Visual Studio 2013 / Visual C++ 12.0
+            return ['msvcr120']
+        elif msc_ver == '1900':
+            # Visual Studio 2015 / Visual C++ 14.0
+            # "msvcr140.dll no longer exists" http://blogs.msdn.com/b/vcblog/archive/2014/06/03/visual-studio-14-ctp.aspx
+            return ['vcruntime140']
         else:
+            # to do: can we make this futureproof?
             raise ValueError("Unknown MS Compiler version %s " % msc_ver)
 


Next, download and unzip aspell-python from https://github.com/WojciechMula/aspell-python

 In setup.3.py, modify get_include_dirs() from
        return [] 
to

        return ['c:/tools/Aspell/include'] 

this is for aspell-python build scripts to find the aspell.h

In  aspell.c, modify static PyTypeObject aspell_AspellType from

    PyVarObject_HEAD_INIT(&PyType_Type, 0)

to

    PyVarObject_HEAD_INIT(NULL, 0)

and insert after
     speller_as_sequence.sq_contains = m_contains;

the following

    Py_TYPE(&aspell_AspellType) = &PyType_Type;

as per instructions on https://docs.python.org/3.5/c-api/typeobj.html#c.PyObject.ob_type

Copy C:\tools\Aspell\lib\libaspell.dll.a to the directory with aspell-python source code, and rename it to aspell.lib.

Then build it normally using

python setup.3.py build
python setup.3.py install

Compiling Aspell on Windows

Download Aspell

Download and extract GNU Aspell 0.60.6.1 from http://aspell.net/
Download and extract the English dictionary (aspell6-en-2016.01.19-0.tar.bz2)

Install Chocolatey machine package manager

Start PowerShell as Administrator
Enter the following commands:
Set-ExecutionPolicy ByPass
iex ((new-object net.webclient).DownloadString('https://chocolatey.org/install.ps1'))

Install the build tools

Start cmd as Administrator
Enter the following commands:
choco install msys2
choco install mingw
choco install activeperl
choco install linkshellextension

Start msys2_shell.bat in C:\tools\msys2
Enter the following commands:

pacman -S make
pacman -S diffutils

Update build script

Overwrite config.guess with file from http://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.guess;hb=HEAD
Overwrite config.sub with file from http://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.sub;hb=HEAD

Build

In msys2 shell, navigate to downloaded Aspell
Enter:
./configure CXXFLAGS='-O2' --enable-win32-relocatable --disable-rpath --prefix="C:/tools/Aspell" --enable-data-dir="C:/tools/Aspell/data" --enable-dict-dir="C:/tools/Aspell/data"

In the downloaded Aspell, add the file gen/dirs.h with the following contents
#define PREFIX "C:/tools/Aspell"
#define DATA_DIR "C:/tools/Aspell/data"
#define DICT_DIR DATA_DIR
#define CONF_DIR PREFIX

Add a line to common/file_util.cpp under the “#ifdef WIN32” section
#include "asc_ctype.hpp"

In msys2 shell, enter
make
make install
Add C:\tools\Aspell\bin to PATH system environment variable

Create hardlink from C:\tools\Aspell\data (drop as hardlink) to C:\tools\Aspell\lib\aspell-0.60 (pick link source)

Restart msys2 shell and navigate to downloaded Aspell dictionary and enter
./configure
make
make install

Final steps

Open cmd, navigate to c:\tools\Aspell\bin and enter

aspell dump config > ../aspell.conf

Open c:\tools\Aspell\aspell.conf in Notepad and add
encoding utf-8
below # default: !encoding = none

Add
sug-mode bad-spellers
below
# sug-mode (string)
#   suggestion mode
# default: normal

To use in Aspell in Python

Download aspell-python from http://www.lfd.uci.edu/~gohlke/pythonlibs/
In cmd, navigate to c:\tools\python\Scripts and enter
pip install [path to the file aspell-python.whl]

Create symbolic link from C:\ProgramData\Aspell to  C:\tools\Aspell
Create symbolic link from C:\tools\Aspell\Dictionaries\en_US to C:\tools\Aspell\data

Start Python
import aspell
spell = aspell.Speller()
enc = spell.ConfigKeys()['encoding'][1]
print(spell.suggest('mispelling'.encode(enc)))


Credits:
http://nosq.com/2005/07/aspell-0603-for-win32/
https://github.com/WojciechMula/aspell-python/blob/master/windows.rst