Configure Theano to use Intel MKL on Windows

In order to use convolution layers and FFT in Theano, Theano requires either BLAS or CUDA to be present. On computers without an Nvidia graphics card, BLAS is the only choice.

There are 3 popular BLAS implementations, namely Intel MKL, OpenBLAS and ATLAS. A benchmark shows that ATLAS is the slowest, while MKL and OpenBLAS are on par. I could not get OpenBLAS to work, as I keep getting the error "DLL load failed: The specified module could not be found" despite ensuring that all the dependent DLLs are on my path.

Getting the MKL library

1. Before Intel would let you download the MKL library, you have to register at https://software.intel.com/en-us/intel-mkl
2. You will receive an email containing the link to the download page of various Intel libraries. Download the Math Kernel Library (MKL) installer
3. Install

 Configuration

1. By default, MKL gets installed into C:\Program Files (x86)\IntelSWTools. I could not figure out what escape sequence I need to enter to get spaces to work in gcc. Thus use Link Shell Extension to create a symbolic link from C:\tools\IntelSWTools to C:\Program Files (x86)\IntelSWTools.
2. Edit C:\Users\.theanorc and add the following
   [blas]
   ldflags = -LC:/tools/IntelSWTools/compilers_and_libraries/windows/mkl/lib/intel64_win -LC:/tools/IntelSWTools/compilers_and_libraries/windows/mkl/lib/intel64_win_mic  -lmkl_core -lmkl_intel_thread -lmkl_lapack95_lp64 -lmkl_blas95_lp64 -lmkl_rt
3. Add C:\tools\IntelSWTools\compilers_and_libraries\windows\redist\intel64_win\mkl to your system environmental variable PATH

Checking that it works

1. Open command line and cd to \misc\
2. python.exe check_blas.py
3. The output should have something like the following
   We executed 10 calls to gemm with a and b matrices of shapes (5000, 5000) and (5000, 5000).
   
   Total execution time: 18.00s on CPU (with direct Theano binding to blas).
   
   Try to run this script a few times. Experience shows that the first time is not as fast as followings calls. The difference is not big, but consistent.

Compiling aspell-python

While the Unofficial Windows Binaries for Python has aspell-python for Python 3.4, it does not have one for Python 3.5. Thus, we've got to compile it ourselves.

Since Python 3.5 requires Visual Studio 2015, we first have to modify C:\tools\python\tools\Lib\distutils\cygwinccompiler.py to recognize Visual Studio 2015
--- cygwinccompiler.py
+++ cygwinccompiler.py
@@ -82,7 +82,18 @@ def get_msvcr():
         elif msc_ver == '1600':
             # VS2010 / MSVC 10.0
             return ['msvcr100']
+        elif msc_ver == '1700':
+            # Visual Studio 2012 / Visual C++ 11.0
+            return ['msvcr110']
+        elif msc_ver == '1800':
+            # Visual Studio 2013 / Visual C++ 12.0
+            return ['msvcr120']
+        elif msc_ver == '1900':
+            # Visual Studio 2015 / Visual C++ 14.0
+            # "msvcr140.dll no longer exists" http://blogs.msdn.com/b/vcblog/archive/2014/06/03/visual-studio-14-ctp.aspx
+            return ['vcruntime140']
         else:
+            # to do: can we make this futureproof?
             raise ValueError("Unknown MS Compiler version %s " % msc_ver)
 


Next, download and unzip aspell-python from https://github.com/WojciechMula/aspell-python

 In setup.3.py, modify get_include_dirs() from
        return [] 
to

        return ['c:/tools/Aspell/include'] 

this is for aspell-python build scripts to find the aspell.h

In  aspell.c, modify static PyTypeObject aspell_AspellType from

    PyVarObject_HEAD_INIT(&PyType_Type, 0)

to

    PyVarObject_HEAD_INIT(NULL, 0)

and insert after
     speller_as_sequence.sq_contains = m_contains;

the following

    Py_TYPE(&aspell_AspellType) = &PyType_Type;

as per instructions on https://docs.python.org/3.5/c-api/typeobj.html#c.PyObject.ob_type

Copy C:\tools\Aspell\lib\libaspell.dll.a to the directory with aspell-python source code, and rename it to aspell.lib.

Then build it normally using

python setup.3.py build
python setup.3.py install

Compiling Aspell on Windows

Download Aspell

Download and extract GNU Aspell 0.60.6.1 from http://aspell.net/
Download and extract the English dictionary (aspell6-en-2016.01.19-0.tar.bz2)

Install Chocolatey machine package manager

Start PowerShell as Administrator
Enter the following commands:
Set-ExecutionPolicy ByPass
iex ((new-object net.webclient).DownloadString('https://chocolatey.org/install.ps1'))

Install the build tools

Start cmd as Administrator
Enter the following commands:
choco install msys2
choco install mingw
choco install activeperl
choco install linkshellextension

Start msys2_shell.bat in C:\tools\msys2
Enter the following commands:

pacman -S make
pacman -S diffutils

Update build script

Overwrite config.guess with file from http://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.guess;hb=HEAD
Overwrite config.sub with file from http://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.sub;hb=HEAD

Build

In msys2 shell, navigate to downloaded Aspell
Enter:
./configure CXXFLAGS='-O2' --enable-win32-relocatable --disable-rpath --prefix="C:/tools/Aspell" --enable-data-dir="C:/tools/Aspell/data" --enable-dict-dir="C:/tools/Aspell/data"

In the downloaded Aspell, add the file gen/dirs.h with the following contents
#define PREFIX "C:/tools/Aspell"
#define DATA_DIR "C:/tools/Aspell/data"
#define DICT_DIR DATA_DIR
#define CONF_DIR PREFIX

Add a line to common/file_util.cpp under the “#ifdef WIN32” section
#include "asc_ctype.hpp"

In msys2 shell, enter
make
make install
Add C:\tools\Aspell\bin to PATH system environment variable

Create hardlink from C:\tools\Aspell\data (drop as hardlink) to C:\tools\Aspell\lib\aspell-0.60 (pick link source)

Restart msys2 shell and navigate to downloaded Aspell dictionary and enter
./configure
make
make install

Final steps

Open cmd, navigate to c:\tools\Aspell\bin and enter

aspell dump config > ../aspell.conf

Open c:\tools\Aspell\aspell.conf in Notepad and add
encoding utf-8
below # default: !encoding = none

Add
sug-mode bad-spellers
below
# sug-mode (string)
#   suggestion mode
# default: normal

To use in Aspell in Python

Download aspell-python from http://www.lfd.uci.edu/~gohlke/pythonlibs/
In cmd, navigate to c:\tools\python\Scripts and enter
pip install [path to the file aspell-python.whl]

Create symbolic link from C:\ProgramData\Aspell to  C:\tools\Aspell
Create symbolic link from C:\tools\Aspell\Dictionaries\en_US to C:\tools\Aspell\data

Start Python
import aspell
spell = aspell.Speller()
enc = spell.ConfigKeys()['encoding'][1]
print(spell.suggest('mispelling'.encode(enc)))


Credits:
http://nosq.com/2005/07/aspell-0603-for-win32/
https://github.com/WojciechMula/aspell-python/blob/master/windows.rst

Creating a constraint to ensure intersection table only joins certain records

I have a database with the following tables

I need to ensure that users only becomes friends with other users of the same department.

Approach 1 – DepartmentId on Friend

On http://stackoverflow.com/a/20443626/263003, diarmuid suggested to add the DepartmentId on the Friend table and use the foreign key constraints of (MyselfId, AppId) and (FriendId, AppId). However, I would have to set the primary key of the User table as (Id, DepartmentId) – this creates a super key and I want to avoid that.

Approach 2 – CHECK CONSTRAINT on VIEW

The simplest approach I could think of is to create a VIEW joining User to Friend to User, and add a CHECK CONSTRAINT on the view. However, CHECK CONSTRAINT only applies on TABLE and not VIEW.

Approach 3 – CROSS JOIN and UNIQUE INDEX on VIEW

On http://stackoverflow.com/a/9751172/263003, Martin Smith uses the approach of selecting the records we do not want, creating duplicate records of those using CROSS JOIN, and ensuring those records cannot exists by using a UNIQUE INDEX. Let’s start by creating the INDEXED VIEW

CREATE VIEW [dbo].[CheckFriendView]
WITH SCHEMABINDING
AS
SELECT myself.Id AS MyselfId, myFriend.Id AS MyFriendId, myself.DepartmentId AS MyDepartmentId, myFriend.DepartmentId AS FriendDepartmentId
FROM  dbo.[User] AS myself
INNER JOIN  dbo.Friend
ON dbo.Friend.MyselfId = myself.Id
INNER JOIN dbo.[User] AS myFriend
ON dbo.Friend.FriendId = myFriend.Id
WHERE myself.DepartmentId <> myFriend.DepartmentId

GO

CREATE UNIQUE CLUSTERED INDEX IDX_CheckFriendView
   ON [dbo].[CheckFriendView] (MyselfId, MyFriendId)

GO

However, I got the error

Cannot create index on view "Demo.dbo.CheckFriendView". The view contains a self join on "Demo.dbo.User".

Let’s avoid self join by moving the join to a WHERE clause

CREATE VIEW [dbo].[CheckFriendView]
WITH SCHEMABINDING
AS
SELECT MyselfId, FriendId
FROM dbo.Friend AS f
WHERE EXISTS (
    SELECT 1
    FROM  dbo.[User] AS myself
    INNER JOIN dbo.Friend
    ON dbo.Friend.MyselfId = myself.Id
    INNER JOIN dbo.[User] AS myFriend
    ON dbo.Friend.FriendId = myFriend.Id
    WHERE myself.Id = f.MyselfId AND
          myFriend.Id = f.FriendId AND
          myself.DepartmentId <> myFriend.DepartmentId
)

GO

CREATE UNIQUE CLUSTERED INDEX IDX_CheckFriendView
   ON [dbo].[CheckFriendView] (MyselfId, FriendId)

GO

Another error comes up

Cannot create index on view "Demo.dbo.CheckFriendView" because it contains one or more subqueries. Consider changing the view to use only joins instead of subqueries. Alternatively, consider not indexing this view.

Looks like SQL Server will never let us join.

Approach 4 – Turning MyselfId and FriendId into a single column

Jonathan avoided the self join by combining the MyselfId and FriendId columns with the help of a numbers table containing 2 entries, then joining into the User table – http://jmkehayias.blogspot.sg/2008/12/creating-indexed-view-with-self-join.html

Let’s start by combining MyselfId and FriendId

CREATE VIEW [dbo].[CheckFriendOneColumnView]
WITH SCHEMABINDING
AS
SELECT MyselfId, CASE WHEN n.Number = 1 THEN MyselfId ELSE FriendId END AS MyselfOrFriendId
FROM dbo.Friend AS f
CROSS JOIN dbo.[Number] AS n

And then add the joining of User

INNER JOIN dbo.[User] AS u
ON f.UserId = u.Id OR f.FriendId = u.Id

Oh wait a minute, this is getting unnecessarily complicated. Could we just use

INNER JOIN dbo.[User] AS u
ON f.UserId = u.Id OR f.FriendId = u.Id

without the cross join?

Finally

We do an INNER JOIN with OR condition. Each row in Friend will become 2 rows, one with the DepartmentId for myself, the other with the DepartmentId for my friend.

Then aggregate the (MyselfId, DepartmentId). Since all the friends should belong to the same department, this aggregation should leave MyselfId as unique.

Lastly add the UNIQUE INDEX on MyselfId.

CREATE VIEW [dbo].[CheckFriendView]
WITH SCHEMABINDING
AS
SELECT f.MyselfId, u.DepartmentId, COUNT_BIG(*) AS BigCount
FROM dbo.Friend AS f
INNER JOIN dbo.[User] AS u
ON f.MyselfId = u.Id OR f.FriendId = u.Id
GROUP BY f.MyselfId, u.DepartmentId

GO

CREATE UNIQUE CLUSTERED INDEX IDX_CheckFriendView
    ON [dbo].[CheckFriendView] (MyselfId)

GO

Note: I had to add COUNT_BIG(*) AS BigCount due to another error

Cannot create index on view 'CheckFriendView' because its select list does not include  a proper use of COUNT_BIG. Consider adding COUNT_BIG(*) to select list.

which basically says I must add it!

Update:

Warning: while the INNER JOIN ON a OR b works, it results in very inefficient query (no JOIN predicate) and slows down every changes in the database!

React JS in ASP.NET MVC part 2—setting up React

Set up the project and libraries for React

Let’s start by creating a new ASP.NET MVC project

Next, we will use NuGet to download the React libraries that we need

Right-click on References and click Manage NuGet Packages

Search for react.js

Install these libraries:

• react.js
  This contains the JavaScript for the browser
• ReactJS.NET (MVC 4 and 5)
  This contains the libraries necessary for server-side rendering
• ReactJS.NET – JSX for ASP.NET Web Optimization Framework
  This contains the library for transforming JSX into plain JavaScript, and then minified and bundled to speed up download
• ReactJS.NET – ClearScript V8
  V8 is a faster JavaScript engine which we will use to run our React scripts server-side. This is much better than the Jint engine bundled with ReactJS.NET (MVC 4 and 5)

Next, we prepare the minification and bundling for the React JavaScript library.

Include the React JavaScript in our App_Start\BundleConfig.cs

// React JS
// Should not use the official React CDN at //fb.me/, as it redirects to HTTP and breaks HTTPS sites.
bundles.Add(new ScriptBundle("~/js/reactjs", "//cdnjs.cloudflare.com/ajax/libs/react/0.12.2/react-with-addons.min.js")
{
     CdnFallbackExpression = "window.React"
}.Include(
     "~/Scripts/react/react-with-addons-{version}.min.js"
));

Include the ~/js/reactjs script bundle in Views\Shared\_Layout.cshtml

<head>
     <meta charset="utf-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0">
     <title>@ViewBag.Title - My ASP.NET Application</title>
     @Styles.Render("~/Content/css")
     @Scripts.Render("~/bundles/modernizr")
     @Scripts.Render("~/js/reactjs")
</head>

Next, we speed up server-side rendering by disabling global members in V8

Replace the jsEngineSwitcher sectionGroup with the following

    <sectionGroup name="jsEngineSwitcher">
      <section name="core" type="JavaScriptEngineSwitcher.Core.Configuration.CoreConfiguration, JavaScriptEngineSwitcher.Core" />
      <section name="msie" type="JavaScriptEngineSwitcher.Msie.Configuration.MsieConfiguration, JavaScriptEngineSwitcher.Msie" />
      <section name="jint" type="JavaScriptEngineSwitcher.Jint.Configuration.JintConfiguration, JavaScriptEngineSwitcher.Jint" />
      <section name="v8" type="JavaScriptEngineSwitcher.V8.Configuration.V8Configuration, JavaScriptEngineSwitcher.V8" />
    </sectionGroup>

Replace the jsEngineSwitcher section in Web.config with the following

  <jsEngineSwitcher xmlns="http://tempuri.org/JavaScriptEngineSwitcher.Configuration.xsd">
    <core>
      <engines>
        <add name="V8JsEngine" type="JavaScriptEngineSwitcher.V8.V8JsEngine, JavaScriptEngineSwitcher.V8" />
        <add name="MsieJsEngine" type="JavaScriptEngineSwitcher.Msie.MsieJsEngine, JavaScriptEngineSwitcher.Msie" />
        <add name="JintJsEngine" type="JavaScriptEngineSwitcher.Jint.JintJsEngine, JavaScriptEngineSwitcher.Jint" />
      </engines>
    </core>
    <v8 enableDebugging="false" debugPort="9222" disableGlobalMembers="true" maxNewSpaceSize="0" maxOldSpaceSize="0" maxExecutableSize="0" />
  </jsEngineSwitcher>

We are ready to start writing our first Hello World in React!

Create the folder Scripts/Home and add a JavaScript File named Index.jsx and paste the following

var HomeIndex = React.createClass({
    render: function() {
        return (<div>Hello world!</div>);
    }
});

We will also bundle this JSX file in App_Start\BundleConfig.cs

bundles.Add(new JsxBundle("~/js/home/index").Include(
     "~/Scripts/Home/Index.jsx"
));

Next, we need to include the JSX for server-side rendering, by adding the following in App_Start\ReactConfig.cs

// If you want to use server-side rendering of React components, 
// add all the necessary JavaScript files here. This includes 
// your components as well as all of their dependencies.
// See http://reactjs.net/ for more information. Example:
ReactSiteConfiguration.Configuration
     .AddScript("~/Scripts/Home/Index.jsx");

Replace the contents in Views\Home\Index.cshtml with the following

@{
     ViewBag.Title = "Home Page";
}
@Html.React("HomeIndex", new { }, containerId: "homeIndex")
@Scripts.Render("~/js/home/index")
@Html.ReactInitJavaScript()

Run our application and we have our Hello World!

Next up—making our grid view

React JS in ASP.NET MVC part 1—why Isomorphic JavaScript?

The conflict of rich web application and search engine optimization

Be it Gmail, Facebook or Github, we have come to expect interactivity without having to wait for a full page reload. Specifically, these web applications are single-page applications (SPA), where new content arising from interactions are sent and retrieved via AJAX.
There are many frameworks to make developing SPA easier. AngularJS, EmberJS and KnockoutJS comes to mind. However, these traditional frameworks only address one part of the equation—the browser side. The server side spits out the HTML template and data separately, causing search engines not to be able to index the page properly, for the HTML template contains no information worthy of mention in a search query. There is thus the need for additional effort to develop a classic version of the SPA in order to optimize for search engine.
Can we cut the additional effort and run our browser side SPA code on the server side too? That would generate the classic version straight away!

React comes to the rescue

React does exactly that!
React, Meteor and several other Isomorphic JavaScript frameworks arise to make it easy to have rich single-page applications, while generating SEO friendly pages. What’s more, the initial blank or placeholder page that appears before data is loaded is gone, since the template with data is already rendered at the server side.
With some additional care to place HTML forms and submit buttons and intercepting them to do AJAX, the rich web application can still function even with JavaScript turn off. Talk about progressive enhancement and graceful degradation!

In part 2, set up the React libraries

Get an extra 5GB of storage for Copy when you use this link!

My 25 GB of Dropbox space earned from Space Race is expiring in a month! I'll be left with a meagre 2.5 GB only. ::sob::

Luckily, there's a Dropbox competitor, Copy, offering 15 GB of base storage. What's more, if you use the referral link below, both you and I will get an additional 5GB! That's a lot, especially when Dropbox only gives 10% of that per referral.

Get your 20GB NOW
https://copy.com?r=QPkByG