# Neural Networks

## Handwriting Recognition Application

User's Manual :

1. Draw a letter (A - Z, a - z) or number（0 - 9) on the canvas.
2. Use the Clear button to erase.
3. The ratios of the pieces in the pie char represents the probabilities of the predictions. The pieces are ordered by area in a clockwise fashion. Only classes with probability greater than 0.05 are labeled.
4. We balanced an unbalanced image data set by generating randomly perturbed images from the existing ones, the resulted data set contains over 2,500,000 28 × 28 pixel images with 62 labels. The predictions are made by a convolutional Neural Network model, trained on the balanced data set.

For Linear Models, target $Y$ (a number or a vector) are approximated by $$f(X) = W \cdot \hat{X}$$ where $\hat{X}= \begin{pmatrix}1\\X \end{pmatrix}$. A natural way to extend the linear model is to take the composition of a linear model with a non-linear function $\sigma$, $$g(X) =\sigma (W \cdot \hat{X})$$ Here, $\sigma$ applies entry-wisely to the vector $W \cdot \hat{X}$. This extension gives us a single-layer neural network, the main goal is to approximate the weights matrix $W$ in the training process.
A multilayer neural network model with $k$ hidden layers is of the form $$X \mapsto (g_k \circ \cdots \circ g_2\circ g_1\circ g_0)(X),\qquad k \in \{0,1,2,3,4,5,\cdots\}$$ The weighs matrices $W_i$ associated to each layer are approximated in the training process.