<jupyter_start><jupyter_text>Arabic Digit Recognition  OverviewThis notebook builds an OCR for handwritten Arabic digits, research in OCR or optical character recognition started a long time ago in order to allow the computer to understand the words in any visual image, but the peak in OCR performances did happen in the deep learning era as it introduced advanced methods and techniques in order to achieve the OCR's outstanding outcomes and uses. The dataset used will be MADbase from the electronics engineering department, in the American University in Cairo, and the CNN will be build using Keras from TensorFlow.  DatasetThe dataset is composed of 70,000 digits written by 700 participants. Each participant wrote each digit (from 0 to 9) twenty times (ten times only used in our database – the other ten times may be used later in writer verification research). To ensure including different writing styles, the database was gathered from different institutions: Colleges of Engineering and Law, School of Medicine, the Open University (whose students span a wide range of ages), a high school, and a governmental institution. Forms were scanned with 300 dpi resolution then digits are automatically extracted, categorized, and bounded by bounding boxes. We adjusted the scanner to produce binary images directly; so we did not need to binarize the resulting images. Some noisy and corrupted digit images were edited manually. The database is partitioned into two sets: a training set (60,000 digits – 6000 images per class) and a test set (10,000 digits – 1000 images per class). Writers of training set and test set are exclusive. Ordering of including writers to test sets are randomized to make sure that writers of test set are not from a single institution (to ensure variability of the test set).http://datacenter.aucegypt.edu/shazeem/  Data Exploring & Preprocessing 1- Let's import our libraries that we will use in this project.<jupyter_code>import tensorflow as tf                                                         # The main framework we will build our model with.
import numpy as np                                                              # Used for mathimatical operations.
import pandas as pd                                                             # Will be used to load our data frame.
import cv2                                                                      # Used for image processing.
from matplotlib import pyplot as plt                                            # Used for plottin our data.
from tensorflow.keras.utils import to_categorical                               # Utility in Tensorflow to convert our true category values.<jupyter_output><empty_output><jupyter_text>2- Now we wil mount google drive for loading our data.<jupyter_code>from google.colab import drive                                                  
drive.mount('/content/gdrive')<jupyter_output>Mounted at /content/gdrive<jupyter_text>3- We will use Pandas library to read our data and load it into our data frame, our data is stored in CSV format so we will use the appropriate function to load it.<jupyter_code>path = '/content/gdrive/MyDrive/Datasets/Kaggle Arabic Digits'                  # Here we specify the path to our data location on my drive
train_data_x = pd.read_csv(path + '/csvTrainImages 60k x 784.csv', header=None) # Then we load the training images.
train_data_y = pd.read_csv(path + '/csvTrainLabel 60k x 1.csv', header=None)    # Training labels.
test_data_x = pd.read_csv(path + '/csvTestImages 10k x 784.csv', header=None)   # Testing images.
test_data_y = pd.read_csv(path + '/csvTestLabel 10k x 1.csv', header=None)      # Testing labels.<jupyter_output><empty_output><jupyter_text>4- Now let's examine our data properties.We will find that we have 60000 training image with their labels in the training set and 10000 in the testing set, along with a total of 10 classes which idecates the number of Arabic digits.The number of pixels in each image is 784, so we can conclude the pixels per image by getting the square root of the number of pixels which will give us 28, so the dimensions of our image is 28x28x1 as it is a grey scale image, we will use this piece of information for preprocessing our images.<jupyter_code>print('We have  %d training images each contains %d pixels.' %(train_data_x.shape[0], train_data_x.shape[1]))
print('We have  %d training labels each contains %d classes.' %(train_data_y.shape[0], len(train_data_y.value_counts())))
print('We have  %d testing images each contains %d pixels.' %(test_data_x.shape[0], test_data_x.shape[1]))
print('We have  %d testing labels each contains %d classes.' %(test_data_y.shape[0], len(test_data_y.value_counts())))<jupyter_output>We have  60000 training images each contains 784 pixels.
We have  60000 training labels each contains 10 classes.
We have  10000 testing images each contains 784 pixels.
We have  10000 testing labels each contains 10 classes.<jupyter_text>We notice also that there are 6000 image per class.<jupyter_code>train_data_y.value_counts()<jupyter_output><empty_output><jupyter_text>Let's see what our images looks like.<jupyter_code>fig = plt.figure(figsize=(8, 8))                                                # Setting the figure size.
columns = 4                                                                     # Selecting the number of columns.
rows = 5                                                                        # Selectin the number of rows.
for i in range(1, columns*rows +1):                                             # Looping through rows & columns.
  img = test_data_x.iloc[i].to_numpy().reshape((28,28))                         # Reshaping the image into its size 32x32
  fig.add_subplot(rows, columns, i)                                             # Adding the image to the plot
  plt.imshow(img, cmap='gray')                                                  # Showing the image using plt
plt.show()                                                                      # Finally shpwing the whole plot containing all the subplots<jupyter_output><empty_output><jupyter_text>5- Now we define a function for us to preprocess the data,We will start with reshaping the image to be with the size of 28x28, so that the training array will be of size of imagesx28x28, then we will pass through each image to flip and rotate them as they are rotated, then we will reshape the entire array with the imagex28x28x1, and the 1 stands for our grey scale images, lastly we will normalize our images by dividing by 255 for pixels normalization.<jupyter_code>def preprocess_data(train_data_x):
  train_data_x = train_data_x.to_numpy().reshape((train_data_x.shape[0], 28, 28)).astype('uint8')
  for i in range(len(train_data_x)):
    train_data_x[i] = cv2.rotate(train_data_x[i], cv2.ROTATE_90_CLOCKWISE)      # Rotating the images.
    train_data_x[i] = np.flip(train_data_x[i], 1)                               # Flipping the images
  train_data_x = train_data_x.reshape([-1, 28, 28, 1]).astype('uint8')          # Reshaping into the required size.
  train_data_x = train_data_x.astype('float32')/255                             # Here we normalize our images.
  return np.asarray(train_data_x)
train_x = preprocess_data(train_data_x)                                         # Returns an array of dimensions (60000,28,28,1).
test_x = preprocess_data(test_data_x)                                           # Returns an array of dimensions (60000,28,28,1).<jupyter_output><empty_output><jupyter_text>6- Now we preprocess our labels by converting them to the categorcal form.<jupyter_code>train_y = to_categorical(train_data_y.values.astype('int32')                    # Returns an array of dimentions (13340, 28).
                         , num_classes=10)
test_y = to_categorical(test_data_y.values.astype('int32')                      # Returns an array of dimentions (3360, 28).
                        , num_classes=10)<jupyter_output><empty_output><jupyter_text>7- We will now shuffle our training and test sets as we will get better results than using the data's classes in sequential form.<jupyter_code>from sklearn.utils import shuffle                                               # Importing shuffle function from sklearn library.
train_x, train_y = shuffle(train_x, train_y)                                    # Now we shuffle x & y in the training set.
test_x, test_y, shuffle(test_x, test_y)                                         # Then x & y in our testing set.<jupyter_output><empty_output><jupyter_text>Building Model 8- We will now create our model's architecture, We will use keras for the creation of our model, we will start by creating a function for use to create our model, we will set the activation, optimizer and our initializing method as variables for us to easly modifiy it in the hyper-parameter tuning phase.We will start by creating our first convolutional layer and setting up the input shape, we will create additional pooling layer along with a batch normalization layer, then we will add three convolutional layers with the same structure but the the double size of filters each layer.Then we will flatten our layer preparing it for the fully connected layers, we will use a small neurons numbered layer with a drop out layer, batch normalization and we will add an L2 regularizer so we will control the overfitting.<jupyter_code>def create_model(activation='relu', optimizer='adam', kernel_initializer='he_normal'):
  model = tf.keras.models.Sequential([
      tf.keras.layers.Conv2D(32, (3,3), padding='same', input_shape=(28, 28, 1), activation= activation, kernel_initializer=kernel_initializer),
      tf.keras.layers.MaxPool2D(2,2),
      tf.keras.layers.BatchNormalization(),
      tf.keras.layers.Conv2D(64, (3,3), padding='same', activation= activation, kernel_initializer=kernel_initializer),
      tf.keras.layers.MaxPool2D(2,2),
      tf.keras.layers.Dropout(0.2),
      tf.keras.layers.BatchNormalization(),
      tf.keras.layers.Conv2D(128, (3,3), padding='same', activation= activation, kernel_initializer=kernel_initializer),
      tf.keras.layers.MaxPool2D(2,2),
      tf.keras.layers.Dropout(0.2),
      tf.keras.layers.BatchNormalization(),

      tf.keras.layers.Flatten(),
      
      tf.keras.layers.Dense(32, activation= activation, kernel_initializer=kernel_initializer, kernel_regularizer='l2'),
      tf.keras.layers.BatchNormalization(),
      tf.keras.layers.Dropout(0.2),
      tf.keras.layers.Dense(10, activation= 'softmax', kernel_initializer=kernel_initializer)
  ])

  model.compile(optimizer=optimizer,
                loss='categorical_crossentropy',
                metrics=['accuracy'])
  return model
model = create_model()                                                          # Now we created an instance of a model with our custom architefture.
model.summary()                                                                 # Then we display our model's summary.<jupyter_output>Model: "sequential_1"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_3 (Conv2D)            (None, 28, 28, 32)        320       
_________________________________________________________________
max_pooling2d_3 (MaxPooling2 (None, 14, 14, 32)        0         
_________________________________________________________________
batch_normalization_4 (Batch (None, 14, 14, 32)        128       
_________________________________________________________________
conv2d_4 (Conv2D)            (None, 14, 14, 64)        18496     
_________________________________________________________________
max_pooling2d_4 (MaxPooling2 (None, 7, 7, 64)          0         
_________________________________________________________________
dropout_3 (Dropout)          (None, 7, 7, 64)          0         
______________________________________________________[...]<jupyter_text>Lets train our model.<jupyter_code>model = create_model(optimizer='adam',                                       # We create our model with the specified hyper parameters
                     kernel_initializer='normal',
                     activation='relu')
from keras.callbacks import ModelCheckpoint                                     # We will import a call back to save the best epoch's weights

checkpointer = ModelCheckpoint(filepath='weights.hdf5', verbose=1, save_best_only=True)
history = model.fit(train_x,                                                    
                    train_y, 
                    validation_split= 0.3,                                      # The model will split the data into 30% of validation.
                    epochs=10,                                                  # We will run the model for 30 epochs
                    batch_size=64,                                              # We will have a batch size of 64
                    callbacks=[checkpointer])                                   # Finally we will use the imported callback<jupyter_output>Epoch 1/10
657/657 [==============================] - 73s 109ms/step - loss: 0.3922 - accuracy: 0.9707 - val_loss: 0.1152 - val_accuracy: 0.9862

Epoch 00001: val_loss improved from inf to 0.11523, saving model to weights.hdf5
Epoch 2/10
657/657 [==============================] - 72s 109ms/step - loss: 0.0821 - accuracy: 0.9902 - val_loss: 0.0798 - val_accuracy: 0.9878

Epoch 00002: val_loss improved from 0.11523 to 0.07978, saving model to weights.hdf5
Epoch 3/10
657/657 [==============================] - 71s 108ms/step - loss: 0.0726 - accuracy: 0.9909 - val_loss: 0.0806 - val_accuracy: 0.9889

Epoch 00003: val_loss did not improve from 0.07978
Epoch 4/10
657/657 [==============================] - 71s 108ms/step - loss: 0.0672 - accuracy: 0.9917 - val_loss: 0.0682 - val_accuracy: 0.9913

Epoch 00004: val_loss improved from 0.07978 to 0.06818, saving model to weights.hdf5
Epoch 5/10
657/657 [==============================] - 71s 108ms/step - loss: 0.0642 - accuracy: 0.9924 - val_loss:[...]<jupyter_text>11- Now lets load the best epoch's weights and then evaluate our model using the test set.<jupyter_code>model.load_weights('weights.hdf5')                                              # Loading the best weights        
model.evaluate(test_x, test_y)                                                  # Evaluating our model<jupyter_output>313/313 [==============================] - 4s 14ms/step - loss: 0.0642 - accuracy: 0.9909<jupyter_text>12- Lets plot our training journy to check the performance and verify that the model is not overfitting.<jupyter_code># PLOT LOSS AND ACCURACY
%matplotlib inline

import matplotlib.image  as mpimg
import matplotlib.pyplot as plt

#-----------------------------------------------------------
# Retrieve a list of list results on training and test data
# sets for each training epoch
#-----------------------------------------------------------
acc=history.history['accuracy']
val_acc=history.history['val_accuracy']
loss=history.history['loss']
val_loss=history.history['val_loss']

epochs=range(len(acc)) # Get number of epochs

#------------------------------------------------
# Plot training and validation accuracy per epoch
#------------------------------------------------
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.title('Training and validation accuracy')
plt.legend(['train', 'val'], loc='upper left')
plt.ylabel('accuracy')
plt.xlabel('epoch')
plt.figure()

#------------------------------------------------
# Plot training and validation loss per epoch
#------------------------------------------------
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.title('Training and validation loss')
plt.legend(['train', 'val'], loc='upper left')
plt.ylabel('accuracy')
plt.xlabel('epoch')


plt.title('Training and validation loss')<jupyter_output><empty_output><jupyter_text>12- Lets save our model for later use<jupyter_code>model.save('/content/my_model/my_model.h5')<jupyter_output><empty_output><jupyter_text>Testing Model Now lets load our model and test it on our custom images.<jupyter_code>model = tf.keras.models.load_model('my_model/my_model.h5')                      # Now we load the model<jupyter_output><empty_output><jupyter_text>In the following function we mapped the categorical output with the Arabic letters to help us better identifing the classes.<jupyter_code>def convert_categorical_label_to_real_label(categorical_label):
  real_labels = []
  real_labels.extend([x for x in range(10)])
  return real_labels[categorical_label]<jupyter_output><empty_output><jupyter_text>Finally lets test them on custom images.<jupyter_code>for i in range(1, 5):
  test = cv2.imread('z' + str(i) + '.PNG')
  test = cv2.cvtColor(test, cv2.COLOR_BGR2GRAY)
  test = cv2.resize(test, (28, 28))
  plt.imshow(test, cmap='gray')
  plt.show()
  test = np.reshape(test, (-1, 28, 28, 1))
  test = test.astype('float32')/255
  print(convert_categorical_label_to_real_label(np.argmax(model.predict(test))))<jupyter_output><empty_output>