<jupyter_start><jupyter_text>Mini Project Three: Autonomous Vehicle Driving with CNN![title](Images\car-running.gif)<jupyter_code>from IPython.display import HTML
from IPython.display import Image
HTML('''<script>
code_show=true; 
function code_toggle() {
 if (code_show){
 $('div.input').hide();
 } else {
 $('div.input').show();
 }
 code_show = !code_show
} 
$( document ).ready(code_toggle);
</script>
The raw code for this IPython notebook is by default hidden for easier reading.
To toggle on/off the raw code, click <a href="javascript:code_toggle()">here</a>.''')<jupyter_output><empty_output><jupyter_text>Table of Contents -  1. [Introduction](Introduction)   -  1.1 [Abstract](abstract)   -  1.2 [Importing Libraries](importing_libraries)   -  1.3 [Setting up the Environment](Environment)-  2.0 [Creation of the Dataset](Creation_Dataset)-  3.0 [Training](Training)-  4.0 [Driving](Driving)  -  5. [Conclusion](Conclusion)-  6. [Contribution](Contribution)-  7. [Citation](Citation)-  8. [License](License)   1.0 Introduction     1.1 Abstract  Welcome to Imitation Learning.[Back to top](Introduction)    1.2 Importing Libraries  This is the official start to any Data Science or Machine Learning Project. A Python library is a reusable chunk of code that you may want to include in your programs/ projects. In this step we import a few libraries that are required in our program. Some major libraries that are used are Numpy, Gym, Torch, PIL, Pyglet etc.[Back to top](Introduction)<jupyter_code># Using Numpy for working with arrays
import numpy as np

# The gym library is a collection of test problems — environments
import gym
from gym import envs

# Imageio for creating and saving the image
import imageio

# The OS module in python provides functions for interacting with the operating system. OS
import os

# The sys module provides functions used to manipulate different parts of the Python runtime environment.
import sys

# Pyglet is a library for developing visually rich GUI, Windows may appear as floating regions 
import pyglet
from pyglet.window import key

# We use copy module for shallow and deep copy operations.
import copy 

# PyTorch is a Python package that provides two high-level features:
#- Tensor computation (like NumPy) with strong GPU acceleration
#- Deep neural networks built on a tape-based autograd system

import torch
from torch.autograd import Variable
from torch import optim, nn
from torch.nn import Softmax

# Python Imaging Library for simple image processing such as resizing (scaling), rotation, and trimming (partial cutout)
import PIL

# Importing our Python scripts
from model import CustomModel
from data import transform_driving_image, LEFT, RIGHT, GO, ACTIONS, CustomDataset, get_dataloader

# For making a table in results
from astropy.table import Table, Column

# To ignore warnings
import warnings; warnings.simplefilter('ignore')<jupyter_output><empty_output><jupyter_text>1.3 Setting up the Environment  Before we start with the setup of our environment, we need to install a few pakages which will make our game and neural network work. 1) Gym facilityInstall OpenAI Gym on the machineFollow the instructions at https://github.com/openai/gyminstallation for extensive and deep guide.**Summary of instructions:**- Install Python 3.5+- Clone the gym repo: git clone https://github.com/openai/gym.git- cd gym- Gym installation, with the box2d environments: pip install -e '.[box2d]'Follow the following steps to play the Car Racing Game- cd gym/envs/box2d- python car_racing.py 2) PytorchPytorch is the deep learning framework that we will be using. It makes it possible to build neural networks very simply.Follow the instructions on http://pytorch.org/ for a deep guide.**Summary of instructions:**- Install Python 3.5+- It is recommended to manage PyTorch with Anaconda. Please install Anaconda- Install PyTorch following instructions at https://pytorch.org/get-started/locally/![title](Images\Pytorch_Installation.png)For example this is the setup for my Computer> pip install torch==1.7.0+cpu torchvision==0.8.1+cpu torchaudio===0.7.0 -f https://download.pytorch.org/whl/torch_stable.html The EnvironmentFor this tutorial, we will use the gym library developed by OpenAI. It provides environments (simple games) to develop reinforcement learning algorithms.The environment we will be using is CarRacing-v0 ( https://gym.openai.com/envs/CarRacing-v0/ ). It is about driving a car on a circuit, the objective being to move forward while staying on the track, which contains many turns. The input to the algorithm (the state provided by the environment) is only the image displayed by the environment: we see the car, and the terrain around it.![title](Images\car-racing.png)The idea is to drive the car by analyzing this image.We are going to use this library in a roundabout way: It is designed for reinforcement learning. The objective is in principle to use the rewards (rewards) provided by the environment to learn the optimal strategy without user action. Here we will not be using these rewards.In addition, we will be doing end-to-end learning , which means that the neural network will directly give us the commands to navigate the car. This is not a road detection module, which will then be analyzed by another program (most true autonomous driving systems are made this way). Here, the neural network takes the field matrix as input, and issues a command to be executed (turn left, turn right, continue straight ahead), without any intermediate program.To use the environment, you need to import it like this:>import gym>env = gym.make('CarRacing-v0').envYou can then access several useful functions:- **env.reset() :** Allows you to restart the environment- **env.step(action) :** Allows you to perform the action `action`. This function returns a tuple `state`, `reward`, `done`, `info` containing the state of the game after the action, the reward obtained, doneindicates if the game is finished, and infocontains debug data.- **env.render() :** Displays the game window.Here, the state `state` that will be returned by env.step(action)is the image displayed on the screen (the pixel matrix). It is this data that we will use to steer our car.[Back to top](Introduction)   2.0 Creation of the dataset  ![title](Images\Racing_start.gif)The car is controllable with the arrows of the keyboard.For the rest, we want to train a neural network which will take the game image as input, and as output, return the command to send (left, right, straight). We will focus first on controlling the direction . The speed control will still have to be done with the up and down keys.The first step in training our neural network is to create a dataset. It is about recording a set of images accompanied by their label . We will represent the possible actions with integers:- 0 to indicate to go left- 1 to indicate go right- 2 to indicate to go straightThus, we will save a set of 3000 images in a folder, accompanied by a file labels.txt indicating on each line  label. We have 3 labels, so we save 1000 images of each label for the training set. For the testing set, we will save 600.We have created a train set, which will be used to train the network, and a test set, which will be used to evaluate its performance during training, to know when to interrupt it. Indeed, given the relatively low number of images that we use (3000), there is a risk of overfitting i.e. the network will lose in power of generalization to be better, in the special cases of the training set . This is a situation that we want to avoid, since we want to use our model subsequently in situations that it has not seen. The technique of stopping training before convergence is called early stopping.    >As the epochs go by, the algorithm leans and its error on the training set naturally goes down, and so does its error on the validation set. However, after a while, the validation error stops decreasing and actually starts to go back up. This indicates that the model has started to overfit the training data. With Early Stopping, you just stop training as soon as the validation error reaches the minimum.    We run this piece of code twice as we want to save different datasets for training and testing.    [Back to top](Introduction)<jupyter_code># %%capture
# record_dataset.py

samples_each_classes = 1000
def action_to_id(a):
    if all(a == [-1, 0, 0]): return LEFT
    elif all(a == [1, 0, 0]): return RIGHT
    else:
        return GO

# is_pressed_esc   = False

if __name__=='__main__':
    quit=False
    if len(sys.argv) < 2:
        sys.exit("Usage : python record_dataset.py path")
    
    env = gym.make('CarRacing-v0').env
    
    envs.box2d.car_racing.WINDOW_H = 750
    envs.box2d.car_racing.WINDOW_W = 1200
    env.reset()

    folder = sys.argv[1]
    images = os.path.join(folder, "train_images") 
    labels = os.path.join(folder, "train_labels.txt")
    os.makedirs(images, exist_ok=True)

    a = np.array([0.0, 0.0, 0.0])
    
    def key_press(k, mod):
        global restart
        global quit
        if k == 65307: quit = True
        if k==key.LEFT:  a[0] = -1.0
        if k==key.RIGHT: a[0] = +1.0
        if k==key.UP:    a[1] = +1.0
        if k==key.DOWN:  a[2] = +0.8   # set 1.0 for wheels to block to zero rotation
#         if k==65307 : is_pressed_esc = True    
    def key_release(k, mod):
        if k==key.LEFT  and a[0]==-1.0: a[0] = 0
        if k==key.RIGHT and a[0]==+1.0: a[0] = 0
        if k==key.UP:    a[1] = 0
        if k==key.DOWN:  a[2] = 0

    env.viewer.window.on_key_press = key_press
    env.viewer.window.on_key_release = key_release
    env.reset()
    for i in range(100):
        env.step([0, 0, 0])
        env.render()

    file_labels = open(labels, 'w')
    samples_saved = {a: 0 for a in ACTIONS}

    i = 0
#     while not is_pressed_esc:
    while not quit :  
        env.render(close = False)
        s, r, done, info = env.step(a)
        action_id = action_to_id(a)
        if samples_saved[action_id] < samples_each_classes:
            samples_saved[action_id] += 1
            samples_each_classes
            imageio.imwrite(os.path.join(folder, 'train_images', 'img-%s.jpg' % i ), s)
            file_labels.write('%s %s\n' % ('img-%s.jpg' % i, action_id))
            file_labels.flush()
            i += 1
            print(samples_saved)
#             env.render()       
    env.render(close=True)<jupyter_output>[2020-12-06 01:09:20,051] Making new env: CarRacing-v0<jupyter_text>3.0 Training   Model training with PyTorchPytorch is a python matrix computing and deep learning library. It consists on the one hand in an equivalent of numpy, but usable both on CPU and on GPU. And on the other hand, in a library which allows to calculate the gradient of each operation performed on the data, so as to apply the backpropagation algorithm, At the base of the training of neural networks Pytorch also has a set of modules to assemble, which makes it possible to create neural networks very simply.In pytorch, the basic object is the module. Each module is a function, or an assembly of pytorch functions, which takes as input Tensors (matrices containing data), and emerges another tensor. All the operations performed in this module will be recorded, because the operation graph is necessary for the backpropagation algorithm.![title](Images\Custom_Model.png)**The __init__ function :** here we define the network architecture. Our network is made up of two parts: self.convnet and self.classifier. The part convnet is the convolutional part: it is this part, which is responsible for analyzing the image, and recognizing the shapes. It is made up of two layers of convolution (pattern recognition), followed by a non-linearity (ReLU), and a pooling layer (which makes the output invariant to translations).The second part is the 'classifier', it takes the output of the convolution network, and comes out a size vector num_classes = 3 which represents the score of each action to be performed.The call nn.Sequential creates the layers in succession. The input will pass successively through all these layers, the input of one layer being the output of the previous one.**The forward function :** This function will be called by pytorch when our module is called. We notice the passage from a 2D input to a 1D input between the two parts convnet and classify thanks to the function input = input.view(input.size(0), -1)(the first dimension being the number of images in a batch). It's a shortcut for input = input.view(input.size(0), input.size(1) * input.size(2) * input.size(3)The entry will indeed have 4 dimensions: the first for the batch, the next 2 for the x and y dimensions of the image, and the last for the number of channels of the image: It will be 3 for the 3 colors to entering the network, then each convolution will create new channels while reducing the x and y size. Thus, as the layers progress, the 1st dimension will remain fixed (the number of images in the batch), but the following two will decrease, and the 3rd (channels) will increase. Data PreparationWe are going to create a class Dataset, which will be used by pytorch to load our dataset into memory thanks to its class DataLoder.First of all, we will define the transformations that will be used to preprocess the images, in order to give them as input to the neural network.`from torchvision import transformstransform_driving_image = transforms.Compose([    transforms.CenterCrop(72),    transforms.ToTensor(),    transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5)),])`This transformation performs the following actions:**Crop the image :** transforms.CenterCrop(72)To keep only a square of size 72 pixels, centered in the same way as the image. Indeed, the image that we get of the environment is like this: ![title](Images\Crop_Image.png)We can see that the screen shows an indication bar on the speed and the direction and acceleration controls. If we do not hide it, CNN may learn to associate the commands we give it, with these indications (this is indeed the best indicator to deduce the command to be made from the screen).After cropping, the resulting image is below. CNN will be forced to analyze the road and the position of the car in order to perform its analysis![title](Images\Crop_Image_2.png)We notice that the images provided to CNN are of much lower quality than those displayed by the environment during the game. They are indeed only 96 pixels apart. This will be enough for the neural network to analyze the shapes, and will make the training much faster (because much less neurons will be needed).**Transform the matrix into Tensor pytorch** transforms.ToTensor()The tensor is the base object in pytorch to store data. It is the analogue to a numpy matrix, except that it can be stored on CPU, or on GPU. We need to transform our image into a pytorch tensor before giving it to the neural network as input.We could also use the function tensor.from_numpy(numpy_array)to transform a numpy array into Tensor .**Standardization:** transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))The images provided by PIL have data between 0 and 1. Here we subtract 0.5, and divide by 0.5 in order to have data between -1 and 1, which is more efficient for training a network. neuron (data centered in 0 and variance close to 1).**The __len__ function** should return the length of the dataset. Here is the total number of images.**The __getitem __ (self, index) function** should return the index object index. Here, we load the image corresponding to this index, we apply the transformations to it, then we return the matrix as well as the labels (in the form of Tensor).**Directions** We have encoded the directions in three variables LEFT, RIGHTand GO, which will be used in the different modules. Code for training the neural networkThis code is taken from the tutorial http://pytorch.org/tutorials/beginner/transfer_learning_tutorial.html .The general idea is as follows:At each epoch, we train on the entire train dataset, then we evaluate on the training dataset. The data is loaded using a DataLoader , provided by pytorch (we give it as argument the object Datasetwe created previously).Some important steps:**Wrapping** them `Tensors` in `Variables`: In Pytorch, it is necessary to do this step `data = Variable(tensor)`, because it is the object `Variable` that will keep in memory the gradient of this variable according to the final loss. A variable is in fact a combination of two tensors, that of the data and that of the gradients.**Backpropagation**To perform the backpropagation in pytorch, the following steps are necessary:- **optimizer.zero_grad() :** at each iteration of the loop. This resets the gradients of each parameter to zero.- **loss.backward() :** this will calculate the gradients for each variable by backpropagation according to the loss, and store them in the Variable object- **optimizer.step() :** Modifies each parameter of our model (network weight) in order to minimize the loss.We have explained the intricate workings of CNN in the following notebook:[CNN](./INFO7390_Assignment_3_Mini_Project_Basics_of_Convolutional_Neural_Network.ipynb)[Back to top](Introduction)<jupyter_code># %%capture
# train.py

def train(model, criterion, train_loader, test_loader, max_epochs=50, 
          learning_rate=0.001):
    
    dataloaders = {
        "train":train_loader, "val": test_loader
    }

    optimizer = optim.SGD(model.parameters(), lr=learning_rate, momentum=0.9)

    best_acc = 0
    for epoch in range(max_epochs):
        print('Epoch {}/{}'.format(epoch, max_epochs - 1))
        print('-' * 10)
        # Each epoch has a training and validation phase
        for phase in ['val', 'train']:
            if phase == 'train':
                model.train(True)  # Set model to training mode
            else:
                model.train(False)  # Set model to evaluate mode

            running_loss = 0.0
            running_corrects = 0

            for data in dataloaders[phase]:
                # get the inputs
                inputs, labels = data
                labels = labels.view(labels.size(0))

                inputs, labels = Variable(inputs), Variable(labels)
                optimizer.zero_grad()

                outputs = model(inputs)
                _, preds = torch.max(outputs.data, 1)
                loss = criterion(outputs, labels)

                # backward + optimize only if in training phase
                if phase == 'train':
                    loss.backward()
                    optimizer.step()
                # statistics
                running_loss += loss.data * inputs.size(0)
                running_corrects += torch.sum(preds == labels.data)

            epoch_loss = running_loss / len(dataloaders[phase])
            epoch_acc = running_corrects / len(dataloaders[phase])

            print('{} Loss: {:.4f} Acc: {:.4f}'.format(
                phase, epoch_loss, epoch_acc))

            # deep copy the model
            if phase == 'val' and epoch_acc > best_acc:
                best_acc = epoch_acc
                best_model_wts = copy.deepcopy(model.state_dict())
                torch.save(best_model_wts, "models2/model-%s.weights" % epoch)

    print('Training complete')
    print('Best val Acc: {:4f}'.format(best_acc))
    return best_model_wts
if __name__=='__main__': 
    num_classes = 3
    model = CustomModel()

    train_path = "train_images"
    test_path = "test_images"
    train_loader = get_dataloader(train_path, batch_size=8)
    test_loader = get_dataloader(test_path, batch_size=30)

    loss = nn.CrossEntropyLoss()
    x=train(model, loss, train_loader, test_loader)<jupyter_output>Epoch 0/49
----------
val Loss: 33.0593 Acc: 10.0000
train Loss: 7.5771 Acc: 4.5227
Epoch 1/49
----------
val Loss: 14.7585 Acc: 25.7000
train Loss: 4.4853 Acc: 6.1653
Epoch 2/49
----------
val Loss: 13.9112 Acc: 22.0500
train Loss: 4.1538 Acc: 6.3280
Epoch 3/49
----------
val Loss: 11.4187 Acc: 26.7500
train Loss: 3.9667 Acc: 6.3493
Epoch 4/49
----------
val Loss: 10.9690 Acc: 26.1000
train Loss: 3.9197 Acc: 6.4187
Epoch 5/49
----------
val Loss: 11.9713 Acc: 25.9000
train Loss: 3.8055 Acc: 6.4587
Epoch 6/49
----------
val Loss: 10.2514 Acc: 27.0000
train Loss: 3.7262 Acc: 6.5120
Epoch 7/49
----------
val Loss: 10.5059 Acc: 26.8000
train Loss: 3.6343 Acc: 6.5333
Epoch 8/49
----------
val Loss: 10.9338 Acc: 26.8500
train Loss: 3.5882 Acc: 6.6320
Epoch 9/49
----------
val Loss: 10.3310 Acc: 26.6500
train Loss: 3.4162 Acc: 6.6027
Epoch 10/49
----------
val Loss: 9.8062 Acc: 26.5000
train Loss: 3.3022 Acc: 6.6640
Epoch 11/49
----------
val Loss: 9.2858 Acc: 26.7500
train Loss: 3.2339 Acc:[...]<jupyter_text>4.0 Driving    Driving the car using our modelWe now have our model being trained. We will now use it to automate the steering of the car.Let's take a closer look at what happens in the loop:`s, r, done, info = env.step(a)s = s.copy()        s  = PIL.Image.fromarray(s)  `We get the pixel matrix, and we read it using PIL (so that it is in the same format as the images read by the dataloader during the training)`input = transform_driving_image(s)`We apply the same transformations as in the dataset (cropping of the sides of the image, transformation into Tensor, and normalization between -1 and 1.)`input = Variable(input[None, :], volatile=True)`We convert it `Tensor` to `Variable` to give it as input to the neuron network. The argument `volatile=True` saves memory, by telling the network not to save the operations performed (useful when you don't want to train the model with these examples).`output = Softmax()(model(input))_, index = output.max(1)   index is a tensorindex = index.data[0]   get the integer inside the tensor`We give the image to the network, we get the output. It is a tensor of size 3, each entry corresponds to the score of each action (left, right or straight). The action to choose will be the one with the highest score (we pass it in a Softmax to have an output between 0 and 1). We get the action with the function `max` which returns the max value, and its index.`a[0] = id_to_steer[index] * output.data[0, index] * 0.3   lateral accelerationenv.render()``a[0]` is lateral acceleration. It is given the value 0, 1 or -1 depending on the action chosen by the neuron network. We multiply this action by a coefficient of 0.3 to avoid too abrupt actions, and also by the probability of the action given by the network (this makes it possible to have more important actions if the network is sure of its action, and less important when the network hesitates).**After launching**, you have to control the speed of the car with the up and down keys of the keyboard. The direction will be chosen by the neural network.![title](Images\car-running-auto.gif)[Back to top](Introduction)<jupyter_code># %%capture
# drive.py

id_to_steer = {
    LEFT: -1,
    RIGHT: 1,
    GO: 0,
}

if __name__=='__main__':
    quit=False
    if len(sys.argv) < 2:
        sys.exit("Usage : python drive.py path/to/weights")
    # load the model
    #model_weights = "models2/model-1.weights"
    #model_weights = sys.argv[1]
    model = CustomModel()
    model.load_state_dict(x)

    env = gym.make('CarRacing-v0').env
    env.reset()

    a = np.array([0.0, 0.0, 0.0])

    def key_press(k, mod):
        global restart
        global quit
        if k == 65307: quit = True
        if k==key.LEFT:  a[0] = -1.0
        if k==key.RIGHT: a[0] = +1.0
        if k==key.UP:    a[1] = +1.0
        if k==key.DOWN:  a[2] = +0.8   # set 1.0 for wheels to block to zero rotation
    def key_release(k, mod):
        if k==key.LEFT  and a[0]==-1.0: a[0] = 0
        if k==key.RIGHT and a[0]==+1.0: a[0] = 0
        if k==key.UP:    a[1] = 0
        if k==key.DOWN:  a[2] = 0

    env.viewer.window.on_key_press = key_press
    env.viewer.window.on_key_release = key_release
    env.reset()
    
    # initialisation
    for i in range(50):
        env.step([0, 0, 0])
        env.render()
    
    i = 0
    while not quit :  
        env.render(close = False)
        s, r, done, info = env.step(a)
        s = s.copy()
        # We transform our numpy array to PIL image
        # because our transformation takes an image as input
        s  = PIL.Image.fromarray(s)  
        input = transform_driving_image(s)
        input = Variable(input[None, :], volatile=True)
        output = Softmax()(model(input))
        _, index = output.max(1)
        index = index.data[0].item()
        print(id_to_steer[index])
        a[0] = id_to_steer[index] * output.data[0, index] * 0.3  # lateral acceleration
#         env.render()
    env.render(close=True)
#     env.close()<jupyter_output>[2020-12-12 16:05:04,564] Making new env: CarRacing-v0
StarcoderdataJupyterStructuredCleanDedup