# ---
# jupyter:
#   jupytext:
#     text_representation:
#       extension: .py
#       format_name: light
#       format_version: '1.5'
#       jupytext_version: 1.14.4
#   kernelspec:
#     display_name: Python [default]
#     language: python
#     name: python3
# ---

# # Developing an AI application
#
# Going forward, AI algorithms will be incorporated into more and more everyday applications. For example, you might want to include an image classifier in a smart phone app. To do this, you'd use a deep learning model trained on hundreds of thousands of images as part of the overall application architecture. A large part of software development in the future will be using these types of models as common parts of applications. 
#
# In this project, I'll train an image classifier to recognize different species of flowers. You can imagine using something like this in a phone app that tells you the name of the flower your camera is looking at. In practice you'd train this classifier, then export it for use in your application. I'll be using [this dataset](http://www.robots.ox.ac.uk/~vgg/data/flowers/102/index.html) of 102 flower categories, you can see a few examples below. 
#
# <img src='assets/Flowers.png' width=500px>
#
# The project is broken down into multiple steps:
#
# * Load and preprocess the image dataset
# * Train the image classifier on the dataset
# * Use the trained classifier to predict image content

# Imports here
% matplotlib inline
% config InlineBachend.figure_format = 'retina'
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os
from PIL import Image
from collections import OrderedDict
import torch
from torch import optim
import torch.nn as nn
import torch.nn.functional as F
from torchvision import transforms, datasets, models


# ## Load the data
#
# Here I'll use `torchvision` to load the data ([documentation](http://pytorch.org/docs/0.3.0/torchvision/index.html)). The data should be included alongside this notebook, otherwise you can [download it here](https://s3.amazonaws.com/content.udacity-data.com/nd089/flower_data.tar.gz). The dataset is split into three parts, training, validation, and testing. For the training, I'll apply transformations such as random scaling, cropping, and flipping. This will help the network generalize leading to better performance. I'll also make sure the input data is resized to 224x224 pixels as required by the pre-trained networks.
#
# The validation and testing sets are used to measure the model's performance on data it hasn't seen yet. For this I don't want any scaling or rotation transformations, but I'll need to resize then crop the images to the appropriate size.
#
# The pre-trained networks I'll use were trained on the ImageNet dataset where each color channel was normalized separately. For all three sets I'll need to normalize the means and standard deviations of the images to what the network expects. For the means, it's `[0.485, 0.456, 0.406]` and for the standard deviations `[0.229, 0.224, 0.225]`, calculated from the ImageNet images.  These values will shift each color channel to be centered at 0 and range from -1 to 1.
#  

data_dir = 'flowers'
train_dir = data_dir + '/train'
valid_dir = data_dir + '/valid'
test_dir = data_dir + '/test'

# +
# Define transforms for the training, validation, and testing sets
normalize = transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])

train_transforms = transforms.Compose([transforms.RandomRotation(45),
                                       transforms.RandomResizedCrop(224), 
                                       transforms.RandomHorizontalFlip(),
                                       transforms.ToTensor(),
                                       normalize,])

valid_transforms = transforms.Compose([ transforms.Resize(256),
                                        transforms.RandomCrop(224), 
                                        transforms.ToTensor(),
                                        normalize,])

# Load the datasets with ImageFolder
trainset = datasets.ImageFolder(train_dir, transform=train_transforms)
validset = datasets.ImageFolder(valid_dir, transform=valid_transforms)
testset = datasets.ImageFolder(test_dir, transform=valid_transforms)

# Using the image datasets and the trainforms, define the dataloaders
trainloader = torch.utils.data.DataLoader(trainset, batch_size=64, shuffle=True)
validloader = torch.utils.data.DataLoader(validset, batch_size=64)
testloader = torch.utils.data.DataLoader(testset, batch_size=64)
# -

# ### Label mapping
#
# I'll also need to load in a mapping from category label to category name. You can find this in the file `cat_to_name.json`. It's a JSON object which I can read in with the [`json` module](https://docs.python.org/2/library/json.html). This will give me a dictionary mapping the integer encoded categories to the actual names of the flowers.

# +
import json

with open('cat_to_name.json', 'r') as f:
    cat_to_name = json.load(f)
# -

# # Building and training the classifier
#
# Now that the data is ready, it's time to build and train the classifier. As usual, I should use one of the pretrained models from `torchvision.models` to get the image features. Build and train a new feed-forward classifier using those features.
#
# * Load a [pre-trained network](http://pytorch.org/docs/master/torchvision/models.html) (The VGG networks work great and are straightforward to use)
# * Define a new, untrained feed-forward network as a classifier, using ReLU activations and dropout
# * Train the classifier layers using backpropagation using the pre-trained network to get the features
# * Track the loss and accuracy on the validation set to determine the best hyperparameters
#

# +
# Build and train the network
model = models.vgg19(pretrained=True)

for param in model.parameters():
    param.requires_grad = False
    
from collections import OrderedDict

classifier = nn.Sequential(OrderedDict([
                                      ('fc1', nn.Linear(25088, 2960,  bias=True)),
                                      ('Relu1', nn.ReLU()),
                                      ('Dropout1', nn.Dropout(p = 0.5)),
                                      ('fc2', nn.Linear(2960, 102,  bias=True)),
                                      ('output', nn.LogSoftmax(dim=1))
                                       ]))

model.classifier = classifier
criterion = nn.NLLLoss()
optimizer = optim.Adam(model.classifier.parameters(), lr=0.001)

def validation(model, testloader, criterion):
    test_loss = 0
    accuracy = 0
    for inputs, labels in testloader:
        inputs, labels = inputs.to('cuda'), labels.to('cuda')
        output = model.forward(inputs)
        test_loss += criterion(output, labels).item()
        ps = torch.exp(output)
        equality = (labels.data == ps.max(dim=1)[1])
        accuracy += equality.type(torch.FloatTensor).mean()
    return test_loss, accuracy

epochs = 8
running_loss = 0

from workspace_utils import keep_awake

for i in keep_awake(range(1)):
    for e in range(epochs):
        model.train()
        model.to('cuda')
        for i, (inputs, labels) in enumerate(trainloader):
            inputs, labels = inputs.to('cuda'), labels.to('cuda')
            optimizer.zero_grad()
            outputs = model.forward(inputs)
            loss = criterion(outputs, labels)
            loss.backward()
            optimizer.step()
            running_loss += loss.item()
            if i % 100 == 99:
                model.eval()
                with torch.no_grad():
                    test_loss, accuracy = validation(model, validloader, criterion)
                    print("Epoch: {}/{}.. ".format(e+1, epochs),
                          "Training Loss: {:.3f}.. ".format(running_loss/100),
                          "Test Loss: {:.3f}.. ".format(test_loss/len(validloader)),
                          "Test Accuracy: {:.3f}".format(accuracy/len(validloader)))
                
                running_loss = 0
                model.train()
            
# -

# ## Testing your network
#
# It's good practice to test the trained network on the test data, images the network has never seen either in training or validation. This will give you a good estimate for the model's performance on completely new images.

# +
# Validation on the test set

with torch.no_grad():   
    test_loss, accuracy = validation(model, testloader, criterion)
    print("Test Loss: {:.3f}.. ".format(test_loss/len(testloader)),
          "Test Accuracy: {:.3f}".format(accuracy/len(testloader)))

# -

# ## Save the checkpoint
#
# Now that the network is trained, I'll save the model so I can load it later for making predictions. I should also save other things such as the mapping of classes to indices which you get from one of the image datasets: `image_datasets['train'].class_to_idx`. I'll attach this to the model as an attribute which makes inference easier later on.
#
# ```model.class_to_idx = image_datasets['train'].class_to_idx```
#
# I'll want to completely rebuild the model later so I can use it for inference. I'll make sure to include any information I'll need in the checkpoint. If I want to load the model and keep training, I'll want to save the number of epochs as well as the optimizer state, `optimizer.state_dict`. I'll likely want to use this trained model in the next part of the project, so best to save it now.

# +
# Save the checkpoint
model.class_to_idx = trainset.class_to_idx

checkpoint = {'input_size': 25088,
              'output_size': 102,
              'epoch': epochs,
              'state_dict': model.state_dict(),
              'class_to_idx': model.class_to_idx,
              'optimizer': optimizer.state_dict()
              }

torch.save(checkpoint, 'checkpoint.pth')

# -

# ## Loading the checkpoint
#
# At this point it's good to write a function that can load a checkpoint and rebuild the model. That way I can come back to this project and keep working on it without having to retrain the network.

# +
# Write a function that loads a checkpoint and rebuilds the model


def load_checkpoint(filename='checkpoint.pth'):
    
    checkpoint = torch.load(filename)
    model = models.vgg19(pretrained=True)
    
    for param in model.parameters():
        param.requires_grad = False
    model.class_to_idx = checkpoint['class_to_idx']
    classifier = nn.Sequential(OrderedDict([ ('fc1', nn.Linear(25088, 2960,  bias=True)),
                                                ('Relu1', nn.ReLU()),
                                                ('Dropout1', nn.Dropout(p = 0.5)),
                                                ('fc2', nn.Linear(2960, 102,  bias=True)),
                                                ('output', nn.LogSoftmax(dim=1))
                                                 ]))
    
    model.classifier = classifier
    model.load_state_dict(checkpoint['state_dict'])
    
    return model

model = load_checkpoint()


# -

# # Inference for classification
#
# Now I'll write a function to use a trained network for inference. That is, I'll pass an image into the network and predict the class of the flower in the image. I'll write a function called `predict` that takes an image and a model, then returns the top $K$ most likely classes along with the probabilities. It should look like 
#
# ```python
# probs, classes = predict(image_path, model)
# print(probs)
# print(classes)
# > [ 0.01558163  0.01541934  0.01452626  0.01443549  0.01407339]
# > ['70', '3', '45', '62', '55']
# ```
#
# First I'll need to handle processing the input image such that it can be used in my network. 
#
# ## Image Preprocessing
#
# I'll use `PIL` to load the image ([documentation](https://pillow.readthedocs.io/en/latest/reference/Image.html)). It's best to write a function that preprocesses the image so it can be used as input for the model. This function should process the images in the same manner used for training. 
#
# First, I'll resize the images where the shortest side is 256 pixels, keeping the aspect ratio. This can be done with the [`thumbnail`](http://pillow.readthedocs.io/en/3.1.x/reference/Image.html#PIL.Image.Image.thumbnail) or [`resize`](http://pillow.readthedocs.io/en/3.1.x/reference/Image.html#PIL.Image.Image.thumbnail) methods. Then you'll need to crop out the center 224x224 portion of the image.
#
# Color channels of images are typically encoded as integers 0-255, but the model expected floats 0-1. I'll need to convert the values. It's easiest with a Numpy array, which I can get from a PIL image like so `np_image = np.array(pil_image)`.
#
# As before, the network expects the images to be normalized in a specific way. For the means, it's `[0.485, 0.456, 0.406]` and for the standard deviations `[0.229, 0.224, 0.225]`. I'll subtract the means from each color channel, then divide by the standard deviation. 
#
# And finally, PyTorch expects the color channel to be the first dimension but it's the third dimension in the PIL image and Numpy array. I can reorder dimensions using [`ndarray.transpose`](https://docs.scipy.org/doc/numpy-1.13.0/reference/generated/numpy.ndarray.transpose.html). The color channel needs to be first and retain the order of the other two dimensions.

def process_image(image_path):
    ''' Scales, crops, and normalizes a PIL image for a PyTorch model,
    
        returns an Numpy array
    '''
    
    # Process a PIL image for use in a PyTorch model
    image = Image.open(image_path)
    width , height = image.size
    image.thumbnail((1000000, 256)) if width > height else image.thumbnail((256, 200000)) 
    width, height = image.size
    left = (width - 224)/2.
    top = (height - 224)/2.
    right = (width + 224)/2.
    bottom = (height + 224)/2.
    im = image.crop((left, top, right, bottom))
    np_im = np.array(im)/255
    mean = np.array([0.485, 0.456, 0.406])
    std = np.array([0.229, 0.224, 0.225])
    np_im = (np_im - mean)/std
    np_im = np_im.transpose((2, 0, 1))
    
    return np_im


# To check my work, the function below converts a PyTorch tensor and displays it in the notebook. If my `process_image` function works, running the output through this function should return the original image (except for the cropped out portions).

# +
def imshow(image, ax=None, title=None):
    
    if ax is None:
        fig, ax = plt.subplots()
    
    # PyTorch tensors assume the color channel is the first dimension
    # but matplotlib assumes is the third dimension
    image = image.transpose((1, 2, 0))
    
    # Undo preprocessing
    mean = np.array([0.485, 0.456, 0.406])
    std = np.array([0.229, 0.224, 0.225])
    image = std * image + mean
    
    # Image needs to be clipped between 0 and 1 or it looks like noise when displayed
    image = np.clip(image, 0, 1)
    
    ax.imshow(image)
    
    return ax


image_path = '/home/workspace/aipnd-project/flowers/train/1/image_06734.jpg'
np_im = process_image(image_path)
# -

imshow(np_im)


# ## Class Prediction
#
# Once I can get images in the correct format, it's time to write a function for making predictions with my model. A common practice is to predict the top 5 or so (usually called top-$K$) most probable classes. I'll want to calculate the class probabilities then find the $K$ largest values.
#
# To get the top $K$ largest values in a tensor I'll use [`x.topk(k)`](http://pytorch.org/docs/master/torch.html#torch.topk). This method returns both the highest `k` probabilities and the indices of those probabilities corresponding to the classes. I need to convert from these indices to the actual class labels using `class_to_idx` which I added to the model.
#
# Again, this method should take a path to an image and a model checkpoint, then return the probabilities and classes.
#
# ```python
# probs, classes = predict(image_path, model)
# print(probs)
# print(classes)
# > [ 0.01558163  0.01541934  0.01452626  0.01443549  0.01407339]
# > ['70', '3', '45', '62', '55']
# ```

# +
def predict(image_path, model, topk=5):
    ''' Predict the class (or classes) of an image using
        a trained deep learning model.
    '''
    
    # Implement the code to predict the class from an image file
    image = process_image(image_path)
    img = torch.from_numpy(image).type(torch.FloatTensor)
    image = img.unsqueeze(0)
    image = image.float().cuda()
    
    model.eval()
    model.to('cuda')
    
    with torch.no_grad():
        model = model.double()
        output = model.forward(image.double())
    
    ps = torch.exp(output)
    
    probs, classes = torch.topk(ps, topk)
    
    probs = probs.cpu().numpy().tolist()[0]
    classes = classes.cpu().numpy().tolist()[0]
    
    idx_to_class = {val: key for key, val in model.class_to_idx.items()}
    
    classes = [idx_to_class[x] for x in classes]
    flowers = [cat_to_name[str(x)] for x in classes]
    
    return probs, flowers


probs, flowers = predict(image_path, model, topk=5)


# -

# ## Sanity Checking
#
# Now that I can use a trained model for predictions, I'll check to make sure it makes sense. Even if the testing accuracy is high, it's always good to check that there aren't obvious bugs. I'll Use `matplotlib` to plot the probabilities for the top 5 classes as a bar graph, along with the input image. It should look like this:
#
# <img src='assets/inference_example.png' width=300px>
#
# I'll convert from the class integer encoding to actual flower names with the `cat_to_name.json` file.

# Display an image along with the top 5 classes
def view_classify(image_path, probs, flowers):  
    ''' Function for viewing an image and it's predicted classes.
    '''
    
    df = pd.DataFrame({'ps':probs, 'clas': flowers})
    im= Image.open(image_path)
    plt.figure()
    plt.subplot(2,1,1)
    plt.imshow(np.array(im))
    plt.show()
    plt.axis('off')
    plt.subplot(2,1,2)
    sns.barplot(x="ps", y="clas", data=df, orient = "h")   
    plt.tight_layout()
    plt.show() 


view_classify(image_path, probs, flowers)

StarcoderdataJupyterScriptsDedupFiltered