<jupyter_start><jupyter_text>Lab 1 Before We BeginRun the following cell of code and type in your cse account credentials. It will download the essential data for our uses. Please do it in Google Collab since they have better internet connection compared to us.Don't worry about your password being revealed as it will be hidden.**Remove the following cell before you submit!! Otherwise, it will not be graded correctly**<jupyter_code>"""
    Download neccesary files for sanity check
"""
username = input("Please enter your username: ")
import getpass
password = getpass.getpass("Please enter your password: ")
url = f'https://{username}:{password}@course.cse.ust.hk/comp2211/labs/Lab1/checking.zip'
!wget $url -O checking.zip
!unzip checking.zip
"""
    Download CIFAR10 dataset from https://www.cs.toronto.edu/~kriz/cifar.html and extract the tarball
"""
!wget 'https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz' -O cifar-10-python.tar.gz
!tar xf cifar-10-python.tar.gz
!rm cifar-10-python.tar.gz
"""
    Download pictures of a present
"""
!wget 'https://cdn.pixabay.com/photo/2017/11/07/19/23/santa-claus-2927962_960_720.png' -O present.png<jupyter_output><empty_output><jupyter_text>Array Transpose and ReshapingIn machine learning, basically all the calculation are done using multi-dimensional arrays. (Just remember this as a fact and you'll see very soon in later labs.) And since each multi-dimension array contains many different values, operations on these array is also a bit more complicated.One of the most common error we would encountered is the mismatching of shapes. We're not talking about the kind of mismatch happening when we trying to broadcast the array, the kind of mismatching usuaslly happens because of the order of the array.To solve this issues, we have mainly two different functions to help us to achieve this goal. ```numpy.ndarray.transpose``` and ```numpy.ndarray.reshape```. While sometimes they might produce similar result, they're actually very different. Let's consider the following two arrays,<jupyter_code>import numpy as np

x = np.arange(6).reshape((2,3))
y = np.arange(6).reshape((3,2))
z = np.arange(6).reshape((3,2))

print(x)
print(y.transpose((1,0)))
print(z.reshape(2,3))<jupyter_output>[[0 1 2]
 [3 4 5]]
[[0 2 4]
 [1 3 5]]
[[0 1 2]
 [3 4 5]]<jupyter_text>As you can see, while y and z started the same, they ended up being two very different arrays even though they share the same shape. This is because, in mathematical sense, a reshaped matrix is actually a completely different matrix. There is infact no such operations that resemble the functions of ```reshape```. However, transpose is an actual mathematical concepts in linear algebra. In fact, it is simply flipping the matrix diagonally (draw a diagonal line from left top to right bottom, and flip along that axis).<jupyter_code>print(y)
print(y.transpose(1, 0))<jupyter_output>[[0 1]
 [2 3]
 [4 5]]
[[0 2 4]
 [1 3 5]]<jupyter_text>You can also say transpose is to permutate the order of the dimension of the numpy array. As demonstrated here<jupyter_code>x = np.arange(8).reshape(2,2,2)
print(x[0, :, 0]) # The second axis of x
x_1 = x.transpose((1, 0, 2)) # x_1 equals to swapping the first and second axis
print(x_1[:, 0, 0])  # The first axis of x_1<jupyter_output>[[[0 1]
  [2 3]]

 [[4 5]
  [6 7]]]
[[[0 1]
  [4 5]]

 [[2 3]
  [6 7]]]<jupyter_text>More on `reshape`So you might ask, why do we need to reshape the array at all if most of the time it doesn't make any mathematical sense? One of the answers is that sometimes, you might have enough axes. For example, in our review, we mentioned we can add new axis using reshape. The reshape would always be valid as long as the product of each axis is the same as the total amount of elements in the array. For example<jupyter_code>x = np.arange(6)
x.reshape((2, 3)) # valid
x.reshape((6, 1)) # valid
x.reshape((2, 3, 1, 1, 1, 1, 1, 1)) # valid, and probably too much axes.
x.reshape((2, 3, -1)) # valid. -1 means let numpy decide the size of that axis
# x.reshape((3, 3)) # invalid, 3x3=9, but only 6 elements are presented.<jupyter_output><empty_output><jupyter_text>Part 1: Extracting Images from CIFAR10Let's do some exercise to get familiar with these functions.CIFAR10 is a very famous small curated dataset among CV-related machine learning research. Due to its size and label quality, it is a very good toy examples to be used for testing, prototyping our models. Therefore, it would be nice for us to know how to extract images from it as they do have some bizarre way to store their images.In the ```cifar-10-batches-py```, there are few files and in this lab we're interested in ```data_batch_1```. This file is $\frac{1}{5}$ of the whole dataset which contains 10k images. In the following cell, I already wrote some code to load the data. ```images``` contains the raw pixel data of the images where ```filenames``` contains the filenames of each raw images. Both arrays have the same order and dimension in the first axis.<jupyter_code>import pickle
import matplotlib.pyplot as plt
with  open('./cifar-10-batches-py/data_batch_1', 'rb') as f:
    data = pickle.load(f, encoding='bytes')
images = data[b'data']
filenames = np.array(data[b'filenames'])
data[b'data'].shape

from PIL import Image
def check_all_pixels(x, y):
    x = np.array(Image.open(x).convert('RGB'))
    y = np.array(Image.open(y).convert('RGB'))
    return np.all(x == y) # Return True if all pixels are the same<jupyter_output><empty_output><jupyter_text>Task 1.1: Finding Images with FilenamesNow, using Boolean indexing, please extract the images with name ```b'coupe_s_002178.png'``` and ```b'capreolus_capreolus_s_000073.png'```. **The 'b' in front of the string cannot be ignored** (For details please search ```Byte string``` in Google).<jupyter_code>"""
    TODO: Extract the three images from ```images``` where their file name is in 
    [b'coupe_s_002178.png', b'capreolus_capreolus_s_000073.png', b'leptodactylus_pentadactylus_s_000004.png'] and save to the variable ```extracted_images```.
    
    Hint: You might use ```np.logical_or``` or ```|``` which are the same. (Please review the review section if you forgot)
    You might also want to place appropriate parathesis ```()``` to avoid ambiguity if you're using ```|```.
"""
### START CODE HERE ### (≈ 1 lines of code)
extracted_images = None
### END CODE HERE ###
### Sanity Check ###
extracted_images.shape == (3, 3072) # It should have shape (3, 3072)<jupyter_output><empty_output><jupyter_text>Task 1.2: Reshape and TransposeNow you have extracted raw data of the three images, and we now want to plot it using libraries. However, we can't plot it directly as our libraries expect our image to have a shape of ```(H, W, C)``` where H, W is the height and width of the images respectively and C represent the number of channels. Since we're using RGB to represent our image, ```C=3```.<jupyter_code># plt.imshow(extracted_images[0]) # We can't plot it directly!
"""
    TODO: Reshape the array such that it is a 4D array with shape (3, 32, 32, 3) and the image looks the same as the provided samples. (look at the png in the folder)
    Save to the variable ```reshaped_images```.
"""
### YOUR CODE HERE ### (≈ 1 lines of code)
reshaped_images = None
### END CODE HERE ###

for i, le_image in enumerate(reshaped_images):
    fig = plt.figure(frameon=False)
    plt.axis('off')
    plt.imshow(le_image)
    plt.savefig(f'./cifar_{i}_submit.png', bbox_inches='tight', pad_inches=0)<jupyter_output><empty_output><jupyter_text>Task 1.3 Visualization of Transpose<jupyter_code>"""
    TODO: Now, try swapping the first and second axis of the image (or second or third axis of the 4d array), how are these images compared to the original images?
    Save to the variable ```transposed_images```.
"""
### START CODE HERE ### (≈ 1 lines of code)
transposed_images = None
### END CODE HERE ###

for i, le_image in enumerate(transposed_images):
    fig = plt.figure(frameon=False)
    plt.axis('off')
    plt.imshow(le_image)
    plt.savefig(f'./cifar_{i}_t_submit.png', bbox_inches='tight', pad_inches=0)
for i in range(3):
    isSame = check_all_pixels(f'./cifar_{i}_submit.png', f'cifar_{i}_check.png')
    print(f'cifar_{i}_submit.png and cifar_{i}_check.png are {("same" if isSame else "different")}')
    
for i in range(3):
    isSame = check_all_pixels(f'./cifar_{i}_t_submit.png', f'cifar_{i}_t_check.png')
    print(f'cifar_{i}_t_submit.png and cifar_{i}_t_check.png are {("same" if isSame else "different")}')<jupyter_output><empty_output><jupyter_text>Task 2 Image Enhancement (Image Augmentation): Contrast AdjustmentIf you have fiddled with your electronics devices for a bit, youg might have heard of contrast adjustment. In short, it's a way to make the image looks dull and sometimes, it can make the image clearer. In this lab, I'm going to walk you through one of the simpler way to adjust contrast.First, let's talk about how an image is represented in multi-dimensional programming.Usually, images are represented in multi-dimensional arrays with shape either like (H, W, C) or (C, H, W) (Which we have shown in previous task). And normally, an Image contains 3 channels. R, G, B. In some sense, you can think as if three images of different channel got smashed and turned into one images. Task 2.1: Grey scaleFor the contrast adjustment, we first need to convert our image to grey scale. To do so, we would calculate the new pixel value like this:$$    0.299 * R + 0.587 * G + 0.114 * B$$For the purpose of teaching numpy, I would like to ask you to use ```np.dot``` (or ```@```) (But you can also do it on each channel individually). It does matrix multiplication (in linear algebra way instead of elementwise). The shape of the matrices would look like: ```(m, n)@(n, k)=(m, k)```<jupyter_code>image = np.array(Image.open('present.png').convert('RGB'))
plt.imshow(image)
"""
    Save greyscaled version of the image to the variable ```grey_image``` according to the given
"""
### START CODE HERE ### (≈ 1 lines of code)
grey_image = None
### END CODE HERE ###
plt.imshow(grey_image, cmap='gray', vmin=0, vmax=255)<jupyter_output><empty_output><jupyter_text>Task 2.2 Take RatioLastly, the new pixel value of the adjust image would be:$$    I_{new, ij} = \alpha\times grey_{ij} + (1-\alpha)\times I_{ori, ij}$$where $\alpha\in[0, 1]$ and in this example, we take $\alpha=0.5$(Hint: the operations in this formula are pixle-wise, as hinted by the subscript ${}_{ij}$. You might want to reshape the greyscale image depends on how you code the previous step)<jupyter_code>alpha = 0.5
### START CODE HERE ### (≈ 1 lines of code)
new_image = None
### END CODE HERE ###

plt.axis('off')
plt.imshow(new_image.astype(np.uint8))
plt.savefig(f'./present_submit.png', bbox_inches='tight', pad_inches=0)
isSame = check_all_pixels('./present_submit.png', './present_check.png')
print(f'present_submit.png and present_check.png are {("same" if isSame else "different")}')<jupyter_output><empty_output><jupyter_text>This is not a very good contrast adjusting algorithm  SubmissionPlease export this as ```.py``` file and submit it. Please make sure there's no error in **every cell**. And remove (or comment) every line that start with **```!```**.ZINC system will evaluate your code so no need to submit any .zip or image files.<jupyter_code>### This is the end of the lab
### Created by <NAME> for the purposes of teaching COMP2211<jupyter_output><empty_output>