<jupyter_start><jupyter_text>CNN - 1D - Toxic CommentsA corpus of manually labeled comments - classifying each comment by its type of toxicity is available on Kaggle. We will aim to do a binary classification of whether a comment is toxic or notApproach:- Learning Embedding with the Task- 1D - Convolution Neural Networks to model the sequence data<jupyter_code>import numpy as np
import pandas as pd
import keras
import matplotlib.pyplot as plt
%matplotlib inline
import vis<jupyter_output>Using TensorFlow backend.<jupyter_text>Get the Data Uncomment these shell lines to get the data<jupyter_code># !wget http://bit.do/deep_toxic_train -P data/
# !mv data/deep_toxic_train data/train.zip
df = pd.read_csv("data/train.zip")
df.head()<jupyter_output><empty_output><jupyter_text>Import the required libraries<jupyter_code>from keras.preprocessing.text import Tokenizer
from keras.preprocessing.sequence import pad_sequences<jupyter_output><empty_output><jupyter_text>Create the Input & Output Data<jupyter_code>train_sentences = df["comment_text"]
train_sentences.head()<jupyter_output><empty_output><jupyter_text>**Pre-processing the train data**- Tokenization: "This is an apple" -> ["This", "is", "an", "apple"]- Indexing: {0: "This", 1: "is", 2: "an", 3: "apple"}- Index Representation: [0, 1, 2, 3]<jupyter_code>from keras.preprocessing.text import Tokenizer
from keras.preprocessing.sequence import pad_sequences
# Tokenizer
max_words = 20000
tokenizer = Tokenizer(num_words=max_words, oov_token='UNK')
tokenizer.fit_on_texts(list(train_sentences))<jupyter_output><empty_output><jupyter_text>Tokenizer Fix from https://github.com/keras-team/keras/issues/8092<jupyter_code>tokenizer.word_index = {e:i for e,i in tokenizer.word_index.items() if i <= max_words} # <= because tokenizer is 1 indexed
tokenizer.word_index[tokenizer.oov_token] = max_words + 1
# Index Representation
tokenized_train = tokenizer.texts_to_sequences(train_sentences)
# Selecting Padding
# find length of each sentence and plot the length
number_of_words = [len(comment) for comment in tokenized_train]
plt.hist(number_of_words, bins = np.arange(0, 500, 10));
# Padding to make it uniform
maxlen = 200
X = pad_sequences(tokenized_train, maxlen = maxlen)
labels = df.iloc[:,2].values
# Baseline Benchmark 
1 - df.iloc[:,2].sum()/df.iloc[:,2].count()
from keras.utils import to_categorical
y = to_categorical(labels)
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
X_train.shape, X_test.shape, y_train.shape, y_test.shape<jupyter_output><empty_output><jupyter_text>Step 2: Create the Model Architecture<jupyter_code>from keras.models import Sequential
from keras.layers import Dense, Conv1D, MaxPooling1D, GlobalMaxPooling1D, Embedding, Dropout
model = Sequential()
model.add(Embedding(max_words, output_dim=128))
model.add(Conv1D(128, 7, activation='relu'))
model.add(MaxPooling1D(5))
model.add(Conv1D(128, 7, activation='relu'))
model.add(GlobalMaxPooling1D())
model.add(Dense(64, activation="relu"))
model.add(Dropout(0.25))
model.add(Dense(2, activation='sigmoid'))
model.summary()<jupyter_output>_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
embedding_1 (Embedding)      (None, None, 128)         2560000   
_________________________________________________________________
conv1d_1 (Conv1D)            (None, None, 128)         114816    
_________________________________________________________________
max_pooling1d_1 (MaxPooling1 (None, None, 128)         0         
_________________________________________________________________
conv1d_2 (Conv1D)            (None, None, 128)         114816    
_________________________________________________________________
global_max_pooling1d_1 (Glob (None, 128)               0         
_________________________________________________________________
dense_1 (Dense)              (None, 64)                8256      
_________________________________________________________________
dropout_1 [...]<jupyter_text>Step 3: Compile the Model & Fit on the Data<jupyter_code>model.compile(loss='binary_crossentropy', optimizer='rmsprop', metrics=['accuracy'])
output = model.fit(X_train, y_train, batch_size=128, epochs=5, validation_split=0.2)<jupyter_output>Train on 102124 samples, validate on 25532 samples
Epoch 1/5
102124/102124 [==============================] - 26s 252us/step - loss: 0.1941 - acc: 0.9332 - val_loss: 0.1595 - val_acc: 0.9442
Epoch 2/5
102124/102124 [==============================] - 21s 201us/step - loss: 0.1501 - acc: 0.9483 - val_loss: 0.1572 - val_acc: 0.9468
Epoch 3/5
102124/102124 [==============================] - 21s 201us/step - loss: 0.1388 - acc: 0.9514 - val_loss: 0.1584 - val_acc: 0.9461
Epoch 4/5
102124/102124 [==============================] - 21s 204us/step - loss: 0.1260 - acc: 0.9560 - val_loss: 0.1679 - val_acc: 0.9412
Epoch 5/5
102124/102124 [==============================] - 21s 202us/step - loss: 0.1117 - acc: 0.9614 - val_loss: 0.1706 - val_acc: 0.9454<jupyter_text>Step 4: Evaluate the Model<jupyter_code>vis.metrics(output.history)
score = model.evaluate(X_test, y_test, verbose=1)
print('Test loss:', score[0])
print('Test accuracy:', score[1])<jupyter_output>Test loss: 0.17662850376192293
Test accuracy: 0.9449788500704998<jupyter_text>Step 5: Visualise evaluation & Make a prediction<jupyter_code>predict_classes = model.predict_classes(X_test)
actual_classes = np.dot(y_test,np.array([[0],[1]])).reshape(-1)
pd.crosstab(actual_classes, predict_classes)<jupyter_output><empty_output>
StarcoderdataJupyterStructuredCleanDedup