<jupyter_start><jupyter_text>Test the full connected layer model  Import necessary modules<jupyter_code># python built-in libraries
import datetime
# choose the tensorflow log level
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'  # or any {'0', '1', '2'}
# import tensorflow
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers
# ohter packages
import numpy as np

import tfhelper.core as tfhelper<jupyter_output><empty_output><jupyter_text>Global Variables<jupyter_code># sampling
FEATURES_FILE = "./data/tensors/features.npy"  # path to the file of the feature tensor
LABELS_FILE = "./data/tensors/labels.npy"  # path to the file of the feature tensor
FRACTIONS = (0.8, 0.1, 0.1)  # train, validation, test
BATCH_SIZE = 128  # size of the batch
BUFFER_SIZE = BATCH_SIZE * 2  # size of the shuffle buffer
# training
LEARNING_RATE = 0.001  # starting learning rate
BETA1 = 0.9  # decay 1
BETA2 = 0.999  # decay 2
EPOCHS = 100  # number of epochs
# saving
TIME_STAMP = datetime.datetime.now().strftime("%Y%m%d-%H%M%S")  # time stamp
SAVE_LOC = "./data/models/cnn_2_{}".format(TIME_STAMP)  # path to the folder to save the model
LOG_LOC = "./data/logs/fit/cnn_2_{}".format(TIME_STAMP)  # path to the log, if you change this, you also need to change it in the run_tensorboard<jupyter_output><empty_output><jupyter_text>Obtain dataThe data is a 209 data point PDF. The label is a one-hot 2 dim vector. `10` means major phase >= threshold, `01` means major phase <= threshold.<jupyter_code>def load_data():
    # load
    features, labels = np.load(FEATURES_FILE), np.load(LABELS_FILE)
    # shuffle
    n = features.shape[0]
    shuffled_idx = np.random.permutation(n)
    features, labels = features[shuffled_idx], labels[shuffled_idx]
    # split
    f0, f1, f2 = FRACTIONS
    i, j, k = round(f0 * n), round((f0 + f1) * n), round((f0 + f1 + f2) * n)
    train_data = tf.data.Dataset.from_tensor_slices((features[:i], labels[:i])).shuffle(BUFFER_SIZE).batch(BATCH_SIZE)
    valid_data = tf.data.Dataset.from_tensor_slices((features[i:j], labels[i:j])).batch(BATCH_SIZE)
    test_data = tf.data.Dataset.from_tensor_slices((features[j:k], labels[j:k])).batch(BATCH_SIZE)
    return train_data, valid_data, test_data
train_data, valid_data, test_data = load_data()<jupyter_output><empty_output><jupyter_text>Create the modelWe use the logistric regression. It is a single layer with a softmax function.<jupyter_code>def create_model():
    model = keras.Sequential()
    model.add(keras.Input(shape=(209,)))
    model.add(keras.layers.Reshape((209, 1)))
    model.add(keras.layers.Conv1D(32, 3, strides=1, activation='relu', padding="same"))
    model.add(keras.layers.Conv1D(32, 3, strides=1, activation='relu', padding="same"))
    model.add(keras.layers.BatchNormalization())
    model.add(keras.layers.MaxPooling1D(2))
    model.add(keras.layers.Conv1D(64, 3, strides=1, activation='relu', padding="same"))
    model.add(keras.layers.Conv1D(64, 3, strides=1, activation='relu', padding="same"))
    model.add(keras.layers.BatchNormalization())
    model.add(keras.layers.MaxPooling1D(2))
    model.add(keras.layers.Dropout(0.2))
    model.add(keras.layers.Flatten())
    model.add(keras.layers.Dense(512, activation='relu'))
    model.add(keras.layers.Dense(128, activation='relu'))
    model.add(keras.layers.Dense(32, activation='relu'))
    model.add(keras.layers.Dense(2, activation='softmax'))
    return model
model = create_model()
model.summary()<jupyter_output>Model: "sequential_3"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
reshape_3 (Reshape)          (None, 209, 1)            0         
_________________________________________________________________
conv1d_12 (Conv1D)           (None, 209, 32)           128       
_________________________________________________________________
conv1d_13 (Conv1D)           (None, 209, 32)           3104      
_________________________________________________________________
batch_normalization_6 (Batch (None, 209, 32)           128       
_________________________________________________________________
max_pooling1d_6 (MaxPooling1 (None, 104, 32)           0         
_________________________________________________________________
conv1d_14 (Conv1D)           (None, 104, 64)           6208      
______________________________________________________[...]<jupyter_text>Choose optimization method<jupyter_code>model.compile(
    # Optimizer
    optimizer=keras.optimizers.Adam(LEARNING_RATE, BETA1, BETA2),
    # Loss function to minimize
    loss=keras.losses.CategoricalCrossentropy(),
    # List of metrics to monitor
    metrics=[
        keras.metrics.CosineSimilarity()
    ],
)<jupyter_output><empty_output><jupyter_text>Train the model<jupyter_code># tensor board
tensorboard_callback = keras.callbacks.TensorBoard(
    log_dir=LOG_LOC, 
    histogram_freq=1
)
# early stopping to avoid over fitting
earlystopping_callback = keras.callbacks.EarlyStopping(
    monitor='val_loss',
    patience=30, 
    restore_best_weights=True
)
history = model.fit(
    x=train_data,
    epochs=EPOCHS,
    callbacks=[
        tensorboard_callback,
        earlystopping_callback
    ],
    validation_data=valid_data
)<jupyter_output>Epoch 1/100
688/688 [==============================] - 13s 17ms/step - loss: 0.6642 - cosine_similarity: 0.7346 - val_loss: 0.6255 - val_cosine_similarity: 0.7509
Epoch 2/100
688/688 [==============================] - 11s 16ms/step - loss: 0.6134 - cosine_similarity: 0.7571 - val_loss: 0.6317 - val_cosine_similarity: 0.7483
Epoch 3/100
688/688 [==============================] - 11s 16ms/step - loss: 0.5746 - cosine_similarity: 0.7762 - val_loss: 0.6120 - val_cosine_similarity: 0.7588
Epoch 4/100
688/688 [==============================] - 11s 16ms/step - loss: 0.5311 - cosine_similarity: 0.7959 - val_loss: 0.5473 - val_cosine_similarity: 0.7902
Epoch 5/100
688/688 [==============================] - 11s 16ms/step - loss: 0.4929 - cosine_similarity: 0.8118 - val_loss: 0.5194 - val_cosine_similarity: 0.8032
Epoch 6/100
688/688 [==============================] - 11s 16ms/step - loss: 0.4644 - cosine_similarity: 0.8231 - val_loss: 0.5178 - val_cosine_similarity: 0.8056
Epoch 7/100
688/688 [=[...]<jupyter_text>Evaluate the model<jupyter_code>result = model.evaluate(
    x=test_data,
    return_dict=True
)
for name, val in result.items():
    print("{:20s} {:.4f}".format(name, val))<jupyter_output>loss                 0.4237
cosine_similarity    0.8858<jupyter_text>Save the model<jupyter_code>model.save(SAVE_LOC)<jupyter_output>INFO:tensorflow:Assets written to: ./data/models/cnn_2_20210914-172931/assets
StarcoderdataJupyterStructuredCleanDedup