<jupyter_start><jupyter_text>Copyright 2020 The TensorFlow Authors.<jupyter_code>#@title Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.<jupyter_output><empty_output><jupyter_text>The Sequential model       View on TensorFlow.org        Run in Google Colab        View source on GitHub        Download notebook    Setup<jupyter_code>import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers<jupyter_output><empty_output><jupyter_text>When to use a Sequential modelA `Sequential` model is appropriate for **a plain stack of layers**where each layer has **exactly one input tensor and one output tensor**.Schematically, the following `Sequential` model:<jupyter_code># Define Sequential model with 3 layers
model = keras.Sequential(
    [
        layers.Dense(2, activation="relu", name="layer1"),
        layers.Dense(3, activation="relu", name="layer2"),
        layers.Dense(4, name="layer3"),
    ]
)
# Call model on a test input
x = tf.ones((3, 3))
y = model(x)<jupyter_output><empty_output><jupyter_text>is equivalent to this function:<jupyter_code># Create 3 layers
layer1 = layers.Dense(2, activation="relu", name="layer1")
layer2 = layers.Dense(3, activation="relu", name="layer2")
layer3 = layers.Dense(4, name="layer3")

# Call layers on a test input
x = tf.ones((3, 3))
y = layer3(layer2(layer1(x)))<jupyter_output><empty_output><jupyter_text>A Sequential model is **not appropriate** when:- Your model has multiple inputs or multiple outputs- Any of your layers has multiple inputs or multiple outputs- You need to do layer sharing- You want non-linear topology (e.g. a residual connection, a multi-branchmodel)  Creating a Sequential modelYou can create a Sequential model by passing a list of layers to the Sequentialconstructor:<jupyter_code>model = keras.Sequential(
    [
        layers.Dense(2, activation="relu"),
        layers.Dense(3, activation="relu"),
        layers.Dense(4),
    ]
)<jupyter_output><empty_output><jupyter_text>Its layers are accessible via the `layers` attribute:<jupyter_code>model.layers<jupyter_output><empty_output><jupyter_text>You can also create a Sequential model incrementally via the `add()` method:<jupyter_code>model = keras.Sequential()
model.add(layers.Dense(2, activation="relu"))
model.add(layers.Dense(3, activation="relu"))
model.add(layers.Dense(4))<jupyter_output><empty_output><jupyter_text>Note that there's also a corresponding `pop()` method to remove layers:a Sequential model behaves very much like a list of layers.<jupyter_code>model.pop()
print(len(model.layers))  # 2<jupyter_output><empty_output><jupyter_text>Also note that the Sequential constructor accepts a `name` argument, just likeany layer or model in Keras. This is useful to annotate TensorBoard graphswith semantically meaningful names.<jupyter_code>model = keras.Sequential(name="my_sequential")
model.add(layers.Dense(2, activation="relu", name="layer1"))
model.add(layers.Dense(3, activation="relu", name="layer2"))
model.add(layers.Dense(4, name="layer3"))<jupyter_output><empty_output><jupyter_text>Specifying the input shape in advanceGenerally, all layers in Keras need to know the shape of their inputsin order to be able to create their weights. So when you create a layer likethis, initially, it has no weights:<jupyter_code>layer = layers.Dense(3)
layer.weights  # Empty<jupyter_output><empty_output><jupyter_text>It creates its weights the first time it is called on an input, since the shapeof the weights depends on the shape of the inputs:<jupyter_code># Call layer on a test input
x = tf.ones((1, 4))
y = layer(x)
layer.weights  # Now it has weights, of shape (4, 3) and (3,)<jupyter_output><empty_output><jupyter_text>Naturally, this also applies to Sequential models. When you instantiate aSequential model without an input shape, it isn't "built": it has no weights(and calling`model.weights` results in an error stating just this). The weights are createdwhen the model first sees some input data:<jupyter_code>model = keras.Sequential(
    [
        layers.Dense(2, activation="relu"),
        layers.Dense(3, activation="relu"),
        layers.Dense(4),
    ]
)  # No weights at this stage!

# At this point, you can't do this:
# model.weights

# You also can't do this:
# model.summary()

# Call the model on a test input
x = tf.ones((1, 4))
y = model(x)
print("Number of weights after calling the model:", len(model.weights))  # 6<jupyter_output><empty_output><jupyter_text>Once a model is "built", you can call its `summary()` method to display itscontents:<jupyter_code>model.summary()<jupyter_output><empty_output><jupyter_text>However, it can be very useful when building a Sequential model incrementallyto be able to display the summary of the model so far, including the currentoutput shape. In this case, you should start your model by passing an `Input`object to your model, so that it knows its input shape from the start:<jupyter_code>model = keras.Sequential()
model.add(keras.Input(shape=(4,)))
model.add(layers.Dense(2, activation="relu"))

model.summary()<jupyter_output><empty_output><jupyter_text>Note that the `Input` object is not displayed as part of `model.layers`, sinceit isn't a layer:<jupyter_code>model.layers<jupyter_output><empty_output><jupyter_text>A simple alternative is to just pass an `input_shape` argument to your firstlayer:<jupyter_code>model = keras.Sequential()
model.add(layers.Dense(2, activation="relu", input_shape=(4,)))

model.summary()<jupyter_output><empty_output><jupyter_text>Models built with a predefined input shape like this always have weights (evenbefore seeing any data) and always have a defined output shape.In general, it's a recommended best practice to always specify the input shapeof a Sequential model in advance if you know what it is.  A common debugging workflow: `add()` + `summary()`When building a new Sequential architecture, it's useful to incrementally stacklayers with `add()` and frequently print model summaries. For instance, thisenables you to monitor how a stack of `Conv2D` and `MaxPooling2D` layers isdownsampling image feature maps:<jupyter_code>model = keras.Sequential()
model.add(keras.Input(shape=(250, 250, 3)))  # 250x250 RGB images
model.add(layers.Conv2D(32, 5, strides=2, activation="relu"))
model.add(layers.Conv2D(32, 3, activation="relu"))
model.add(layers.MaxPooling2D(3))

# Can you guess what the current output shape is at this point? Probably not.
# Let's just print it:
model.summary()

# The answer was: (40, 40, 32), so we can keep downsampling...

model.add(layers.Conv2D(32, 3, activation="relu"))
model.add(layers.Conv2D(32, 3, activation="relu"))
model.add(layers.MaxPooling2D(3))
model.add(layers.Conv2D(32, 3, activation="relu"))
model.add(layers.Conv2D(32, 3, activation="relu"))
model.add(layers.MaxPooling2D(2))

# And now?
model.summary()

# Now that we have 4x4 feature maps, time to apply global max pooling.
model.add(layers.GlobalMaxPooling2D())

# Finally, we add a classification layer.
model.add(layers.Dense(10))<jupyter_output><empty_output><jupyter_text>Very practical, right?  What to do once you have a modelOnce your model architecture is ready, you will want to:- Train your model, evaluate it, and run inference. See our[guide to training & evaluation with the built-in loops](https://www.tensorflow.org/guide/keras/train_and_evaluate/)- Save your model to disk and restore it. See our[guide to serialization & saving](https://www.tensorflow.org/guide/keras/save_and_serialize/).- Speed up model training by leveraging multiple GPUs. See our[guide to multi-GPU and distributed training](/guides/distributed_training).  Feature extraction with a Sequential modelOnce a Sequential model has been built, it behaves like a [Functional APImodel](https://www.tensorflow.org/guide/keras/functional/). This means that every layer has an `input`and `output` attribute. These attributes can be used to do neat things, likequicklycreating a model that extracts the outputs of all intermediate layers in aSequential model:<jupyter_code>initial_model = keras.Sequential(
    [
        keras.Input(shape=(250, 250, 3)),
        layers.Conv2D(32, 5, strides=2, activation="relu"),
        layers.Conv2D(32, 3, activation="relu"),
        layers.Conv2D(32, 3, activation="relu"),
    ]
)
feature_extractor = keras.Model(
    inputs=initial_model.inputs,
    outputs=[layer.output for layer in initial_model.layers],
)

# Call feature extractor on test input.
x = tf.ones((1, 250, 250, 3))
features = feature_extractor(x)<jupyter_output><empty_output><jupyter_text>Here's a similar example that only extract features from one layer:<jupyter_code>initial_model = keras.Sequential(
    [
        keras.Input(shape=(250, 250, 3)),
        layers.Conv2D(32, 5, strides=2, activation="relu"),
        layers.Conv2D(32, 3, activation="relu", name="my_intermediate_layer"),
        layers.Conv2D(32, 3, activation="relu"),
    ]
)
feature_extractor = keras.Model(
    inputs=initial_model.inputs,
    outputs=initial_model.get_layer(name="my_intermediate_layer").output,
)
# Call feature extractor on test input.
x = tf.ones((1, 250, 250, 3))
features = feature_extractor(x)<jupyter_output><empty_output>