Advanced

TensorFlow Best Practices

Optimize performance with tf.data pipelines and mixed precision, debug common issues, deploy models to production with SavedModel and TF Serving, and follow proven patterns for robust ML systems.

Efficient Data Pipelines with tf.data

The tf.data API is essential for building high-performance input pipelines that keep your GPU fed with data:

Python 
import tensorflow as tf

# Build an optimized data pipeline
dataset = tf.data.Dataset.from_tensor_slices((x_train, y_train))
dataset = (dataset
    .shuffle(buffer_size=10000)
    .batch(32)
    .prefetch(tf.data.AUTOTUNE)  # Overlap data loading with training
)

# For image data from directories
train_ds = tf.keras.utils.image_dataset_from_directory(
    'data/train',
    image_size=(224, 224),
    batch_size=32
)

# Apply augmentation and caching
train_ds = (train_ds
    .cache()                        # Cache in memory after first epoch
    .shuffle(1000)
    .map(augment, num_parallel_calls=tf.data.AUTOTUNE)
    .prefetch(tf.data.AUTOTUNE)
)

Mixed Precision Training

Mixed precision uses float16 for most computations while keeping float32 for numerical stability, providing up to 3x speedup on modern GPUs:

Python 
from tensorflow import keras

# Enable mixed precision globally
keras.mixed_precision.set_global_policy('mixed_float16')

# Build model as usual - layers automatically use float16
model = keras.Sequential([
    keras.layers.Dense(256, activation='relu'),
    keras.layers.Dense(10, activation='softmax', dtype='float32')  # Output in float32
])

Debugging Tips

  1. Use tf.debugging assertions

    Add tf.debugging.assert_all_finite(tensor, "NaN detected") to catch numerical issues early in your pipeline.

  2. Run eagerly for debugging

    Set model.compile(run_eagerly=True) to disable graph compilation and enable standard Python debugging with breakpoints.

  3. Check input shapes

    Shape mismatches are the most common error. Use model.summary() and print tensor shapes at each stage.

  4. Monitor with TensorBoard

    Use the TensorBoard callback to visualize training curves, detect overfitting, and inspect model graphs.

Model Deployment

Python 
# Save as SavedModel (recommended for production)
model.save('saved_model/my_model')

# Convert to TF Lite for mobile/edge
converter = tf.lite.TFLiteConverter.from_saved_model('saved_model/my_model')
converter.optimizations = [tf.lite.Optimize.DEFAULT]
tflite_model = converter.convert()

with open('model.tflite', 'wb') as f:
    f.write(tflite_model)

# Serve with TF Serving (Docker)
# docker run -p 8501:8501 \
#   --mount type=bind,source=/path/to/saved_model,target=/models/my_model \
#   -e MODEL_NAME=my_model \
#   tensorflow/serving

Production Checklist

Practice Why It Matters
Use tf.data pipelines Prevent GPU starvation with efficient data loading
Enable mixed precision Up to 3x faster training on modern GPUs
Use @tf.function Graph compilation for faster execution in production
Version your models SavedModel format supports versioned serving
Profile with TensorBoard Identify bottlenecks in your training pipeline
Set random seeds Reproducible results with tf.random.set_seed(42)
Keep Learning: TensorFlow's ecosystem is vast. Explore official tutorials, join the TensorFlow community on GitHub, and check out PyTorch for an alternative perspective on deep learning.

Course Complete!

Congratulations on finishing the TensorFlow & Keras course. You now have the foundations to build, train, and deploy deep learning models. Continue your journey with PyTorch or explore specialized topics.

Next Course: PyTorch Deep Dive →
← NLP with TF Course Overview →