Train and Use a TensorFlow* Model on Intel® Architecture

Introduction

TensorFlow* is a leading deep learning and machine learning framework, and as of May 2017, it now integrates optimizations for Intel® Xeon® processors and Intel® Xeon Phi™ processors. This is the third in a series of tutorials providing information for developers who want to build, install, and explore TensorFlow optimized on Intel® architecture from sources available in the GitHub* repository.

The first tutorial in this series Build and Install TensorFlow for Intel Architecture demonstrates how to build and install TensorFlow optimized on Intel architecture from sources in the GitHub* repository.

The second tutorial in the series Build and Install TensorFlow Serving on Intel Architecture describes how build and install TensorFlow Serving, a high-performance serving system for machine learning models designed for production environments.

In this tutorial we will train and save a TensorFlow model, build a TensorFlow model server, and test the server using a client application. This tutorial is based on the MNIST for ML Beginners and Serving a TensorFlow Model tutorials on the TensorFlow website. You are encouraged to review these tutorials before proceeding to fully understand the details of how models are trained and saved.

Train and Save a MNIST Model

According to Wikipedia, the MNIST (Modified National Institute of Standards and Technology) database contains 60,000 training images and 10,000 testing images used for training and testing in the field of machine learning. Because of its relative simplicity, the MNIST database is often used as an introductory dataset for demonstrating machine learning frameworks.

To get started, open a terminal and issue the following commands:

cd ~/serving
bazel build //tensorflow_serving/example:mnist_saved_model
rm -rf /tmp/mnist_model
bazel-bin/tensorflow_serving/example/mnist_saved_model /tmp/mnist_model

Troubleshooting: At the time of this writing, the TensorFlow Serving repository identified an error logged as “NotFoundError in mnist_export example #421.” If you encounter an error after issuing the last command try this workaround:

1. Open serving bazel-bin/tensorflow_serving/example/mnist_saved_model.runfiles/ org_tensorflow/tensorflow/contrib/image/__init__.py
2. Comment-out (#) the following line as shown:
   #from tensorflow.contrib.image.python.ops.single_image_random_dot_stereograms import single_image_random_dot_stereograms
3. Save and close __init__.py.
4. Try issuing the command again:
   bazel-bin/tensorflow_serving/example/mnist_saved_model /tmp/mnist_model

Since we omitted the training_iterations and model_version command-line parameters when we ran mnist_saved_model, they defaulted to 1000 and 1, respectively. Because we passed /tmp/mnist_model for the export directory, the trained model was saved in /tmp/mnist_model/1.

As explained in the TensorFlow tutorial documentation, the “1” version sub-directory contains the following files:

• saved_model.pb is the serialized tensorflow::SavedModel. It includes one or more graph definitions of the model, as well as metadata of the model such as signatures.
• variables are files that hold the serialized variables of the graphs.

Troubleshooting: In some instances you might encounter an issue with the downloaded training files getting corrupted when the script runs. This error is identified as "Not a gzipped file #170" on GitHub. If necessary, these files can be downloaded manually by issuing the following commands from the /tmp directory:

wget http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz
wget http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz
wget http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz
wget http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz

Build and Start the TensorFlow Model Server

Build the TensorFlow model server by issuing the following command:

bazel build //tensorflow_serving/model_servers:tensorflow_model_server

Start the TensorFlow model server by issuing the following command:

bazel-bin/tensorflow_serving/model_servers/tensorflow_model_server --port=9000 --model_name=mnist --model_base_path=/tmp/mnist_model/ &

Test the TensorFlow Model Server

The last command started the ModelServer running in the terminal. To test the server using the mnist_client utility provided in the TensorFlow Serving installation, enter the following commands from the /serving directory:

bazel build //tensorflow_serving/example:mnist_client
bazel-bin/tensorflow_serving/example/mnist_client --num_tests=1000 --server=localhost:9000

If everything worked, you should see results similar to Figure 1.

Figure 1. TensorFlow client test results

Troubleshooting: There is an error identified on GitHub as “gRPC doesn't respect the no_proxy environment variable” that may result in an “Endpoint read failed” error when you run the client application. Issue the env command to see if the http_proxy environment variable is set. If so, it can be temporarily unset by issuing the following command:

unset http_proxy

Summary

In this series of tutorials we explored the process of building the TensorFlow machine learning framework and TensorFlow Serving, a high-performance serving system for machine learning models, optimized for Intel architecture. A simple model based on the MNIST dataset was trained and saved, and it was then deployed using a TensorFlow model server. Lastly, the mnist_client example included in the GitHub repository was used to demonstrate how a client-side application can leverage a TensorFlow model server to do simple machine learning inference.

For additional information on this subject please visit the TensorFlow website, a key resource for learning more about the framework. The article entitled “TensorFlow Optimizations on Modern Intel Architecture” introduces the specific graph optimizations, performance experiments, and details for building and installing TensorFlow with CPU optimizations.