Machine Learning and Mammography

Abstract

This article, Machine Learning and Mammography, shows how existing deep learning technologies can be utilized to train artificial intelligence (AI) to be able to detect invasive ductal carcinoma (IDC)¹ (breast cancer) in unlabeled histology images. More specifically, I show how to train a convolutional neural network² using TensorFlow*³ and transfer learning⁴ using a dataset of negative and positive histology images. In addition to showing how artificial intelligence can be used to detect IDC, I also show how the Internet of Things (IoT) can be used in conjunction with AI to create automated systems that can be used in the medical industry.

Breast cancer is an ongoing concern and one of the most common forms of cancer in women. In 2018 there is expected to be an estimated 266,120 new diagnoses in the United States alone. The use of Artificial Intelligence can drastically reduce the need for medical staff to examine mammography slides manually, saving not only time, but money, and ultimately lives. In this articles I show how we can use Intel technologies to create a deep learning neural network that is able to detect IDC.

Introducing the IDC Classifier

To create the IDC classifier, I use the Intel® AI DevCloud⁵ to train the neural network, an Intel® Movidius™ product⁶ for carrying out inference on the edge, and an UP Squared*⁷ device to serve the trained model making it accessible via an API, and an IoT connected alarm system built using a Raspberry Pi*⁸ device that demonstrates the potential of using the IoT via the IoT JumpWay*⁹ combined with AI to create intelligent, automated medical systems.

The project evolved from a computer vision project that I have been developing for a number of years named TASS¹⁰. TASS is an open source facial recognition project that has been implemented using a number of different techniques, frameworks, and software developer kits (SDKs).

Invasive Ductal Carcinoma

IDC is one of the most common forms of breast cancer. The cancer starts in the milk duct of the breast and invades the surrounding tissue. This form of cancer makes up around 80 percent of all breast cancer diagnosis, with more than 180,000 women a year in the United States alone being diagnosed with IDC, according to the American Cancer Society.

Convolutional Neural Networks

Inception v3 architecture diagram

Figure 1. Inception v3 architecture (Source).

Convolutional neural networks are a type of deep learning¹¹ neural network. These types of neural nets are widely used in computer vision and have pushed the capabilities of computer vision over the last few years, performing exceptionally better than older, more traditional neural networks; however, studies show¹² that there are trade-offs related to training times and accuracy.

Transfer Learning

Inception v3 model diagram

Figure 2. Inception V3 Transfer Learning (Source)

Transfer learning allows you to retrain the final layer of an existing model, resulting in a significant decrease in not only training time, but also the size of the dataset required. One of the most famous models that can be used for transfer learning is the Inception V3 model created by Google*.¹³ This model was trained on thousands of images from 1,001 classes on some very powerful devices. Being able to retrain the final layer means that you can maintain the knowledge that the model had learned during its original training and apply it to your smaller dataset, resulting in highly accurate classifications without the need for extensive training and computational power. In one version of TASS, I retrained the Inception V3 model using transfer learning on a Raspberry Pi 3 device, so that should give you some idea of the capabilities of transfer learning.

Intel® AI DevCloud

The Intel AI DevCloud is a platform for training machine learning and deep learning models. The platform is made up of a cluster of servers using Intel® Xeon® Scalable processors. The platform is free and provides a number of frameworks and tools including TensorFlow, Caffe*, Keras*, and Theano*, as well as the Intel® Distribution for Python*. The Intel AI DevCloud is great for people getting started with learning how to train machine learning and deep learning models, as graphics processing units (GPUs) can be quite expensive, and access to the DevCloud is free.

In this project I use the Intel AI DevCloud to sort the data, train the model, and evaluate it. To accompany this article I created a full tutorial and provided all of the code you need to replicate the entire project; read the full tutorial and access the source code.

Intel® Movidius™ Neural Compute Stick

The Intel® Movidius™ Neural Compute Stick is a fairly new piece of hardware used for enhancing the inference process of computer vision models on low-powered edge devices. The Intel Movidius product is a USB appliance that can be plugged into devices such as Raspberry Pi and UP Squared, and basically takes the processing power off the device and onto the Intel Movidius brand chip, making the classification process a lot faster. Developers can train their models using their existing TensorFlow and Caffe scripts and, by installing the Intel Movidius Neural Compute Stick SDK on their development machine, can compile a graph that is compatible with the Intel Movidius product. A less-bulky API can be installed on the lower-powered device allowing inference to be carried out via the Intel Movidius product.

Ready to Code

Hopefully, by now you are eager to get started with the technical walkthrough of creating your own computer vision program for classifying negative and positive breast cancer cells, so let’s get to the nitty gritty. Here I walk you through the steps for training and compiling the graph for the Intel Movidius product. For the full walkthrough, including the IoT connected device, please follow the GitHub* repository. Before following the rest of this tutorial, please follow the steps in the repository regarding setting up your IoT JumpWay device, as this step is required before the classification test happens.

Installing the Intel Movidius Neural Compute Stick SDK on Your Development Device

The first thing you need to do is to install the Intel Movidius Neural Compute Stick SDK on your development device. This is used to convert the trained model into a format that is compatible with the Intel Movidius product.

 $ mkdir -p ~/workspace
 $ cd ~/workspace
 $ git clone https://github.com/movidius/ncsdk.git
 $ cd ~/workspace/ncsdk
 $ make install

Next, plug your Intel Movidius product into your device and issue the following commands:

$ cd ~/workspace/ncsdk
$ make examples

Installing the Intel Movidius Neural Compute Stick SDK on Your Inference Device

Next, you need to install the Intel Movidius Neural Compute Stick SDK on your Raspberry Pi 3/UP Squared device. This is used by the classifier to carry out inference on local images or images received via the API we will create. Make sure you have the Intel Movidius product plugged in.

 $ mkdir -p ~/workspace
 $ cd ~/workspace
 $ git clone https://github.com/movidius/ncsdk.git
 $ cd ~/workspace/ncsdk/api/src
 $ make
 $ sudo make install
 $ cd ~/workspace
 $ git clone https://github.com/movidius/ncappzoo
 $ cd ncappzoo/apps/hello_ncs_py
 $ python3 hello_ncs.py

Preparing Your Training Data

For this tutorial, I used a dataset from Kaggle* (Predict IDC in Breast Cancer Histology Images), but you are free to use any dataset you like. I have uploaded the collection I used for positive and negative images that you will find in the model/train directory. Once you decide on your dataset you need to arrange your data into the model/train directory. Each subdirectory should be named with integers; I used 0 and 1 to represent positive and negative. In my testing I used 4400 positive and 4400 negative examples, giving an overall training accuracy of 0.8596 (See Training Results below) and an average confidence of 0.96 on correct identifications. The data provided is 50px x 50px; as Inception V3 was trained on images of size 299px x 299px, the images are resized to 299px x 299px. Ideally the images would be that size already so you may want to try different datasets and see how your results vary.

Fine-Tuning Your Parameters

You can fine-tune the settings of the network at any time by editing the classifier settings in the model/confs.json file.

"ClassifierSettings":{
    "dataset_dir":"model/train/",
    "log_dir":"model/_logs",
    "log_eval":"model/_logs_eval",
    "classes":"model/classes.txt",
    "labels":"labels.txt",
    "labels_file":"model/train/labels.txt",
    "validation_size":0.3,
    "num_shards":2,
    "random_seed":50,
    "tfrecord_filename":"200label",
    "file_pattern":"200label_%s_*.tfrecord",
    "image_size":299,
    "num_classes":2,
    "num_epochs":60,
    "dev_cloud_epochs":60,
    "test_num_epochs":1,
    "batch_size":10,
    "test_batch_size":36,
    "initial_learning_rate":0.0001,
    "learning_rate_decay_factor":0.96,
    "num_epochs_before_decay":10,
    "NetworkPath":"",
    "InceptionImagePath":"model/test/",
    "InceptionThreshold": 0.54,
    "InceptionGraph":"igraph"
}

Time to Start Training

Now you are ready to upload the files and folders outlined below to the Intel AI DevCloud.

model
tools
DevCloudTrainer.ipynb
DevCloudTrainer.py
Eval.py

Once uploaded, follow the instructions in DevCloudTrainer.ipynb, this notebook will help you sort your data, train your model and evaluate it.

Training Results

Training Accuracy Tensorboard graph

Figure 3. Training Accuracy Tensorboard

Training Total Loss graph

Figure 4. Training Total Loss

Evaluate Your Model

Once you have completed your training on the Intel AI DevCloud, complete the notebook by running the evaluation job.

Evaluation Results

INFO:tensorflow:Global Step 1: Streaming Accuracy: 0.0000 (2.03 sec/step)
INFO:tensorflow:Global Step 2: Streaming Accuracy: 0.8889 (0.59 sec/step)
INFO:tensorflow:Global Step 3: Streaming Accuracy: 0.8750 (0.67 sec/step)
INFO:tensorflow:Global Step 4: Streaming Accuracy: 0.8981 (0.65 sec/step)
INFO:tensorflow:Global Step 5: Streaming Accuracy: 0.8681 (0.76 sec/step)
INFO:tensorflow:Global Step 6: Streaming Accuracy: 0.8722 (0.64 sec/step)
INFO:tensorflow:Global Step 7: Streaming Accuracy: 0.8843 (0.64 sec/step)

-------------------------------------------------------------------------

INFO:tensorflow:Global Step 68: Streaming Accuracy: 0.8922 (0.81 sec/step)
INFO:tensorflow:Global Step 69: Streaming Accuracy: 0.8926 (0.70 sec/step)
INFO:tensorflow:Global Step 70: Streaming Accuracy: 0.8921 (0.63 sec/step)
INFO:tensorflow:Global Step 71: Streaming Accuracy: 0.8929 (0.84 sec/step)
INFO:tensorflow:Global Step 72: Streaming Accuracy: 0.8932 (0.75 sec/step)
INFO:tensorflow:Global Step 73: Streaming Accuracy: 0.8935 (0.61 sec/step)
INFO:tensorflow:Global Step 74: Streaming Accuracy: 0.8942 (0.67 sec/step)
INFO:tensorflow:Final Streaming Accuracy: 0.8941

So here we can see that the evaluation shows a final streaming accuracy of 0.8941.

evaluation accuracy graph

Figure 5. Evaluation Accuracy

evaluation total loss graph

Figure 6. Evaluation Total Loss

Download Your Model

When the training completes you need to download model/DevCloudIDC.pb and model/classes.txt to the model directory on your development machine. Ensure that the Intel Movidius product is set up and connected, and then run the following commands on your development machine:

$ cd ~/IoT-JumpWay-Intel-Examples/master/Intel-Movidius/IDC-Classification
$ ./DevCloudTrainer.sh

The contents of DevCloudTrainer.sh are as follows:

#IDC Classification Trainer
mvNCCompile model/DevCloudIDC.pb -in=input -on=InceptionV3/Predictions/Softmax -o igraph
python3.5 Classifier.py InceptionTest

Compile the model for the Intel Movidius product
Test

Testing on Unknown Images

Once the shell script has finished the testing program will start. In my example I had two classes, 0 and 1 (IDC negative and IDC positive); a classification of 0 shows that the AI thinks the image is not IDC positive, and a classification of 1 is positive.

-- Loaded Test Image model/test/negative.png

-- DETECTION STARTING
-- STARTED: :  2018-04-24 14:14:26.780554

-- DETECTION ENDING
-- ENDED:  2018-04-24 14:14:28.691870
-- TIME: 1.9114031791687012

*******************************************************************************
inception-v3 on NCS
*******************************************************************************
0 0 0.9873
1 1 0.01238
*******************************************************************************

-- Loaded Test Image model/test/positive.png

-- DETECTION STARTING
-- STARTED: :  2018-04-24 14:14:28.699254

-- DETECTION ENDING
-- ENDED:  2018-04-24 14:14:30.577683
-- TIME: 1.878432035446167ß

TASS Identified IDC with a confidence of 0.945

-- Published to Device Sensors Channel

*******************************************************************************
inception-v3 on NCS
*******************************************************************************
1 1 0.945
0 0 0.05542
*******************************************************************************

-- INCEPTION V3 TEST MODE ENDING
-- ENDED:  2018-04-24 14:14:30.579247
-- TESTED:  2
-- IDENTIFIED:  1
-- TIME(secs): 3.984593152999878

So, on the development machine you should see results similar to the ones above. We can see in my results that the program has successfully classified both the negative and the positive. Now it is time to test this out on the edge.

Inference on the Edge

Now that it is all trained and tested, it is time to set up the server that will serve the API. For this I have provided Server.py and Client.py.

The following instructions will help you set up your server and test a positive and negative prediction:

If you used the Predict IDC in Breast Cancer Histology Images dataset, you can use the positive.png and negative.png as they are from that dataset; if not, you should choose a positive and negative example from your testing set and replace these images.
The server is currently set to start up on localhost. If you would like to change this you need to edit line 281 of Server.py and line 38 of Client.py to match your desired host. Once you have things working, if you are going to be leaving this running and access it from the outside world, you should secure it with Let's Encrypt* or similar.
Upload the following files and folders to the UP Squared or Raspberry Pi 3 device that you are going to use for the server.
```
model/test/
model/classes.txt
model/confs.json
tools
igraph
Server.py
```
Open up a terminal and navigate to the folder containing Server.py, then issue the following command. This starts the server and waits to receive images for classification.
```
$ python3.5 Server.py
```
If you have followed all of the above steps, you can now start the client on your development machine with the following commands:

$ python3.5 Client.py

This sends a positive and negative histology slide to the Raspberry Pi 3 or UP Squared device, which will return the predictions.

!! Welcome to IDC Classification Client, please wait while the program initiates !!

-- Running on Python 3.5.2 (default, Nov 23 2017, 16:37:01)
[GCC 5.4.0 20160609]

-- Imported Required Modules
-- IDC Classification Client Initiated

{'Response': 'OK', 'ResponseMessage': 'IDC Detected!', 'Results': 1}
{'Response': 'OK', 'ResponseMessage': 'IDC Not Detected!', 'Results': 0}
* Running on http://0.0.0.0:7455/ (Press CTRL+C to quit)

-- IDC CLASSIFIER LIVE INFERENCE STARTING
-- STARTED: :  2018-04-24 14:25:36.465183

-- Loading Sample
-- Loaded Sample
-- DETECTION STARTING
-- STARTED: :  2018-04-24 14:25:36.476371

-- DETECTION ENDING
-- ENDED:  2018-04-24 14:25:38.386121
-- TIME: 1.9097554683685303

TASS Identified IDC with a confidence of 0.945

-- Published: 2
-- Published to Device Warnings Channel

-- Published: 3
-- Published to Device Sensors Channel

*******************************************************************************
inception-v3 on NCS
*******************************************************************************
1 1 0.945
0 0 0.05542
*******************************************************************************

-- IDC CLASSIFIER LIVE INFERENCE ENDING
-- ENDED:  2018-04-24 14:25:38.389217
-- TESTED:  1
-- IDENTIFIED:  1
-- TIME(secs): 1.9240257740020752

192.168.1.40 - - [24/Apr/2018 14:25:38] "POST /api/infer HTTP/1.1" 200 -

-- IDC CLASSIFIER LIVE INFERENCE STARTING
-- STARTED: :  2018-04-24 14:25:43.422319

-- Loading Sample
-- Loaded Sample
-- DETECTION STARTING
-- STARTED: :  2018-04-24 14:25:43.432647

-- DETECTION ENDING
-- ENDED:  2018-04-24 14:25:45.310354
-- TIME: 1.877711534500122

-- Published: 4
-- Published to Device Warnings Channel

-- Published: 5
-- Published to Device Sensors Channel

*******************************************************************************
inception-v3 on NCS
*******************************************************************************
0 0 0.9873
1 1 0.01238
*******************************************************************************

-- IDC CLASSIFIER LIVE INFERENCE ENDING
-- ENDED:  2018-04-24 14:25:45.313174
-- TESTED:  1
-- IDENTIFIED:  0
-- TIME(secs): 1.89084792137146

192.168.1.40 - - [24/Apr/2018 14:25:45] "POST /api/infer HTTP/1.1" 200 -

Here we can see that, using the Intel Movidius product on an UP Squared device, there is no difference in classification accuracy to the development machine; which in my case was a Linux* device with NVIDIA* GTX 750ti, and only a slight difference in the time it took the classification process to complete. It is interesting to note here that the results above were actually more accurate than training the model on my GPU.

IoT Connectivity

To set up the IoT device you are welcome to complete the tutorial on the GitHub repo, but I will go through in some detail here on exactly what this part of the project does, and explain how the proof of concept provided could be used in other medical applications.

The device we create is an IoT connected alarm system built on a Raspberry Pi device. Once set up, the results that are captured from the classification of images sent to the server trigger actions on the IoT that communicate with the Raspberry Pi device. In this case, the actions are turning on a red LED and a buzzer when cancer is detected, and turning on a blue LED when the classification results in no cancer being detected. Obviously this is a very simple proof of concept, but it shows a possibility for powerful applications that can save time for medical staff and, hopefully, in the right hands could help save lives through early and accurate detection.