Categories
linux machine learning Solutions

Cross-compile latest Tensorflow (1.5+) for the Nvidia Jetson TK1

Instructions on how to cross-compile Tensorflow 1.5+ on a Ubuntu x86_64 host for the ARM-based Jetson TK1 from Nvidia.

Been looking around for a solid resource on how to get Tensorflow to run on the Jetson TK1. Most what I found was how to get TF 0.8 to run, which was the last TF version to allow usage of cuDNN 6 that is the latest version available for the TK1.
The TK1 is an aging platform with halted support, but it is still a cheap option for high-powered embedded compute. Unfortunately, being so outdated it’s impossible to get the latest and greatest of DNN to work on the CUDA GPU on the TK1, but we can certainly use the CPU!
So a word of disclaimer – this compiled TF version will not use the GPU, just the CPU. However, it will let you run the most recent NN architectures with the latest layer implementations.
Cross compilation for the TK1 solves the acute problem of space on the device itself, as well as speed of compilation. On the other hand it required bringing up a compilation toolchain, which took a while to find.
I am going to be assuming a Ubuntu 16.04 x86_64 machine, which is what I have, and really you can do this in a VM or a Docker container just as well on Windows.

I’ve learned a great deal from the following resources:

But I had my own recipe of course, otherwise I wouldn’t have posted this.

Preparation

Starting from the host, the following toolchain needs to be installed: http://releases.linaro.org/components/toolchain/binaries/4.9-2017.01/arm-linux-gnueabihf/gcc-linaro-4.9.4-2017.01-x86_64_arm-linux-gnueabihf.tar.xz
It’s just GCC 4.9, which is based on libc 2.19 that powers the TK1 vanilla install.
Initially I toyed with the gcc-5-arm-linux-gnueabihf from the apt sources on ubuntu – but don’t bother with it. It uses libc 2.21, which is too recent for the TK1.
Once you have that untared in a location you know,
Get tensorflow from github, and checkout v1.5 (although this will probably work with later versions):

$ git clone https://github.com/tensorflow/tensorflow
$ cd tensorflow/
$ git checkout r1.5

Next up, install Bazel.
Follow the instructions from the TF docs: https://www.tensorflow.org/install/install_sources, which direct you here: https://docs.bazel.build/versions/master/install.html
Install relevant python libs:

$ sudo apt-get install python-numpy python-dev python-pip python-wheel

Now I followed the instructions here https://lengerrong.blogspot.com/2017/09/cross-compile-tensorflow-for-armv7l.html – to modify and add the Bazel scripts to cross compile.
But it didn’t work right off the bat, I had to make some modifications.
In essence, that post is correct.
You’d need to add the following to end of your $TF_ROOT/WORKSPACE:

new_local_repository(
    name = 'toolchain_target_arm_linux_gnueabi',
    path = '$GCC_ROOT',
    build_file = 'arm-compiler/cross_toolchain_target_arm_linux_gnueabi_host_x86-64.BUILD'
)

(replace $GCC_ROOT with the actual path to the linaro gcc toolchain)
Open a new directory:

$ mkdir $TF_ROOT/arm-compiler

Then you’d need the $TF_ROOT/arm-compiler/cross....BUILD file, the $TF_ROOT/arm-compiler/BUILD file and the $TF_ROOT/arm-compiler/CROSSTOOL file.
They’re pretty standard, and come from Bazel’s docs, but still modified for this purpose.
Remember to replace $GCC_ROOT with the place you unpacked gcc-4.9-arm cross compile toolchain.

Build TF

At this point you’re ready to configure Tensorflow:

$ echo -e "\n\nn\nn\nn\nn\n\n\n\n\n\n\n-march=armv7-a -std=c++11\n\n" | ./configure

This will set most everything to No, and ask to add some compiler flags for architecture and stdc++11.
And now for the main event – building TF:

$ bazel build --crosstool_top=//arm-compiler:toolchain --cpu=armeabi-v7a --config=opt -s tensorflow/examples/label_image/...

We’re not only going to build TF but also the “label_image” sample.
This will take ~45 minutes to build.

Run the example on the TK1

At the end, if no errors arose, you should be able to see a bazel-bin directory with the outputs.
Copy the directory to the TK1 and let it rip!
(to run label-image you’d need the Inception precompiled net and the sample image, follow these instructions from the TF team: https://github.com/tensorflow/tensorflow/tree/master/tensorflow/examples/label_image)

ubuntu@tegra-ubuntu:~/Downloads/label_image$ LD_LIBRARY_PATH=label_image.runfiles/org_tensorflow/tensorflow/:/lib/arm-linux-gnueabihf/ ./label_image2018-03-08 19:56:47.825285: I tensorflow/examples/label_image/main.cc:250] military uniform (653): 0.834305
2018-03-08 19:56:47.825620: I tensorflow/examples/label_image/main.cc:250] mortarboard (668): 0.0218693
2018-03-08 19:56:47.825826: I tensorflow/examples/label_image/main.cc:250] academic gown (401): 0.010358
2018-03-08 19:56:47.825996: I tensorflow/examples/label_image/main.cc:250] pickelhaube (716): 0.00800815
2018-03-08 19:56:47.826166: I tensorflow/examples/label_image/main.cc:250] bulletproof vest (466): 0.00535088

(see the LD_LIBRARY_PATH needs to be set properly for the exec to find TF)
Best of luck!
Roy

Leave a Reply

Your email address will not be published. Required fields are marked *