Cross Compile TensorFlow C++ app for the Jetson TK1

Last time I've posted about cross compiling TF for the TK1. That however was a canned sample example from TF, based on the bazel build system.
Let's say we want to make our own TF C++ app and just link vs. TF for inference on the TK1.
Now that's a huge mess.

First we need to cross-compile TF with everything built in.
Then we need protobuf cross-compiled for the TK1.
Bundle everything together, cross(-compile) our fingers and pray.

The prayer doesn't help. But let's see what does...

I've learned a lot from:

Build TF

To make TF run meaningfully, we need to make two libraries: and

Then bazel build it with the CROSSTOOL we created last time:

$ bazel build --crosstool_top=//arm-compiler:toolchain --cpu=armeabi-v7a --config=opt -s //
$ bazel build --crosstool_top=//arm-compiler:toolchain --cpu=armeabi-v7a --config=opt -s //

If you haven't bazel cleaned since last time this step should only take a minute or so.

Build Protobuf

First we need to compile PB for the host. Trust me on this.
TF team has done an excellent job to facilitate this:

$ sudo apt-get install autoconf automake libtool curl make g++ unzip zlib1g-dev git python
$ cd ${TF_ROOT}
$ bash tensorflow/contrib/makefile/
$ bash tensorflow/contrib/makefile/

Then we move to compiling PB for arm-linux

$ cd ${TF_ROOT}/tensorflow/contrib/makefile/downloads/protobuf/
$ ./autogen
$ ./configure --with-protoc=$(pwd)/../../gen/protobuf/bin/protoc --host=arm-linux --target=arm-linux --disable-shared --enable-cross-compile --prefix=$(pwd)/build --exec-prefix=$(pwd)/build --with-sysroot=${GCC_LINARO_DIR}/gcc-linaro-4.9.4-2017.01-x86_64_arm-linux-gnueabihf/arm-linux-gnueabihf/libc/ CXX=${GCC_LINARO_DIR}/gcc-linaro-4.9.4-2017.01-x86_64_arm-linux-gnueabihf/bin/arm-linux-gnueabihf-g++
$ make
$ make install

See what I did there? we needed a working protoc, and we can't use the cross-compiled one since it doesn't run on the host, it's made for arm...

Gather everything for app cross-compilation

I've created a script that takes everything from the TF directory and places it where you need it:

Example app

We'll set up the example app with CMake:

cmake_minimum_required(VERSION 2.8)

set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11")
set(SOURCE_FILES main.cpp)
set(EXECUTABLE tf_cpp_example)

add_executable(${EXECUTABLE} ${SOURCE_FILES})
target_link_libraries(${EXECUTABLE} tensorflow_cc tensorflow_framework)

With a very basic TF C++ code from here:

#include "tensorflow/cc/client/client_session.h"
#include "tensorflow/cc/ops/standard_ops.h"
#include "tensorflow/core/framework/tensor.h"

int main() {
using namespace tensorflow;
using namespace tensorflow::ops;
Scope root = Scope::NewRootScope();
// Matrix A = [3 2; -1 0]
auto A = Const(root, { {3.f, 2.f}, {-1.f, 0.f} });
// Vector b = [3 5]
auto b = Const(root, { {3.f, 5.f} });
// v = Ab^T
auto v = MatMul(root.WithOpName("v"), A, b, MatMul::TransposeB(true));
std::vector outputs;
ClientSession session(root);
// Run and fetch v
TF_CHECK_OK(session.Run({v}, &outputs));
// Expect outputs[0] == [19; -3]
LOG(INFO) << outputs[0].matrix();
return 0;

We also need a CMake cross-compilation toolchain file, which I've called arm-linux-gnueabihf.cmake:


set(CMAKE_SYSROOT ${GCC_ARM_ROOT}/gcc-linaro-4.9.4-2017.01-x86_64_arm-linux-gnueabihf/arm-linux-gnueabihf/libc/)

set(CMAKE_C_COMPILER ${GCC_ARM_ROOT}/gcc-linaro-4.9.4-2017.01-x86_64_arm-linux-gnueabihf/bin/arm-linux-gnueabihf-gcc)
set(CMAKE_CXX_COMPILER ${GCC_ARM_ROOT}/gcc-linaro-4.9.4-2017.01-x86_64_arm-linux-gnueabihf/bin/arm-linux-gnueabihf-g++)


Then compile with cmake as per usual, specifying the cross compilation toolchain file

$ mkdir build
$ cd build
$ cmake -DCMAKE_TOOLCHAIN_FILE=../arm-linux-gnueabihf.cmake ..
$ make VERBOSE=1 # just for kicks

Once it builds, push the executable, libraries, etc. to the TK1

$ scp ${TF_ROOT}/bazel-bin/tensorflow/lib*.so ubuntu@***.***.***.***:/home/ubuntu/Documents/tfexample

You can also execute it remotely on the TK1, to save a terminal window:

$ ssh ubuntu@***.***.***.*** 'cd /home/ubuntu/Documents/tfexample && LD_LIBRARY_PATH=/home/ubuntu/Documents/tfexample ./tf_cpp_example'
2018-03-16 18:09:32.078588: I /home/****/Documents/tf_cpp_example/main.cpp:20] 19

And that's about it - you can now build standalone Tf 1.5+ apps in C++ for the Jetson TK1!