TensorFlow is an open source software library for numerical computation using data flow graphs. Nodes in the graph represent mathematical operations, while the graph edges represent the multi-dimensional data arrays (tensors) communicated between them. 

The flexible architecture allows you to deploy computation to one or more CPUs or GPUs in a desktop, server, or mobile device with a single API. TensorFlow was originally developed by researchers and engineers working on the Google Brain Team within Google's Machine Intelligence research organization for the purposes of conducting machine learning and deep neural networks research, but the system is general enough to be applicable in a wide variety of other domains as well, such as language translation, medical research, autonomous vehicles, image recognition, security and more. 

RDMA technology has extended the performance boundary of distributed systems. TensorFlow takes advantage of high performance, open-source universal RPC framework (gRPC) to transfer data, and leverages RDMA-based gRPC which is about 322% better than TCP-based gRPC. 

Another way to leverage RDMA with TensorFlow is to utilize MPI (Message Passing Interface). An MPI version of TensorFlow is available with the MaTEx: (Machine Learning Toolkit for Extreme Scale), a collection of high performance machine learning and data mining (MLDM) algorithms.

The Mellanox® Solution

Mellanox I/O adapters eliminate processing bottlenecks by delivering up to 200Gb/s of reliable, lossless connectivity with less than one microsecond of latency. One adapter can provide optimum connectivity for an entire blade server system, saving data centers the cost of maintaining multiple Ethernet adapters. Optimal for distributed machine learning frameworks, only Mellanox provides native acceleration engines such as RDMA, GPUDirect® RDMA, and SHARP™ v2 to dramatically improve performance across cognitive computing applications. Mellanox InfiniBand switches and gateways enable data center fabric consolidation that further improves performance.