Evaluation Engineering - 17

rack applications, and looking at TCP as a solution for aisle scale
or datacenter scale applications.

TCP
The most recent addition, ratified in December 2018, of the
NVMe-oF transport family is TCP, or NVMe/TCP. TCP has a
few key advantages. First, TCP is a well-known protocol. Network
engineers understand its behavior very well, and know how to
use it. From that perspective, the 'learning curve' of enabling
NVMe/TCP in the datacenter is shorter and smoother.
Next, similar to FC, TCP takes advantage of the queueing
characteristics of NVMe. Each TCP connection is mapped to
an NVMe queue. Thus, again, admin and I/O commands can
be given separate queues so as to not block one another. Multithreaded applications can leverage multiple CPU cores, with
each core having its own queue to access storage. This simplicity
in mapping in the transition from the network to within the
storage array keeps latency relatively low.
Further, TCP can be run on simple switches without the extended capabilities of RoCE capable switch, which generally
speaking, can add cost to a switch, but do offer performance
enhancements.
Relative to RoCE, comparisons show that TCP does take a
latency hit of several microseconds. However, it's important to

Subscribe
Today
EVALUATION
For Electronic

PASSING THE TEST

The electronic test & measurement field is often
overlooked, but its role in the manufacturing
cycle cannot be overstated. Evaluation Engineering,
a monthly magazine, is the voice of electronic
test & measurement as the only trade resource
solely dedicated to this market. EE keeps test &
measurement engineers updated about trends, best
practices, market developments, and much more
as one of a handful of industrial publications under
Endeavor Business Media.

remember that not all workloads will be sensitive to that difference. The applications needing the absolute highest performance,
where cost is a secondary consideration relative to performance,
may likely be better served by RoCE. Again, the IT team needs
to understand their workload, and make an informed assessment of their needs.

Conclusions
NVMe-oF is very compelling for many storage-use cases, but the
exact flavor of NVMe-oF used will depend on existing infrastructure, and whether the absolute lowest latency is really needed.
FC-NVMe allows existing FC users to continue to use FC infrastructure, while getting the benefits of low latency streamlined
access to flash. RoCE allows using ethernet networks, which
are well known, lower latency than TCP, at a cost and complexity premium relative to TCP. In greenfield applications, TCP is
quite attractive, for a lower cost point with a marginal latency
sacrifice relative to RoCE. Managing congestion on the TCP
network can go a long way to ensuring a highperforming fabric.
David Woolf is senior engineer, datacenter technologies at the University of New Hampshire
InterOperability Laboratory (UNH-IOL).

JANUARY 2019

EvaluationEngineering.com

ENGINEERIN

ement
Test & Measur

D DIG ITA L
HIG H-S PEElimits
SPECIAL REPORTS

PAM4, PCIe, jitter
move the needle

VN A

ts,
Frequency requiremen
g VNA trends
lower costs drivin

ALS O
IONS
2019 5G PREDICT
ABLE FPGA
USER-PR OGRAMM N ADVICE
INSTRUM ENTATIO

evaluationengineering.com/subscribe

MAY 2019 EVALUATIONENGINEERING.COM

http://www.EvaluationEngineering.com http://www.evaluationengineering.com/subscribe http://www.rsleads.com/905ee-917 http://www.EVALUATIONENGINEERING.COM

Evaluation Engineering

Table of Contents for the Digital Edition of Evaluation Engineering