© Copyright 2017 Xilinx

Stack Overview

© Copyright 2017 Xilinx

1 | P a g e

Overview

The Xilinx reVISION stack includes a broad range of development resources for platform, algorithm and application development. This includes support for the

most popular neural networks including AlexNet, GoogLeNet, SqueezeNet, SSD, and FCN. Additionally, the stack provides library elements including pre-defined

and optimized implementations for CNN network layers, required to build custom neural networks (DNN/CNN). The machine learning elements are

complemented by a broad set of acceleration-ready OpenCV functions for computer vision processing. For application level development, Xilinx supports

industry-standard frameworks including Caffe for machine learning and OpenVX for computer vision. The reVISION stack also includes development platforms

from Xilinx and third parties, including various types of sensors.

© Copyright 2017 Xilinx

2 | P a g e

COMPUTER VISION

xFOpenCV Library Functions

OpenCV library functions are essential to developing many computer vision applications. Xilinx’s xFAST library for computer vision, based on key OpenCV

functions, will allow you to easily compose and accelerate computer vision functions in the FPGA fabric through SDx or HLx environments. In addition, xFAST

functions are consistent with OpenCV and are optimized for performance, resource utilization and ease of use. Visit www.xilinx.com/revision for details.

Thousands of functions in OpenCV 3.1 library for Cortex A9 and Cortex A53

OpenCV functions (including the OpenVX subset) available as a library of optimized functions for Xilinx SoCs

Complete library user guide with device utilization and performance

The figure below details a typical design flow with reVISION for Computer Vision.

© Copyright 2017 Xilinx

3 | P a g e

MACHINE LEARNING

Frameworks, Networks and Functions

Xilinx’s SoCs are an ideal fit for machine learning achieving up to 6x efficiency over the state-of-the-art embedded GPUs, Xilinx’s reVISION stack removes the

traditional FPGA design barrier by allowing you to quickly take a trained network and deploy it on Zynq SoCs and MPSoCs for inference.

Features:

Full software stack for deploying machine learning applications

Hardware optimized libraries supporting Conv, ReLU, Pooling, Dilated conv, Deconv, FC, Detector & Classifier, SoftMax layers

Caffe inter-operability allows easy porting from Proto-Text files for Network definition and trained weights

Optimized reference models available for a wide range of network topologies, such as AlexNet, GoogLeNet, SqueezeNet, FCN and SSD

Networks can be customized through software running on ARM processor without compiling for FPGA

The figure below details a typical design flow with reVISION for Machine Learning.

© Copyright 2017 Xilinx

4 | P a g e

reVISION Design Examples for Machine Learning and Computer Vision

The reVISION stack includes four initial design examples (with more to come) that are intended to get you up-and-running in a very short period of time. These design examples will help you easily see the distinct advantage Xilinx All Programmable SOCs have in high performance embedded vision applications. The following is a brief description of these four design examples.

LK Dense Optical Flow @ 4K60 – Real-time dense implementation of optical flow, detecting object motion for every single pixel. This example uses non-iterative, non-pyramidal

implementation on 4K@60 FPS input coming from a Sony IMX274 sensor via the MIPI interface

Stereo Vision – Real-time stereo disparity map calculation including remap, rectification and local block matching. It can process dual 1080p30 stereo camera input

via USB3

Deep Learning: GoogLeNet – GoogleNet benchmark with INT8 demonstrated using standard ImageNet inputs.

Combined dense optical flow, stereo vision and deep learning example This demonstration combines the three major, complex algorithms commonly used in vision-guided systems today including Convolutional Neural Network (CNN) for object detection or scene segmentation, Dense Optical Flow for motion tracking and Stereo Vision for depth perception, running on a single Zynq Ultrascale+ MPSoC device.

Optical Flow

Stereo Vision

CNNCNN

© Copyright 2017 Xilinx

5 | P a g e

SDSoC Environment Overview

The reVISION stack builds on the SDSoC Environment. SDSoC enables algorithm and/or application development in C, C++ and/or OpenCL using the reVISION resources. The SDSoC Environment can also be used to expand the reVISION resources with new acceleration-ready software libraries. Familiar Embedded C/C++/OpenCL Application Development Experience The SDSoC™ development environment provides a familiar embedded C/C++/OpenCL application development experience including an easy to use Eclipse IDE and a comprehensive design environment for heterogeneous Zynq® All Programmable SoC and MPSoC deployment. Complete with the industry's first C/C++/OpenCL full-system optimizing compiler, SDSoC delivers system level profiling, automated software acceleration in programmable logic, automated system connectivity generation, and libraries to speed programming. It also enables end user and third party platform developers to rapidly define, integrate, and verify system level solutions and enable their end customers with a customized programming environment.

© Copyright 2017 Xilinx

6 | P a g e

Boards, Kits and Modules

The reVISION stack is targeted to both the Zynq SoC and Zynq MPSoC families. Xilinx and its Alliance Program members produce several boards that can enable your development with the reVISION stack. The following is a list of Xilinx development boards and sensors supported by the stack. Xilinx reVISION Targeted Boards (& BSPs) & Kits

© Copyright 2017 Xilinx

7 | P a g e

Connectivity and Sensor Support

Xilinx’s Zynq platforms provide acceleration of computer vision and machine learning algorithms for fast response, the reconfigurability required for hardware

platform developers to rapidly upgrade to the best available type and mix of sensors, and the ability to rapidly upgrade connectivity to new machines and/or the

cloud.

Xilinx supports a number of connectivity standards for sensor interfacing as well as video input and output.

Interface Type Provider Description

LVDS Sensor Xilinx Natively supported in Zynq-7000 and Zynq UltraScale+ devices. Refer to device datasheet for more

information.

MIPI-CSI 2.0 Sensor Xilinx

MIPI CSI2 Receiver Subsystem and MIPI CSI 2 Transmitter Subsystems implement the Mobile Industry

Processor Interface (MIPI) based Camera Serial Interface (CSI-2) according to version 1.1 on Xilinx's

UltraScale+™ devices allowing users to capture raw images from MIPI CSI2 sensors.

logiSLVS_RX Camera

Sub-LVDS Receiver

Sensor Xylon IP core supporting interfacing of ultra-high resolution Sony CMOS image sensors to image signal processing

pipelines and application processors implemented in Xilinx All Programmable devices

HDMI In/Out Xilinx HDMI TX and RX subsystems. The HDMI Subsystems are designed in compliance with the HDMI Forum

version 2.0 of the HDMI specification.

DisplayPort In/Out Xilinx DisplayPort LogiCORE™ and DisplayPort TX and RX subsystems help users implement DisplayPort video

interface as defined by VESA DisplayPort v1.2 specification.

UHD-SDI (up to 12G) In/Out Xilinx

UHD Serial Digital Interface (UHD-SDI)is used for the transport of uncompressed digital video streams up to

4K resolutions over coax cable. The LogiCORE™ IP UHD-SDI interface provides receiver and transmitter

interfaces for the SMPTE SD-SDI, HD-SDI, 3G-SDI, 6G-SDI and 12G-SDI standards.

GigE Vision In/Out Sensor to

Image

GigE solution consists of one or more FPGA IP cores to design GigE Vision compliant devices mainly for the

machine vision market.

© Copyright 2017 Xilinx

8 | P a g e

USB3 Vision In/Out Sensor to

Image

U3V solution consists of one or more FPGA IP cores to design U3V compliant camera applications, primarily

for the machine vision market.

Copyright 2017 Xilinx, the Xilinx logo, Artix, ISE, Kintex, Spartan, Virtex, Zynq, Vivado, UltraFast and other designated brands included herein are trademarks of

Xilinx in the United States and other countries. All other trademarks are the property of their respective owners.