bring out the best in pixels · bring out the best in pixels . ... content-adaptive per-pixel...

Bring out the Best in Pixels Video Pipe in Intel® Processor Graphics

Victor H. S. Ha and Yi-Jen Chiu

Graphics Architecture, Intel Corp.

INFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEL PRODUCTS. NO LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE, TO ANY INTELLECTUAL PROPERTY RIGHTS IS GRANTED BY THIS DOCUMENT. EXCEPT AS PROVIDED IN INTEL'S TERMS AND CONDITIONS OF SALE FOR SUCH PRODUCTS, INTEL ASSUMES NO LIABILITY WHATSOEVER AND INTEL DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY, RELATING TO SALE AND/OR USE OF INTEL PRODUCTS INCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR INFRINGEMENT OF ANY PATENT, COPYRIGHT OR OTHER INTELLECTUAL PROPERTY RIGHT.

A "Mission Critical Application" is any application in which failure of the Intel Product could result, directly or indirectly, in personal injury or death. SHOULD YOU PURCHASE OR USE INTEL'S PRODUCTS FOR ANY SUCH MISSION CRITICAL APPLICATION, YOU SHALL INDEMNIFY AND HOLD INTEL AND ITS SUBSIDIARIES, SUBCONTRACTORS AND AFFILIATES, AND THE DIRECTORS, OFFICERS, AND EMPLOYEES OF EACH, HARMLESS AGAINST ALL CLAIMS COSTS, DAMAGES, AND EXPENSES AND REASONABLE ATTORNEYS' FEES ARISING OUT OF, DIRECTLY OR INDIRECTLY, ANY CLAIM OF PRODUCT LIABILITY, PERSONAL INJURY, OR DEATH ARISING IN ANY WAY OUT OF SUCH MISSION CRITICAL APPLICATION, WHETHER OR NOT INTEL OR ITS SUBCONTRACTOR WAS NEGLIGENT IN THE DESIGN, MANUFACTURE, OR WARNING OF THE INTEL PRODUCT OR ANY OF ITS PARTS.

Intel may make changes to specifications and product descriptions at any time, without notice.

All products, dates, and figures specified are preliminary based on current expectations, and are subject to change without notice.

Intel processors, chipsets, and desktop boards may contain design defects or errors known as errata, which may cause the product to deviate from published specifications. Current characterized errata are available on request.

Any code names featured are used internally within Intel to identify products that are in development and not yet publicly announced for release. Customers, licensees and other third parties are not authorized by Intel to use code names in advertising, promotion or marketing of any product or services and any such use of Intel's internal code names is at the sole risk of the user.

Intel product plans in this presentation do not constitute Intel plan of record product roadmaps. Please contact your Intel representative to obtain Intel’s current plan of record product roadmaps.

Ultrabook™ products are offered in multiple models. Some models may not be available in your market. Consult your Ultrabook™ manufacturer. For more information and details, visit http://www.intel.com/ultrabook

Performance claims: Software and workloads used in performance tests may have been optimized for performance only on Intel® microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. For more information go to http://www.Intel.com/performance

Copyright © 2013 Intel Corporation. All rights reserved. Intel, Intel Inside, the Intel logo, Intel Core, and Ultrabook are trademarks of Intel Corporation in the United States and other countries. *Other names and brands may be claimed as the property of others.

Legal

http://www.intel.com/ultrabook

http://www.intel.com/ultrabook

Contents Intel Processor Graphics

Video Processing Pipe Video Processing Modules Control Panel Adaptive Processing

Summary

Q&A

3

• 4th Generation Intel® Core™ Microprocessor, built on 22 nm process technology

• Next Generation Intel® HD Graphics with Microsoft* DirectX*11.1, OpenGL* 4.0, OpenCL* 1.2 support

• Three Simultaneous Display, HDMI, DisplayPort*, with high-resolution up to 4Kx2K

• Significant 3D and Media performance improvement

5

Vertex Processing

Rasterize

Pixel Ops

Multi-Format Codec

Media Processing Display

Command Streamers

Array of Unified Execution Units

Instruction Cache

EU EU EU

EU EU EU

Texture Sampler

Media Sampler

Video Quality Engine

6

Intel Graphics Media Processing

WiDi

Interlaced / Progressive

Chroma Format: 420/422/444

JPG, MPG, H.264, VC1

DVI/HDMI

Display Port

VGA Ethernet/LAN

WiFi

HDD

DVD/BD Network Mobility

Source Format, Content, Quality WebCam

USB

SD, FHD, UHD 4K

BT

7

Multi-format Codec

VQE

Media Sampler

0101010001001011101010101… Denoise

Deinterlace Contrast

Saturation Skin-tone

Gamut Color Correction

Stabilization

Decode MPEG2 / VC1 AVC SVC MVC JPEG / MJPEG

Scaling Sharpness

Bring out the Best in Pixels



Summary

Q&A

8

Video Quality Engine • Video Processing • Color Processing

9

Command Streamers

3D Geometry Processing

Rasterizer

Media FrontEnd

Multi -format CodecVideo Processing Pipeline

VQE

$L3 Cache

EU EU

EU

EU EU

EU EU EU

Array of Execution UnitsMedia

Samplers

TextureSamplers

Pixel Ops

Media Accelerators

Introducing Video Quality

Engine

Multi-Format Codec • Video Decode and Encode • High Performance Parallel Engine

Front-End Thread Dispatch/Management

Media Optimized Execution Units

Video Processing Pipe

Dedicated video processing on newly designed Video Quality Engine (VQE)

Support for an extensive suite of functions for higher quality video at lower power

10

Denoise Deinterlace

Scaler Sharpness Image Stabilization

Skin-tone Gamut Compression Saturation

Contrast ProcAmp

CSC Gamut Expansion



Summary

Q&A

11

De-noise

Spatial and Temporal De-noise Filter Global noise level measurement Content-adaptive spatiotemporal filtering of noise Motion history-based blending of spatial and temporal filter results

Block Interface Input: YCbCr 420/422 Output: YCbCr 420/422

12

Noisy Images Clean Images

De-interlace

Convert interlaced contents to progressive format Edge-adaptive Spatial Interpolation result + Motion-compensated Interpolation result

Motion-adaptive Blending with Temporal Filtering results based on Spatial Temporal Motion Measure (STMM)

Detection of most common cadences

Chroma Upsampling from 420 to 422

13

Skin-tone Processing

Per-pixel Enhancement of Skin-tone Pixels Reproduce the natural skin colors on the display screen Skin Tone Detection identifies pixels with skin-like colors with per-pixel indicator Skin Tone Enhancement modifies the Saturation and Hue of the skin-tone pixels

Block Interface Input: YCbCr444 Output: YCbCr444 with modified CbCr components; Per-pixel skin tone indicator

14

Skin-tone Detection

Skin-tone Definition

Skin-tone Enhancement

Skin-tone Indicator

Original Input Skin-tone Enhanced

Output

Contrast Enhancement

Automatic Contrast Enhancement: Per-pixel mapping of luma to enhance contrast 1. Histogram of luma Y pixel values is generated for the input video frame 2. Piece-Wise Linear Function (PWLF) is generated from luma histogram 3. Pixel values are modified according to the PWLF

Block Interface Input: YCbCr 444 Output: YCbCr 444 with modified Y

15

Histogram

Computation

PWLF

Mapping

Input Image

Enhanced Image

Saturation Enhancement

Per-Pixel Saturation Enhancement Utilize 6 basic colors as primaries/anchors (Red, Green, Blue, Magenta, Yellow, Cyan) Adjust colorfulness (saturation) of pixels while maintaining their color (hue)

Block Interface Input: YCbCr 444 Output: YCbCr 444 with modified CbCr components

16

Input Image Saturation Enhanced

Image

Color Correction

Display proper colors on display screen 1. Inverse gamma correction via PWLF 2. 3x3 matrix multiplication with input/output offset 3. Forward gamma correction via PWLF

Block Interface Input: RGB Output: RGB

17

Input Image

Color Corrected

Image

Image Stabilization

Stabilize shaky video contents captured by handheld camera devices Usage models

Real-time playback: Watch real-time playback of video streams

Offline processing: Stabilize shaky video and save/transmit for video sharing

Block Interface Input: YCbCr 420

Output: YCbCr 420

18

Scaler

Adaptive Video Scaler Advanced scaling method for adaptive scaling mode Content-adaptive per-pixel blending of polyphase filtering and bilinear filtering reduces ringing Programmable coefficient tables available for Y / CbCr channels in horizontal and vertical directions Support for up to 16:1 downscaling and max output picture size of 16Kx16K for

YUV420/422/444/RGB input picture

19

Sharpness Enhancement

Enhancement of Image Sharpness and Details Content adaptive spatial filtering applied to luma channel

User control of sharpness enhancement strength = [0, 63]

Adaptive to Skin Tone Pixels Skin tone information is utilized

Enhance detail information without over-sharpening of the skin regions

Interface Input: YCbCr 444

Output: YCbCr 444

20



Summary

Q&A

21

22

Control Panel • Standard Color Correction

– Brightness – Contrast – Hue – Saturation

23

Control Panel • Input Range

– Limited – Full

• Total Color Correction – Saturation Enhancement – Adjustment = [0, 255]

24

Control Panel • Sharpness

– Driver Custom Settings = [0, 64]

• Skin Tone Enhancement – On = [0, 9] – Off

25

Control Panel • Noise Reduction

– Luma only = [0, 64] – Luma and Chroma = [0, 64]

• Contrast Enhancement – On/Off

• Film Mode Detection – On/Off


Video Processing Pipe Control Panel Video Processing Modules Adaptive Processing

Summary

Q&A

26

Hardware Statistics

Statistics are gathered on Per-block (16x4) basis for encoder stats Per-frame basis for other stats

16 bytes of encoder statistics data for per-block statistics is available Temporal Variances De-noise: Sum of block noise estimates and Number of blocks per frame

A variety of per-frame data is stored in a linear surface Skin-tone: Ymax = Max luma value, Ymin = Min luma value, Counter = Number of skin pixels Contrast: Luma histogram with 256 bins and 24-bits per bin Gamut Compression: Sum of distances of out-of-gamut-range pixels clipped to 32 bits, Number of

out-of-gamut-range pixels in 29 bits

27

Adaptive Processing & Control

The goal of adaptive processing is to deliver optimized video experience

Hardware statistics and adaptive control logic allow end-to-end optimization and effective sharing of data and information across all processing blocks

28

APC Example

29

Skin-tone Detection

Saturation Enhancement Noise Reduction

Motion Detection

Skin-tone Indicator Global Motion Indicator

Noise Level Estimator

Global Noise Level

Sharpness Enhancement

Brightness Information

Contrast Enhancement

Luma Histogram



Summary

Q&A

30

Summary

Intel is building Media Solutions for Quality, Power, and Scalable Performance

Dedicated HW acceleration for codecs, video and image processing, and analytics

Media processing in 4th Generation Processor Graphics delivers: VQE: A new feature with a full-set of video processing pipe Adaptive Processing & Control for integrated and optimized video pipe behavior Ready for various application scenarios and network/mobility environments

Looking Ahead Future generations of Intel media processing will address further HW acceleration in video

processing, camera image processing, and perceptual computing/machine vision applications Improvement in power and performance fitted to exciting new form-factors

31



Summary

Q&A

32

bring out the best in pixels · bring out the best in pixels . ... content-adaptive per-pixel...

Documents