![Page 1: Community works for muli core embedded image processing](https://reader034.vdocuments.net/reader034/viewer/2022051412/54be7f5f4a7959e8318b4660/html5/thumbnails/1.jpg)
multi-core embedded image processing
Nov. 2, 2013 KELP
Jeong-pyo Kong / ntekLogic [email protected]
Community works for
![Page 2: Community works for muli core embedded image processing](https://reader034.vdocuments.net/reader034/viewer/2022051412/54be7f5f4a7959e8318b4660/html5/thumbnails/2.jpg)
Agenda
1. Why we do this?
2. Technical backgrounds
3. Coworking with ETRI, KESSIA
4. Current our works
5. Future of our works
OpenSEED imageA 2
![Page 3: Community works for muli core embedded image processing](https://reader034.vdocuments.net/reader034/viewer/2022051412/54be7f5f4a7959e8318b4660/html5/thumbnails/3.jpg)
1. WHY WE DO THIS?
OpenSEED imageA 3
![Page 4: Community works for muli core embedded image processing](https://reader034.vdocuments.net/reader034/viewer/2022051412/54be7f5f4a7959e8318b4660/html5/thumbnails/4.jpg)
IT trends are changing quite quickly
OpenSEED imageA 4
![Page 5: Community works for muli core embedded image processing](https://reader034.vdocuments.net/reader034/viewer/2022051412/54be7f5f4a7959e8318b4660/html5/thumbnails/5.jpg)
OpenSEED imageA 5
http://www.wired.com/magazine/2013/04/convergence/
40 Gadgets
8 Imaging
6 Gaming
α
Convergence
![Page 6: Community works for muli core embedded image processing](https://reader034.vdocuments.net/reader034/viewer/2022051412/54be7f5f4a7959e8318b4660/html5/thumbnails/6.jpg)
Connected world
OpenSEED imageA 6
http://press.teleinteractive.net/media/blogs/tialife/InternetofThingsVector.svg
IoT/M2M Big data
Wearable
![Page 7: Community works for muli core embedded image processing](https://reader034.vdocuments.net/reader034/viewer/2022051412/54be7f5f4a7959e8318b4660/html5/thumbnails/7.jpg)
Computational world
OpenSEED imageA 7
UAV Robot
Autonomous Vehicle
![Page 8: Community works for muli core embedded image processing](https://reader034.vdocuments.net/reader034/viewer/2022051412/54be7f5f4a7959e8318b4660/html5/thumbnails/8.jpg)
OpenSEED imageA 8
http://www.slideshare.net/robman/divergence-what-happens-after-convergence
Divergence
![Page 9: Community works for muli core embedded image processing](https://reader034.vdocuments.net/reader034/viewer/2022051412/54be7f5f4a7959e8318b4660/html5/thumbnails/9.jpg)
Appropriate Technology for IT business environment
OpenSEED imageA 9
![Page 10: Community works for muli core embedded image processing](https://reader034.vdocuments.net/reader034/viewer/2022051412/54be7f5f4a7959e8318b4660/html5/thumbnails/10.jpg)
Problem solving
• Complex problem -> Complex solution
• Complex problem -> Simple solution
• Simple problem -> Complex solution
• Simple problem -> Simple solution
OpenSEED imageA 10
![Page 11: Community works for muli core embedded image processing](https://reader034.vdocuments.net/reader034/viewer/2022051412/54be7f5f4a7959e8318b4660/html5/thumbnails/11.jpg)
OpenSEED imageA 11
http://www.youtube.com/watch?v=nrWfLAh2T3k
![Page 12: Community works for muli core embedded image processing](https://reader034.vdocuments.net/reader034/viewer/2022051412/54be7f5f4a7959e8318b4660/html5/thumbnails/12.jpg)
What we thought
• Interesting image processing and connected world
• Let’s assume if we can do below things – Using improved hardware
– Improving (for example) • Response time to 30%
• Power saving to 15%
• Accurate rate to 10%
OpenSEED imageA 12
Isn’t it a big deal?
![Page 13: Community works for muli core embedded image processing](https://reader034.vdocuments.net/reader034/viewer/2022051412/54be7f5f4a7959e8318b4660/html5/thumbnails/13.jpg)
Is it a chance for us?
OpenSEED imageA 13
• Many changes are
hidden from
ordinary people,
usually
• Next generation infra
• Eco system
![Page 14: Community works for muli core embedded image processing](https://reader034.vdocuments.net/reader034/viewer/2022051412/54be7f5f4a7959e8318b4660/html5/thumbnails/14.jpg)
So, what is the result of our works?
OpenSEED imageA 14
![Page 15: Community works for muli core embedded image processing](https://reader034.vdocuments.net/reader034/viewer/2022051412/54be7f5f4a7959e8318b4660/html5/thumbnails/15.jpg)
2. TECHNICAL BACKGROUNDS
OpenSEED imageA 15
1. why we do this?
![Page 16: Community works for muli core embedded image processing](https://reader034.vdocuments.net/reader034/viewer/2022051412/54be7f5f4a7959e8318b4660/html5/thumbnails/16.jpg)
Requirements for embedded image processing
OpenSEED imageA 16
![Page 17: Community works for muli core embedded image processing](https://reader034.vdocuments.net/reader034/viewer/2022051412/54be7f5f4a7959e8318b4660/html5/thumbnails/17.jpg)
System components
• Input – Image sensor – Video files on local storage – Network streaming
• Processing – ISP – Low/high level image processing application
• Output – Display, Signal – Network streaming
OpenSEED imageA 17
![Page 18: Community works for muli core embedded image processing](https://reader034.vdocuments.net/reader034/viewer/2022051412/54be7f5f4a7959e8318b4660/html5/thumbnails/18.jpg)
Requirements
• Low power
• High performance
• Easy to release : Productivity
• Easy to use for user : UX/UI
• Compatibility
OpenSEED imageA 18
![Page 19: Community works for muli core embedded image processing](https://reader034.vdocuments.net/reader034/viewer/2022051412/54be7f5f4a7959e8318b4660/html5/thumbnails/19.jpg)
How can we meet these requirements
OpenSEED imageA 19
![Page 20: Community works for muli core embedded image processing](https://reader034.vdocuments.net/reader034/viewer/2022051412/54be7f5f4a7959e8318b4660/html5/thumbnails/20.jpg)
Hardware based approach(1)
• Silicon gate length : e.g. 20nm
• Architecture : e.g. RISC
• Instruction processing : e.g. ARM NEON
OpenSEED imageA 20
![Page 21: Community works for muli core embedded image processing](https://reader034.vdocuments.net/reader034/viewer/2022051412/54be7f5f4a7959e8318b4660/html5/thumbnails/21.jpg)
Hardware based approach(2)
• CPU trend : Operating Frequency
OpenSEED imageA 21
![Page 22: Community works for muli core embedded image processing](https://reader034.vdocuments.net/reader034/viewer/2022051412/54be7f5f4a7959e8318b4660/html5/thumbnails/22.jpg)
Hardware based approach(3)
• Multi-core
OpenSEED imageA 22
Processor 1
Processor 3 Processor 4
Processor 2
General Purpose
Processor 1
General Purpose
Processor 2 DSP
GPU
Homogeneous
Architecture
Heterogeneous
Architecture
![Page 23: Community works for muli core embedded image processing](https://reader034.vdocuments.net/reader034/viewer/2022051412/54be7f5f4a7959e8318b4660/html5/thumbnails/23.jpg)
Software based approach
• Efficient algorithm
• Multi-core framework
– OpenMP
– OpenCL(WebCL)
– CUDA
• Efficient application design with multi-
core framework
OpenSEED imageA 23
![Page 24: Community works for muli core embedded image processing](https://reader034.vdocuments.net/reader034/viewer/2022051412/54be7f5f4a7959e8318b4660/html5/thumbnails/24.jpg)
Heterogeneous multi-core system application
OpenSEED imageA 24
![Page 25: Community works for muli core embedded image processing](https://reader034.vdocuments.net/reader034/viewer/2022051412/54be7f5f4a7959e8318b4660/html5/thumbnails/25.jpg)
For desktop
• Games
• Web acceleration
• UHD video : 4K, 8K
• Computer graphics, 3D, authoring tools
• Financial
OpenSEED imageA 25
![Page 26: Community works for muli core embedded image processing](https://reader034.vdocuments.net/reader034/viewer/2022051412/54be7f5f4a7959e8318b4660/html5/thumbnails/26.jpg)
For embedded system
• General purpose – Games
– Web acceleration
– UHD video : 4K, 8K
• Specialized purpose – Computer vision
– Robot
– Autonomous vehicles, UAV
– Medical imaging
OpenSEED imageA 26
![Page 27: Community works for muli core embedded image processing](https://reader034.vdocuments.net/reader034/viewer/2022051412/54be7f5f4a7959e8318b4660/html5/thumbnails/27.jpg)
Embedded multi-core image processing
application framework
OpenSEED imageA 27
![Page 28: Community works for muli core embedded image processing](https://reader034.vdocuments.net/reader034/viewer/2022051412/54be7f5f4a7959e8318b4660/html5/thumbnails/28.jpg)
•
OpenSEED imageA 28
HSA (Heterogeneous System Architecture)
Current Future
Desktop, smart device application
http://www.tomshardware.com/reviews/fusion-hsa-opencl-history,3262-8.html
![Page 29: Community works for muli core embedded image processing](https://reader034.vdocuments.net/reader034/viewer/2022051412/54be7f5f4a7959e8318b4660/html5/thumbnails/29.jpg)
OpenCL(1)
• Low level hardware abstraction
• Supporting two parallelism methods
– Data parallelization
– Task parallelization
• User is in charge of optimization
– Engineer needs to know big picture
– Regression testing…
OpenSEED imageA 29
![Page 30: Community works for muli core embedded image processing](https://reader034.vdocuments.net/reader034/viewer/2022051412/54be7f5f4a7959e8318b4660/html5/thumbnails/30.jpg)
• Programming Model
OpenCL(2)
OpenSEED imageA 30
![Page 31: Community works for muli core embedded image processing](https://reader034.vdocuments.net/reader034/viewer/2022051412/54be7f5f4a7959e8318b4660/html5/thumbnails/31.jpg)
• Memory
model
OpenCL(3)
OpenSEED imageA 31
![Page 32: Community works for muli core embedded image processing](https://reader034.vdocuments.net/reader034/viewer/2022051412/54be7f5f4a7959e8318b4660/html5/thumbnails/32.jpg)
OpenCV
• Image processing library
• OpenCV is being worked on supporting discrete GPU(nVidia, AMD) with x86 only, currently
• OpenCV doesn’t support ARM OpenCL (Planed to release in near future)
OpenSEED imageA 32
![Page 33: Community works for muli core embedded image processing](https://reader034.vdocuments.net/reader034/viewer/2022051412/54be7f5f4a7959e8318b4660/html5/thumbnails/33.jpg)
3. COWORKING WITH ETRI, KESSIA
OpenSEED imageA 33
2. technical backgrounds
![Page 34: Community works for muli core embedded image processing](https://reader034.vdocuments.net/reader034/viewer/2022051412/54be7f5f4a7959e8318b4660/html5/thumbnails/34.jpg)
ETRI providing
• Platform hardware
– Target board
• Platform software
– Kernels
– Middleware
– Development tools
• Community base
– OpenSEED
OpenSEED imageA 34
![Page 35: Community works for muli core embedded image processing](https://reader034.vdocuments.net/reader034/viewer/2022051412/54be7f5f4a7959e8318b4660/html5/thumbnails/35.jpg)
ETRI roadmap
OpenSEED imageA 35
ETRI
![Page 36: Community works for muli core embedded image processing](https://reader034.vdocuments.net/reader034/viewer/2022051412/54be7f5f4a7959e8318b4660/html5/thumbnails/36.jpg)
KESSIA providing
• Making interconnection between ETRI
and community
• Community operation
– OpenSEED
OpenSEED imageA 36
![Page 37: Community works for muli core embedded image processing](https://reader034.vdocuments.net/reader034/viewer/2022051412/54be7f5f4a7959e8318b4660/html5/thumbnails/37.jpg)
4. CURRENT OUR WORKS
OpenSEED imageA 37
3. coworking with ETRI, KESSIA
![Page 38: Community works for muli core embedded image processing](https://reader034.vdocuments.net/reader034/viewer/2022051412/54be7f5f4a7959e8318b4660/html5/thumbnails/38.jpg)
imageA = 이미자?
OpenSEED imageA 38
![Page 39: Community works for muli core embedded image processing](https://reader034.vdocuments.net/reader034/viewer/2022051412/54be7f5f4a7959e8318b4660/html5/thumbnails/39.jpg)
imageA = image processing Acceleration
OpenSEED imageA 39
![Page 40: Community works for muli core embedded image processing](https://reader034.vdocuments.net/reader034/viewer/2022051412/54be7f5f4a7959e8318b4660/html5/thumbnails/40.jpg)
First Trial(1)
• From ETRI – Hardware : Exynos 4412 based development
board. Mali-400 GPU (http://www.hardkernel.com/renewal_2011/products/prdt_info.php?g_code=G133888637376)
– Kernel : high performance
resource management
kernel
• Long journey for the works OpenSEED imageA 40
![Page 41: Community works for muli core embedded image processing](https://reader034.vdocuments.net/reader034/viewer/2022051412/54be7f5f4a7959e8318b4660/html5/thumbnails/41.jpg)
First Trial(2)
• Our activities
– Referring user manuals in OpenSEED
– Kernel porting
– OpenCV library porting
– OpenCV test application development
OpenSEED imageA 41
![Page 42: Community works for muli core embedded image processing](https://reader034.vdocuments.net/reader034/viewer/2022051412/54be7f5f4a7959e8318b4660/html5/thumbnails/42.jpg)
First Trial(3)
• How about now?
OpenSEED imageA 42
![Page 43: Community works for muli core embedded image processing](https://reader034.vdocuments.net/reader034/viewer/2022051412/54be7f5f4a7959e8318b4660/html5/thumbnails/43.jpg)
Second Trial
• From ETRI
– Hardware : x86 based AMD SBC
– Kernel
• Our activities (~2013.11)
– OpenCV library
– OpenCV test application development
– Performance test
OpenSEED imageA 43
![Page 44: Community works for muli core embedded image processing](https://reader034.vdocuments.net/reader034/viewer/2022051412/54be7f5f4a7959e8318b4660/html5/thumbnails/44.jpg)
5. FUTURE OF OUR WORKS
OpenSEED imageA 44
4. current our woks
![Page 45: Community works for muli core embedded image processing](https://reader034.vdocuments.net/reader034/viewer/2022051412/54be7f5f4a7959e8318b4660/html5/thumbnails/45.jpg)
First Trial
• From ETRI – Hardware : x86 based AMD SBC
– Kernel
– Development tools
• Our activities – OpenCV test application development
– OpenCL test application
– Performance test
OpenSEED imageA 45
![Page 46: Community works for muli core embedded image processing](https://reader034.vdocuments.net/reader034/viewer/2022051412/54be7f5f4a7959e8318b4660/html5/thumbnails/46.jpg)
Second Trial
• From ETRI – Hardware : ARM based development board
– Kernel
– Development tools
• Our activities – OpenCL test application
– OpenCV test application development
– Performance test
OpenSEED imageA 46
![Page 47: Community works for muli core embedded image processing](https://reader034.vdocuments.net/reader034/viewer/2022051412/54be7f5f4a7959e8318b4660/html5/thumbnails/47.jpg)
Further Trial
• From ETRI – Hardware : ARM based development board
– Kernel
– Development tools
• Our activities – Existing activities
– + Specific application
– + For smart devices
– + Connected application : streaming and so on
OpenSEED imageA 47
![Page 48: Community works for muli core embedded image processing](https://reader034.vdocuments.net/reader034/viewer/2022051412/54be7f5f4a7959e8318b4660/html5/thumbnails/48.jpg)
Agenda, again
1. Why we do this?
2. Technical backgrounds
3. Co-working with ETRI, KESSIA
4. Current our works
5. Future of our works
OpenSEED imageA 48
![Page 49: Community works for muli core embedded image processing](https://reader034.vdocuments.net/reader034/viewer/2022051412/54be7f5f4a7959e8318b4660/html5/thumbnails/49.jpg)
Thank you!