www.ngi-central.org Traditional Image Processing Data Structures Images, Palettes, Histograms, Profiles, etc. For many datatypes (8bit, 16 bit, float, etc.) Algorithms Dependent on Data Structures Dependent on datatypes Combinatorial Explosion -> unmanageable To implement one new algorithm, it must be written many times for each type www.ngi-central.org Generic Image Processing Generic Programming comes to the rescue Three key design techniques Iterators decouple data structures and algorithms Functors allow to exchange parts of computation Generic algorithms in terms of abstract iterators and functors Enabling technology of compilers is the template mechanism www.ngi-central.org Algorithms Iterators Data Standard Template Library STL is an implementation of generic programming concepts STL is available with all compilers since it is part of the C++ standard STL is around 10 years old and considered mature Educational material to learn generic programming in general and STL in particular is available www.ngi-central.org4 Example: RGB to Gray Using STL style programming struct RGBValue { unsigned char red, green, blue; }; struct RGBToGray { unsigned char operator()(const RGBValue& rgb) const { return 0.3*rgb.red *rgb.green *rgb.blue; } }; vector rgb; vector gray;... transform(rgb.begin(), rgb.end(), gray.begin(), RGBToGray()); www.ngi-central.org New: works for all types! Better example Templated on the datatype struct RGBValue { T red, green, blue; }; struct RGBToGray { T operator()(const RGBValue & rgb) const { return 0.3*rgb.red *rgb.green *rgb.blue; } }; vector > rgb; vector gray;... transform(rgb.begin(), rgb.end(), gray.begin(), RGBToGray ()); www.ngi-central.org STL leaves things to desire STL is designed to work with 1D data Images are inherently two(multi)dimensional Some algorithms do not need the dimensionality Use STL in this case Some algorithms need the dimensionality Cannot use STL, need something different www.ngi-central.org Multidimensional Locators A locator is a multidimensional iterator Logical extension to an iterator Move the iterator ++it; --it; it+=100; it-=50; Move the locator ++it.x; --it.y; it.x+=100; it.y-=50; www.ngi-central.org Algorithms change as well // STL implementation template template IT transform(IT First, IT Last, IT Dest, F Func) { for (; First != Last; ++First, ++Dest) *Dest = Func(*First); return (Dest); } // 2D implementation template template IT transform(IT First, IT Last, IT Dest, F Func) { for (; First.y != Last.y; ++First.y, ++Dest.y) for (; First.x != Last.x; ++First.x, ++Dest.x) *Dest = Func(*First); return (Dest); } www.ngi-central.org What do we gain? An algorithm like transform replaces many functions in traditional programming style It can do this, because it is templated on the function We still need to program the functionality, but decoupled from navigation It saves us to rewrite the loops many times Bonus: the function objects can be reused in completely different algorithms as well It can do this, because it is templated on the datatype (by iterator indirection) It saves us to rewrite the complete thing for each datatype The gain is tremendous (time, functionality, flexibility) www.ngi-central.org NGI - Goals Image Processing Image -> Image Image Analysis Image -> Numbers Generic Library C++ Templates Source code Independence of type High Performance no penalty for generic code Portability clean source code for easy portability www.ngi-central.org Directory Structure Code (include/) Third party code (toolkits/) Documentation (book/, reference/, presentations/) Sample code (samples/) and sample images (images/) Automated tests (tests/) www.ngi-central.org NGI - Tests Unit Tests Features of a class/function are tested in isolation Regression Tests Outcome of a function is tested for regressions Image/Text comparisons (now == previous)nowprevious Benchmarks Measured in clocks per pixel www.ngi-central.org NGI - Documentation User Documentation NGI Book Built from Docbook sources (HTML, HTML Help, PDF)HTMLHTML HelpPDF Includes results from tests and sample programs Reference Documentation Built with Doxygen from source code (HTML)HTML Always up-to-date www.ngi-central.org Third Party Code Mandatory boost (www.boost.org)www.boost.org Optional FreeImage (freeimage.sourceforge.net) Cairo www.ngi-central.org NGI Build System NANT is used as build engine recursive, type nant build at any directory modular select compiler (VS2005, VS2008, Intel) run tests build documentation Currently setting up a continuous build machine with various virtual build environments www.ngi-central.org Sample Code Many code samples Console samples MFC samples.NET samples Tests are also samples Using NUnit Unit tests, benchmarks, regression tests www.ngi-central.org NGI Guidelines Header only library Source code Heavy use of templates Concentration on core issues using other Open Source libraries where applicable boost FreeImage www.ngi-central.org NGI Concepts Buffer stores data handles allocation and deallocation can use external buffers View level of indirection light, just a pointer and a few pitches all accesses through views can handle foreign data buffers view manipulations sub-views (moving_view.exe)moving_view.exe rotated views (dimension_swap_view.exe)dimension_swap_view.exe reverse scanning directions (reverse_view.exe)reverse_view.exe subsampling (subsampling_view.exe)subsampling_view.exe www.ngi-central.org NGI Functionality Overview (1) Basic functions used everywhere else Interpolation linear, spline Geometry point, line_segment, ray, line, triangle, quadrilateral, rectangle, polygon, circle, ellipse widgets for all geometric elements Other examples Base64, ZIP/GZIP Compression/Decompression, Fixed point, Matrix solver, solving quadratic and cubic equations, etc www.ngi-central.org NGI Functionality Overview (2) STL like functions copy, count, equal, find,... Image Processing functions Point, Filters, Morphology,... Image Analysis functions Blob, Pattern match, OCR/OCV,... Display Engine Images, Palettes, Curves, Widgets, Geometry,... Framegrabber/Camera interface www.ngi-central.org NGI STL Like Functions accumulate copy count equal fill find generate inner_product max_element min_element replace transform www.ngi-central.org 1D 2D 3D NGI Image Processing Functions (1) Point functions (point_operations.exe, profile.exe)point_operations.exe profile.exe Filter and morphological functions 1D, 2D, 3D Separable Fast box filters independent of kernel size Many predefined kernels, user can easily add Framing (framing.exe)framing.exe Morphology Erode, Dilate, Open, Close, Gradient Gray and binary behavior www.ngi-central.org NGI Image Processing Functions (2) Binning (binning.exe)binning.exe binning method selectable/user writable Geometric Transformation (resample.exe)resample.exe affine / polar choice of resampling filter (nearest neighbor, box, triangle, quadratic, cubic, bspline, sinc, kaiser, lanczos) Autofocus based on variance, gradient www.ngi-central.org NGI Image Processing Functions (3) Color Processing Color models (color_transform.exe)color_transform.exe RGB, CMY(K), HLS, HSI, Lab, Lchab, Luv, Lchuv Color twist (color_twist.exe)color_twist.exe Bayer demosaic Statistic Functions min, max, average, variance, percentiles Camera Calibration www.ngi-central.org NGI Image Analysis Functions (1) Blob analysis Thresholding via Otsus method 2D, 3D Labelling Flche/Volumen Chain Code (inner, outer), perimeter Moments binary, gray ordinary, central, Hu, Flusser www.ngi-central.org NGI Image Analysis Functions (2) Pattern matching and searching (search.exe)search.exe gray or color using pyramid for acceleration OCR/OCV train a character set read/verify text www.ngi-central.org NGI - Acceleration Written for Multi-Core Uses OpenMP Most functions scale nicely, can be seen with the benchmarks Working on SIMD acceleration Vector Processing (128 byte chunks) 16 Pixels at a time First for point functions These two methods are orthogonal www.ngi-central.org NGI Display Engine (1) Display Engine GDI Plus, OpenGL, Cairo Alpha channel support (blit.exe)blit.exe Images (file_access.exe)file_access.exe Palettes (palette.exe, false_color.exe)palette.exefalse_color.exe Curves (histogram.exe)histogram.exe www.ngi-central.org NGI Display Engine (2) Widgets (scale.exe)scale.exe Hierarchical widget composition (widget_box.exe)widget_box.exe Callbacks for smart interaction (horizontal_cursor.exe)horizontal_cursor.exe www.ngi-central.org NGI Framegrabber/Camera Grabbing into ringbuffer asynchronous on different thread live/snapshot pre-trigger/post-trigger Gen cam support planned www.ngi-central.org More Information www.ngi-central.org32