j2k workshop for the wellcome library

JPEG 2000 WorkshopShortened version for publication

Wellcome Library

8th February 2010

Presented by Richard Clark, Elysium [email protected] - +44 (0) 1892 667411

+44 1892 667411 - [email protected]

Background and JPEG 2000 in practice

Plan to cover– Background to the standards and their practical implementation

– Development and usage

– Elysium's role and experience

– Major players in the market

– Distribution to client software

– Examples of usageand if we don't know, suspect we know a man who does.....


BackgroundJPEG 2000 standards

As basis, see presentation for British Library and DPC -September 2007 - no real substantive changes

Later work has concentrated on JPEG-XR (MS inspired from HD Photo / Vista), and comparisons

Also work on visual searching


BackgroundElysium and JPEG 2000

Part of committee since inception, and UK Head of Delegation for 15+ years

Involved in multiple EC funded research studies and demonstrators (most recent Migrator 2000 and 2KAN both JPEG 2000 specific)

Wrote PD6777 - SI implementors guide - referenced in Library of Congress and other sites

Key assistance in JPIP protocol, file format and other work

Editors for reference software and JPIP standards

Operate JPEG web site, historical archive and MPEG and ICC web sites


JPEG 2000 in practice:Major players in JPEG 2000

Software vendors (* = Mac)

– Aware

– Luratech

– Accusoft / Pegasus

– Leadtools

– Kakadu (*)

– Adobe (*)Fnord, Luratech etc plugins

– Megachips

– Analog

– Apple Quicktime (Kakadu..*)+ others....

Implementors– ITTvis

– Dolby

– Thales

– Picture Elements

– Aerospace

– HP

– Xerox

– Ricoh

– SAIC

– DICOM


JPEG 2000 in practice:Major players in JPEG 2000

Open Source– Kakadu? (*)

– OpenJPEG (*)several derivatives, 3D, wireless etc (used in GIMP)

– Jasper

– JJ2000

– GTK

– ffMPEG

Researchers– EPFL

– Vrij Uni Brussels

– UNSW

– Uni Stuttgart

– Fraunhofer

– Uni Poitiers

– Uni Perugia

– Uni Cagliari

– Uni Arizona

– Uni Catalunya


JPEG 2000 in practice:Viewer programs

Free

Irfanview

KDShow

Expressview

IASViewer

iPhoto

GIMP

Safari / Konqueror

XNView

ErMapper

Paid for

Acdsee

PaintShop Pro

Photopaint

etc - see Wikipedia...

See EXIFTOOL... for metadata!


JPEG 2000 in practice:Browser and imaging server support

IE - no internal support, and unlikely to be. Can however use Java based solutions(IAS Viewer, JuGeMu), plugins (Morgan) or external apps (KDShow)

Firefox - was rumour of direct support via Google summer of code. However see bugzilla #36351 - still havent discovered OpenJPEG and worried re. patents

Chrome - no, plus Google not active in JPEG. However rumoured to use internally in some imaging apps, and host OpenJPEG in code repository

Safari - supported via Image I/O not Quicktime (from Apple!)

Konqueror (can offer direct support)


JPEG 2000 in practice:Browser and imaging server support

Implementations of JPIP– Aware

– Kakadu - http://www.kakadusoftware.com/Demos.html

– CADI (O/S)

– Leadtools

– Pegasus

– JuGeMu -http://jpimg.digital.archives.go.jp/kouseisai/category/ezu/kuniezu_genroku_e.html

JPEG 2000 under alternatives– Open: IIPserver - Djatoka (Los Alamos), IIPImage

http://african.lanl.gov/adore-djatoka/, http://help.oldmapsonline.org/jpeg2000

Non JPEG 2000 Proprietary– GeoExpress, Media Manager, eRez, Zoomify etc


JPEG 2000 in practice:Examples of use

Geographic information, remote sensing, meteo– Many implementations, widely accepted. Large images, multi

spectral, etc.

Medical imaging– DICOM - seems to be cross industry support, very wary of

proprietary standards

Archival – Library of Congress, and many others now looking...!

Digital Cinema– DCI - core system for cinemas

Specific applications– Japanese driving licenses, Yahoo chat, surveillance etc


Integrating JPEG 2000 into workflow

Look at all aspects of workflow - however cannot be expert across the board!– Rationale for JPEG 2000

– Capture issues

– Transcoding - what's lost, and does it matter?

– Annotations and metadata

– Storage

– Delivery and output formats

– Client browsing


JPEG 2000 workflow:Rationale

Standardised - documented, multi-vendor, hopefully zero patent one off/royalty cost

Many variants - JPSec security, JPIP for streamed access, JP3D etc

Repurposing - same bit stream can be re-ordered for variable resolution, quality etc

Supports wide range of colour spaces and metadata standards


JPEG 2000 workflow:Capture

Continuous improvements in capture equipment– resolution , bit depth, noise, speed, power consumption

Nearing lens and film limits in quality terms?

JPEG 2000 offers some efficiencies?– can compress any bit depth (e.g current Canon cameras with

DIGIC 4 at 14 bit level)

– best in class compression (certainly significantly better than JPEG-XR or original JPEG), however probably not best in pure losslesscompression terms

However - no JPEG 2000 native devices, issues with metadata handling


JPEG 2000 workflowRAW v. DNG v. TIFF v. JPEG 2000

RAW format - basically takes camera sensor data, prior to demosaicing. Has EXIF makernotes (e.g. CR2, NEF)

– Demosaicing quality changevarious algorithms: - http://www.rawtherapee.com/RAW_Compare/

significant grounds for concern? e.g. compare Adobe Camera Raw to Bibble.. Haven't confirmed different results, but?

DNG as an intermediate? – Designed as archival format, claimed open and IPR free, based on

TIFF/EP, could be used in camera (38 out of 230, but only Pentaxfrom majors) - does it preserve all metadata?

however see http://forums.adobe.com/thread/528900?tstart=0

TIFF after demosaic, white balance, contrast etc - however 16 bit (in Adobe)

– issue! - use of absolute offsets, pointer v. integers may make inclusion in JP2 problematic, if unknown tags (e.g. makernotes?)


JPEG 2000 workflow:Transcoding

How to convert to JPEG 2000?– Assume Wellcome typical workflow - Canon camera (CR2?),

Lightroom to TIFF, Photoshop CS3(?)

Alternatives– RAW conversion software? Plenty of options from ImageMagick

to GraphicConverter - metadata an issue?e.g. for GraphicConverter - "Further metadata can be displayed on the XMP tab. Photoshop

CS writes this data. It is, however, only of interest for users who need metadata in XML format. The XMP data can only be saved in JPG, JP2 (JPEG 2000), PNG and TIFF image formats. When you use the JP2 and PNG formats, the most important EXIF information is written in the XMP section if the file contains EXIF information."

– Bespoke? - could use Kakadu, exactly as per Buckley

– Photoshop plugin - Adobe / Fnord / Luratech(?) etc

– Alternative to Lightroom etc?

– Quicktime?


JPEG 2000 workflow:Annotation and metadata

Metadata in RAW format– EXIF

– CR2 makernotes - camera #, lens etc.

– de facto (e.g. from knowledge of camera, processor)

Metadata from conversion– Adobe XMP, (+IPTC IIM, JPEG app tags etc. from other sources)

Metadata from asset management system– controlled vocabulary etc.

– ANSI Z39.87, Dublin Core, etc.

Where to store– In database, in JP2/JPX, in UUID or XML box

– Multiple versions (e.g. colour space?)


JPEG 2000 workflow:Storage

Suggestion (at least short term)– Keep JPEG 2000 as per Buckley

– Add XMP metadata from EXIF metadata plus DB

– For key images (i.e. expensive / unique) also keep CR2 (or otherRAW files), plus copy of test calibration image, plus specification documents

– should master and single access copies have same structure?


JPEG 2000 workflow:Delivery

choice of image servers– majority are Windows or Linux platform

– is objective on the fly or batch conversion?

– any need for direct image delivery (e.g. maps or other high definition objects)

– scope for JPEG 2000 direct access (now/planned)?


JPEG 2000 workflow:Client viewing

List already provided in previous slide– some useful tools (eg KDShow) which can show JPEG 2000

characteristics

– also some file validation tools such as JHove (Harvard)

– need to test final format against all know (at least popular) viewers (e.g. Wikipedia list)

– some viewers also support JPIP (e.g. jpip://...)


File formats

Alternatives exist - what are differences?– JP2 - basic format, defined in core system as a baseline

– JPX - comprehensive format, used by Adobe and others

– Other possibilities?


File formats:JP2

based on Quicktime, included in the baseline definition of the JPEG 2000 standard

simple, supported by most if not all JPEG 2000 tools and viewers

only allows sRGB and simple variant colour space, or a restricted ICC profile (have to recognise Adobe RGB as a result)

single codestream


File formats:JPX

more complex, less support

includes multiple enumerated colorspaces (e.g. ROMMRGB (ProPhoto), but not Adobe RGB by name)

allows full ICC profiles to be specified

multiple codestreams and 'clever' constructs

defines XML metadata based on DIG 35 (I3A), but still allows all JP2 options


Embedding metadata

What metadata exists– EXIF data from camera

– IPTC / XMP metadata for description

– Rights protection

– Colour management

need to define a minimum set of metadata that --must--be included in the file

should optional metadata be included directly, or added during image serving/conversion


Metadata:Issues

Need to ensure that metadata is potentially duplicated in various boxes

Need to decide which metadata constructions are included

Digital Economy Bill may make it offence to remove (some?) metadata downstream


Metadata:Initiative and standards.....

opportunity does exist to amend standard, or add further information via contribution

existing contributions have requested better information on how to handle – EXIF metadata,

– RAW archiving

– metadata and digital signature encryption

– region of interest metadata, annotation

– suggested tag for IPTC/IIM registration in JP2

– Dublin Core mapping, GML and other schema

Metadata Working Group guidelines for handling M'dta– Canon / Nokia/ MS / Adobe / Sony / Apple (Feb 2009)


Things that need doing......

Possible things to look at– performance metrics for conversion process

– validation of JPEG 2000 data as written

– compatibility with multiple viewers / decoders

– archival of selected RAW files

– quality measures and feedback to control conversion?

– 'best' image server (from requirements capture)

– Avenues to deliverConsultancy

Internal investigation

Shared experience

Networking and academic inputs

j2k workshop for the wellcome library

Documents