functionality working group report 2010-05-27
DESCRIPTION
Functionality Working Group Report 2010-05-27. Dagobert Soergel University at Buffalo. y. Scenario 1. Simple solution. Provider functionality (Web-based) Find correct town name Find composite object for that town, show outline - PowerPoint PPT PresentationTRANSCRIPT
Functionality Working GroupReport 2010-05-27
Dagobert SoergelUniversity at Buffalo
1
y
2
Scenario 1. Simple solution
• Provider functionality (Web-based)– Find correct town name– Find composite object for that town, show outline– User clicks on outline item to invoke get function for the
component
• Interoperability problems focus on consumer ability to use data formats– cross-domain problem: content - functionality
3
Scenario 1. Simple solution - 2• Consumer needs
– either functions, such as a video editor, that are data-interoperable with the provider data format
– or mediators that can convert (map) from the provider format to the consumer format
• Supported by a function repository that can find the needed functions to run on he consumer's computer or as a Web service– Must be able to deal with needed formats– Must have functionality required by consumer– Must run in consumer's computing environment (functional
interoperabilty)
• Needs function representation language
4
Scenario 2. Simple solution
• DB-mix replicates content from DB A an DB B(establish update schedule)– Cross-domain interperability problem Content –Functionality– DB mix format needs to preserve all information from either format
A or format B, both document structure and metadata, including rights
– Need to convert from format A and format B to format mix. Need mediatorsApplies to document format and metadata format
– Special problem: Mapping thesauri A and B to thesaurs mix
5
Scenario 2. Simple solution - 2
• Search functionality implemented on DB mix– Needs complete feature analysis of search A and search B using
function representation language– Need functions to implement all search features– Can pieces of search A code and search B code be reused?
Question of function interoperability– Also data dependence of search features. Only documents
coming originally from A or B may have data required for a search feature – need to warn userGeneral issue: Warn users of incomplete interoperability
– Look and feel of search interface - product compatibilityMay need to provide several interfaces: native DB mix, A, and BNote: Mix can have A interface for invoking a feature even if the implementation of the feature is different ini mix
6
Scenario 1. Simple solution - 3
• Same problems for document display functionality
7
Function repository
• Enhanced function registries for system interoperability Paper for the ECDL 2010 workshop
• Two-pronged approach– Review existing repositories of Web services and software modules– Specify requirements and vision from WG prespective– Describe how exisiting repositories can be enhanced
• It was noted that a function repositories and software libraries used by programmers are the same thing
• Extended function description template next slides
8
Description / specification /profileof functions
Function Specificationfacilitates the identification of what a function does and how either a system or a human may interact with it.
9
Description / specification /profileof functions - 2
Function description/specification template• The template shown below applies to
– the description of an abstract function, such as search or annotate;– the description of specific software components implementing a
function. – Not all items apply to the general level, or the description stays very
broad, for example with regard to data formats. • Hierarchical inheritance:
The template instance for an abstract function has many subordinate template instances for software components implementing that function
• Two hierarchies of functions: isa and partOf
10
Description / specification /profileof functions - 2
• Template focuses on semantics of function specification• Function description template as a specialization of a
general resource description template (in line with the Reference Model)For example, usual metadata are assumed
• This is a preliminary template. It will be amended as it is applied.
• Template needs to be developed with Architecture WG
11
Function specification template
12
Function Behavior
API/Interface Specification
Dependencies/Relationships/Use
Interoperability Concerns
Assessment. Performance. Advice for use
Usage conditions
Plus all other elements from the general Resource Description Template
Function specification template
13
Function BehaviorDescription: What is doneInteraction with Actors (Systems/Users)
Is the function invoked by the user or the systemWhat actions does the user takeWhat actions does the system takeSpecial user groups /roles; user characteristicsCan the function be applied to different contexts
API/Interface Specification
Dependencies/Relationships/Use
Interoperability Concerns
Assessment. Performance. Advice for use
Usage conditions
Function specification template
14
Function Behavior
API/Interface SpecificationInput: Data and parameters, data formats / standardsOutput: Data and parameters, data formats / standardsPreconditionsPostconditions
Dependencies/Relationships/UseOperating environment in which the function runs. Other functions it needsOther functions that invoke this functionOther functions invoked. Composite functionsWork flow
Interoperability Concerns
Assessment. Performance. Advice for use
Usage conditions
Function specification template
15
Function Behavior
API/Interface Specification
Dependencies/Relationships/Use
Interoperability ConcernsWhat is required for interoperability (distinguish type of interoperability, for example product compatibility).How does a specific implementation meet these requirements
Assessment. Performance. Advice for useFor specific types of functions, such as search, the template should include quality parameters (for example from the DLRM to assist with assessement and facilitate comparison of assessments – software evaluation criteria
Usage conditions
General dl.org issues• The Reference Model has not taken hold
– Partially a problem of presentation– Reference model can serve as the beginning of a detailed
function ontology needed for function representation
• Relationship between the Reference Model, the State-of-the-Art report, and the Cookbook needs to be clarified. – The three documents need to be seen as three sides of the same
pyramid and strongly cross-referenced
• Cookbook needs to have best practices, need input from practice
• All documents need to be edited by a an expert who is a native English speaker
• All documents need to be live, Wikipedia-style (with stronger central editorial control)
16
General dl.org issues - 2
• All domains need to specify – requirements for data formats to capture data in that domain, such
as rules that codify policies– requirements for functions that are needed to process these data for
domain-specific purposes, such as the interpretation of rules to enforce policies
• All domains need ontologies
• Bilateral interoperability is not enough, need multilateral and union solutions
• Provider – consumer is not the best terminology. Often there is interchange rather than one-way flow
17
WG Functionality
• Other members
• Questions
18
Scenario 1. Simple solution
• S
19
Ultimate objectives• Promote rich functionality over a wide range
of systems with a consistent interface • Promote best practices and innovation by
educating DL designers, developers, administrators, and users about the rich array of DL functionality
• Enable finding and reusing software modules that implement desired functionality– for developers: reuse existing modules and design for interoperability– for DL managers:cutting-edge functionality in configuring a DL system– for users: run a module "on the fly" to accomplish a task.
• Enable federated search
20
Outline
• Objectives and scope• Function Interoperability Framework
– Definition of function– E-R schema for database of function descriptions– Definition of function interoperability– Function description template
• Conclusion
21
Objectives and Scope
22
Ultimate objectives• Promote rich functionality over a wide range
of systems with a consistent interface • Promote best practices and innovation by
educating DL designers, developers, administrators, and users about the rich array of DL functionality
• Enable finding and reusing software modules that implement desired functionality– for developers: reuse existing modules and design for interoperability– for DL managers:cutting-edge functionality in configuring a DL system– for users: run a module "on the fly" to accomplish a task.
• Enable federated search
23
Interoperability and reuse scenariosScenarios1 Find desired functions, and modules that implement them, and
assess their interoperability. Enable functionality sharing2 Enable content sharing and federated search3 Make switching from one DL to another easy for the user
Dealing with these scenarios requires1 Understanding the many ways in which functions interoperate2 A database with detailed descriptions of functions,
revising and extending the DELOS Digital Library Reference Model
Solution: Function Interoperability Framework
24
Interoperability and reuse scenariosExamples
• The developer of a Browse module looks for anautomatic clustering module to incorporate browsing by cluster
• A DL administrator wants to make available a better image search system
• A user found 30 documents in a DL.Wants to invoke a Web service to create a multi-document summary
25
The role of the WG Functionality
26
27
Function Interoperability Framework
28
What is a function?
A function in the DLRM is an action a DL component or a DL user performs (abstract function, emphasis on semantics, what does the function do).
In the followingfunction is used broadly to mean either
abstract function or software component implementing a function
29
Some Functions illustrating Interoperability issues
30
Behind the scene For usersFeature extraction
Classification / clustering
Sharing authority files
Log file analysis
Sharing user profiles
Harvesting , aggregating
Shared storage and backup
Federated search
Incorporating content from other places on the fly
Display and visualization
Timelines
Maps
Playing videos
Same look-and-feel browse
Three parts of the Function Interoperability Framework
• An entity-relationship schema• A taxonomy of ways in which functions can
interoperate• A template for the description of functions
and software components
Note: Strong overlap with Architecture WG
31
What is function interoperability 1
1 Interoperability (system perspective, focus on software components)1.1 Composability (f2 can work with f1)1.2 Replaceability / interchangeability (f2 can replace f1)
(f1 and f2 serve same purpose)1.1 and 1.2 can be based on process or on data (exchanging data or using same data)
2 Cross-function (cross-product ) compatibility (user perspective)Similar detailed functionality and user interface
32
Description / specification /profileof functions
Function Specification: facilitates the identification of what a function does and how one (either a system or a human) may interact with it.
Function description/specification template• The template shown below applies to
– the description of a general function, such as search or annotate;– the description of specific software components implementing a
function. – Not all items apply to the general level, or the description stays very
broad, for example with regard to data formats.
• Template focuses on semantics of function specification• This is a preliminary template. It will be amended as it is applied.
33
Function specification template
34
Function BehaviorDescription: What is doneInteraction with Actors (Systems/Users)
Is the function invoked by the user or the systemWhat actions does the user takeWhat actions does the system takeSpecial user groups /roles; user characteristicsCan the function be applied to different contexts
API/Interface Specification
Dependencies/Relationships/Use
Interoperability Concerns
Emerging function ontology
Ontology of functionsFunction specification vocabulary
Will emerge over time as the database of function descriptions is populated through wide collaboration (crowd-sourcing)
35
Sub-functions of annotate
36
Select object to be annotated(need to indicate selection method)
Mark region in the object(many different methods depending on the object)
Select type of annotation (highlight, mark with special meaning, text, image, sound)
If text, image, sound
Specify relationship to object to be annotated
Select or create the annotating object (possibly specifying a region
Annotating within one system
Annotating across systems
Conclusion
37
Recap
• ObjectivesInteroperability use cases
• WG will produce a Function Interoperability Framework
• dl.org should set up an environment in which the DL community can produce a database of function descriptions
38
Expected Outcomes
• Interoperability State-of-the-Art survey• Extensions to the Delos Reference Model• Contributions to the Cookbook:
Best practices in functionalityIdea of design patterns for DLs
• One or more papers• Training course materials
WG Activities
• Definition of Functionality WG scope• Specification of Function Interoperability• Contribution to the State-of-the-Art Survey
– Function Interoperability Issues and Solutions
• Contribution to the Reference Model– Extensions to facilitate additional topics e.g. composite
functions
WG Activities cont.
• Contribution to the Cookbook– Interoperability Issues and Solutions
• Drafting of a “Functionality Specification Template”• Specification of related Use-Cases
– Integration of D4Science and DRIVER platforms
• Presentation of WG outcomes at the 1st DL.org Workshop in Corfu (ECDL 2009)
WG Activities cont.
• Publications– “A Functionality Perspective on Digital Library
Interoperability”, to be presented in ECDL 2010
• WG Meetings– Three (3) face-to-face meetings– Four (4) virtual meetings
Thank you
Interoperability use casesFocusing on behind-the-scenes operations• Centralized services that provide the same service to
multiple libraries – classification servers– format conversion– data validation
• The services must be interoperable − based on data − with many DLs
• Can be achieved through– standards for data formats – standardized authority files.
44
What is function interoperability 2
A Interoperability of functions based on process
A1 Interoperability of use (composability)Function f1 can use function f2 (conversely, f2 can work in the
framework of f1))
A2 Special case: Interoperability with environment E (composability)Function f1 can work in environment E
A3 Interoperability based on working in same operating environment E (replaceability / interchangeability) If Function f1 can operate in environment E AND f2 (having the same purpose as f1) can also operate in E, Then f2 can replace f1
45
What is function interoperability 3B Interoperability of functions based on data (content)
B1 Interoperability based on exchanging data (composability)If Function f1 can operate on the output of f2, Then f1 and f2 can work together. f1 and f2 may also exchange data as they run concurrently
B2 Interoperability of functions with data (composability) If f1 can make use of data set D or of data formatted according to DFThen Function f1 is interoperable with a data set D or a data format DF
B3 Interoperability of function based on using same data (replaceability)If Function f1 is interoperable with a data set D or data format DFAND Function f2 (serving the same purpose as f1)is also interoperable with D or DF, respectively,Then f2 can replace f1
46
Issues in interoperability
• API mismatch• Mismatch in programming environments
Needed components missing• Mismatch in data formats
(overlap with WG Content)
47
Interoperability use cases
Exchange of program modules between D4 Science and Driver•Each system would describe the functions it implements (e.g. feature extraction from documents or data transformation using grid resources), considering
– the semantics of the function (what the program module can do) – the technical (and, as relevant, administrative) conditions of use.
•Each system could then search the functions offered by the other and reuse program modules.
48
Interoperability use cases
Single depositEuropean project OpenAIRE
– Central portal where users come to deposit their publications.
– The internal deposition service subsequently forwards/deposits them in the corresponding local repositories.
– Requires interoperable functionalities among the various repository platforms.
49
50
Browsing Collaborating Customizing Filtering Providing access Recommending Requesting Searching Visualizing
Annotating Classifying Clustering Evaluating Extracting Indexing
Measuring Publicizing
Rating Reviewing (peer)
Surveying Translating
(language)
Conserving Converting
Copying/Replicating Emulating Renewing
Translating (format)
Acquiring Cataloging
Crawling (focused) Describing Digitizing
Federating Harvesting Purchasing Submitting
Preservational Creational
Add Value
Repository-Building
Information Satisfaction
Services
Infrastructure Services
E-R schemafor a function description database
Entity types (resource types) (examples)
Relationship types (examples)
Function
SoftwareComponent(a software system, software module, or code snippet)
DesignPattern (Rike Brecht, Doct. Cons.)
Data set
Data format
Resource <hasComponent> Resource
Function <implementedBy> SoftwareCo.
Function <represented by>DesignPattern
Resource <interoperableWith> Resource
51
Function specification template
52
Function Behavior
API/Interface SpecificationInput: Data and parameters, data formats / standardsOutput: Data and parameters, data formats / standardsPreconditionsPostconditions
Dependencies/Relationships/UseOperating environment in which the function runs. Other functions it needsOther functions that invoke this functionOther functions invoked. Composite functionsWork flow
Interoperability ConcernsWhat is required for interoperability (distinguish type of
interoperability, for example product compatibility).How does a specific implementation meet these
requirements
Sub-functions of search
53
Quick Search Advanced SearchEnter a query and click search
Enter keywords or phrases for selected field
Limit results to
Search subscribed titels
Clear
Enter a query and click search
Enter keywords or phrases for selected fields
Select keyword from a list
Select Boolean operator (explicit)
Define phrase match (explicit)
Clear
Search within results
Limit results to (preselection)
Sort by (preselection)
Select display options
Display X results per page
Display search history
y
54