markus keuneke partial data-models

Software SystemsEngineering

Managing the Evolution and Managing the Evolution and Customization of Database Schemas Customization of Database Schemas in Information System Ecosystemsin Information System Ecosystems

Markus [email protected]

Hendrik [email protected]

Klaus [email protected]

2CAISE 2013 Brummermann, Keunecke, Schmid

DB Customization and Evolution Software SystemsEngineering

• Motivation

• Problem

• Approach

• Evaluation

• Conclusion

Agenda

Motivation | Problem | Approach | Evaluation | Conclusion



Context

• HIS is a non profit company that is jointly owned by the Federal Republic and the Federal States of Germany

• University Management System called “HISinOne”

• 400 employees / 200 in the IT department / 30 developers

• HISinOne developed since 2007

• 7 versions released up to now

– 5.3 Mio LoC (current development)

– 814 database tables, 6362 columns (current development)




Ecosystem

core

PublicationManagement

QualityAssessment

Project Management

OutputMeasurement

Customer 1

core

Customer 2

core

core

Customer 2




V1 V2

V1' ?

Evolution

Customi- zation

Evolution and Customizig

Time




• Two problem areas:

– Development strategies

– Integration at the customers' sites

• Research Questions

– RQ1 How can separate organization-wide data-models be split into small-scale feature-specific models and be managed on this level?

– RQ2 How can features, developed by customers and third party vendors, be integrated into one combined data-model?

– RQ3 How can the database be modified at the customer site to fulfill the requirements of installed and updated feature packs?

Scope




Feature Pack

Program CodeUser Interface…Partial Object ModelPartial Database ModelPartial Default Data

Content of a Feature Pack




Extending the object model




Lifecycle integration

upload

install activate

deactivate de-install

preserving de-install

deletepurge

update


installednon-existent data preserved available active



Integration Model

• Based on comparison of nominal and actual state of database

• Integration of partial models in 4 steps

– Combine the partial models

– Initial structure adjustment

– Data modifications

– Final structure adjustment




Combine partial models

publication

quality_value

publication_qa

publication_project

research_project

author

Development organization Third party vendor

Quality Assessment (F3)

Project Output Measurement (F1)

Publication Management

Research ProjectManagement

Data modelFeature PackTableForeign key constraintFK pair for exactly 1:1




Update Publication Management, De-install Quality Assess.Motivation | Problem | Approach | Evaluation | Conclusion

Before

idpub_idvalue_id

quality_assessment

idname

quality_value

publication

idtitleurl...

Old Publication Management Quality Assessment installed

Old Publication Management Quality Assessment installed

Original Database State




Before Desired

idpub_idvalue_id

quality_assessment

idname

quality_value

publication

idtitleurl...

publication

idpub_idurl

publication_url

idtitle

...




idpub_idvalue_id

quality_assessment

idname

quality_value

publication

idpub_idurl

publication_url

idtitleurl...

Initial Structure Adjustment

Before Desired

idpub_idvalue_id

quality_assessment

idname

quality_value

publication

idtitleurl...

publication

idpub_idurl

publication_url

idtitle

...




idpub_idvalue_id

quality_assessment

idname

quality_value

publication

idpub_idurl

publication_url

idtitleurl...

Initial Structure Adjustment

Before Desired

idpub_idvalue_id

quality_assessment

idname

quality_value

publication

idtitleurl...

publication

idpub_idurl

publication_url

idtitle

...

Copy content of url column to new table

Copy content of url column to new table

Data Modification




idpub_idvalue_id

quality_assessment

idname

quality_value

publication

idpub_idurl

publication_url

idtitleurl...

Initial Structure Adjustment Final Structure Adjustment

Before Desired

idpub_idvalue_id

quality_assessment

idname

quality_value

publication

idtitleurl...

idpub_idvalue_id

quality_assessment

idname

quality_value

publication

idpub_idurl

publication_url

idtitle

...

publication

idpub_idurl

publication_url

idtitle

...

url



Data modification• Set of Patterns

– Load default data

– Add default data

– Copy table

– Copy column

– Move column

– Introduce key table

– Disambiguate sort order

– SQL statements

– Program code




Evaluation – Methods• E1 Small prototype implementation by authors• E2 Large feature implementation by fellow developer• E3 Experience gathering from customers, support, product

manager and fellow developers, most oft them have morethan 10 years of experience

• E4 Metric about the db structure and performance of HISinOne

RQ1 Model Splitting E1, E2

RQ2 Model Combining E3, E4

RQ3 DB Adjustment E3




Evaluation – E4 Metrics Collection

• Gathered metrics

– Statistics on table and column structure

– Performance of database operations

– Performance of database adjustment

• Noteworthy results

– Approach leads to tables with few columns

– Original system has tables with few columns (~8 columns per table)→ Many small tables have no impact on performance

– e.g. 1 database server handles the load of 20 application servers

– Combining all partial models takes about 50 ms




Conclusion

• Approach to database management and evolution supporting customization and (partial) evolution of the database schema

• Core functionality has been in production use for many years, with over a hundred different customers

• Recent extension to support fine-grained evolution and customization in production for a few years

• Further work

– Improve feature pack support on the application level

– Integrate with dependency management system




Questions?


markus keuneke partial data-models

Technology

id quality

id value

id url publication

assessment id

schmid db customization

value publication id

value publication id

install quality