systematic document generation from xml...

23
1 SE Day @ Rutgers 4/22/08 TAXI TAXI Systematic document generation from XML Schema Antonia Bertolino , Jinghua Gao, Eda Marchetti, Andrea Polini [email protected] Istituto di Scienza e Tecnologie dell'Informazione “A. Faedo" (ISTI-CNR), Pisa

Upload: others

Post on 13-Jul-2020

25 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Systematic document generation from XML Schemaprolangs.cs.vt.edu/rutgers/meetings/seday2008/slides/Bertolino/slides.pdfSystematic document generation from XML Schema Antonia Bertolino,

1SE Day @ Rutgers 4/22/08TAXITAXI

Systematic document generation fromXML Schema

Antonia Bertolino, Jinghua Gao, Eda Marchetti, Andrea Polini

[email protected]

Istituto di Scienza e Tecnologie dell'Informazione “A. Faedo"(ISTI-CNR), Pisa

Page 2: Systematic document generation from XML Schemaprolangs.cs.vt.edu/rutgers/meetings/seday2008/slides/Bertolino/slides.pdfSystematic document generation from XML Schema Antonia Bertolino,

2SE Day @ Rutgers 4/22/08TAXITAXI

Agenda

XML and XML Schema Motivating XML-based Partition Testing (XPT) Category Partition(CP) & Mapping from CP to

XPT XPT Methodology TAXI Tool Applications Conclusion

Page 3: Systematic document generation from XML Schemaprolangs.cs.vt.edu/rutgers/meetings/seday2008/slides/Bertolino/slides.pdfSystematic document generation from XML Schema Antonia Bertolino,

3SE Day @ Rutgers 4/22/08TAXITAXI

The eXtensible Markup Language(XML)

<?xml version="1.0"encoding="ISO88591"?><card><name>John Doe</name><title>CEO, Widget Inc.</title><email>[email protected]</email><phone>(202) 4561414</phone></card>

card

name title phoneemail

John Doe CEO, Widget Inc

[email protected]

(202) 4561414

The eXtensible Markup Language (XML) is a MarkupLanguage which is today a de-facto standard to storeinformation and data.

XML documents are tree structured documents inwhich data are formatted/organised using tags

Page 4: Systematic document generation from XML Schemaprolangs.cs.vt.edu/rutgers/meetings/seday2008/slides/Bertolino/slides.pdfSystematic document generation from XML Schema Antonia Bertolino,

4SE Day @ Rutgers 4/22/08TAXITAXI

XML & XML Schema

XML Schema provides a means for defining the structure andcontent of XML documents

In the open networked world, XML Schema supportinteroperability between independently developed applications

Chinese

Italian

Page 5: Systematic document generation from XML Schemaprolangs.cs.vt.edu/rutgers/meetings/seday2008/slides/Bertolino/slides.pdfSystematic document generation from XML Schema Antonia Bertolino,

5SE Day @ Rutgers 4/22/08TAXITAXI

Automatic XML-Based Testing andBenchmarking

What we would liketo achieve

Page 6: Systematic document generation from XML Schemaprolangs.cs.vt.edu/rutgers/meetings/seday2008/slides/Bertolino/slides.pdfSystematic document generation from XML Schema Antonia Bertolino,

6SE Day @ Rutgers 4/22/08TAXITAXI

Automatic XML-Based Testing andBenchmarking

Some tools like that exist: XMLSpy, sunXMLGenerator, …

EASY WAY

Page 7: Systematic document generation from XML Schemaprolangs.cs.vt.edu/rutgers/meetings/seday2008/slides/Bertolino/slides.pdfSystematic document generation from XML Schema Antonia Bertolino,

7SE Day @ Rutgers 4/22/08TAXITAXI

Our proposal:A Systematic Automatic Approach

The approach has been inspired at-large by the well-knownCategory Partition methodology for systematic semi-automatedtest generation …

..or, you can think of it as grammar-based generation, on the XSDsyntax, although we have also introduced practical rules

XML-based PartitionTesting XPT

Page 8: Systematic document generation from XML Schemaprolangs.cs.vt.edu/rutgers/meetings/seday2008/slides/Bertolino/slides.pdfSystematic document generation from XML Schema Antonia Bertolino,

8SE Day @ Rutgers 4/22/08TAXITAXI

Mapping CP to XPT

CP XPT

Analyze Specifications Preprocessor Identify Functional Units Identify Sub-Schema Sets Partition Categories Identify Types Selecte Choices Partition Values and Structures Determine Constraints Determine “valid/invalid” constraints Generate Test Specification Generate Intermediate Instances Generate Test Cases Generate Final Instances

Page 9: Systematic document generation from XML Schemaprolangs.cs.vt.edu/rutgers/meetings/seday2008/slides/Bertolino/slides.pdfSystematic document generation from XML Schema Antonia Bertolino,

9SE Day @ Rutgers 4/22/08TAXITAXI

Identification of Sub-Schema Sets

<choice> elements partition the XML Schema intodistinct set corresponding to the CP functional units

XML Schema

choice AB

12choice

XML Schema

sequence A

1sequence

XML Schema

sequence A

2sequence

XML Schema

sequence B

1sequence

XML Schema

sequence B

2sequence

preprocessorAnalyzeSpecifications

Mapping from CP to XPT

Identify Functional Units

Identify Sub-Schema Sets

Partition Categories

Identify Types

Selecte ChoicesPartition

Values and Structures

Determine Constraints

Determine “valid/invalid” Constraints

Generate Test Specification

Generate Intermediate

Instances

Generate Test Specification

Generate Final Instances

Page 10: Systematic document generation from XML Schemaprolangs.cs.vt.edu/rutgers/meetings/seday2008/slides/Bertolino/slides.pdfSystematic document generation from XML Schema Antonia Bertolino,

10SE Day @ Rutgers 4/22/08TAXITAXI

Identification of Types

The CP categories in XPTcorrespond to theoccurrence and types ofXML elements. EX: String, sequence, all

preprocessorAnalyzeSpecifications

Identify Functional Units

Identify Sub-Schema Sets

Partition Categories

Identify Types

Selecte ChoicesPartition

Values and Structures

Determine Constraints

Determine “valid/invalid” Constraints

Generate Test Specification

Generate Intermediate

Instances

Generate Test Specification

Generate Final Instances

Mapping from CP to XPT

Page 11: Systematic document generation from XML Schemaprolangs.cs.vt.edu/rutgers/meetings/seday2008/slides/Bertolino/slides.pdfSystematic document generation from XML Schema Antonia Bertolino,

11SE Day @ Rutgers 4/22/08TAXITAXI

Partition of Values and Structures

Values of the elements: Element attributes: fixed, default … Restrictions: “minInclusive”, “maxInclusive”,

“minExclusive”, “maxExclusive”,“minLength”, “maxLength”

Information of the structure ofthe final instances Element: “minOccurs”, “maxOccurs” Attribute: “use”

preprocessorAnalyzeSpecifications

Identify Functional Units

Identify Sub-Schema Sets

Partition Categories

Identify Types

Selecte ChoicesPartition

Values and Structures

Determine Constraints

Determine “valid/invalid” Constraints

Generate Test Specification

Generate Intermediate

Instances

Generate Test Specification

Generate Final Instances

Mapping from CP to XPT

Page 12: Systematic document generation from XML Schemaprolangs.cs.vt.edu/rutgers/meetings/seday2008/slides/Bertolino/slides.pdfSystematic document generation from XML Schema Antonia Bertolino,

12SE Day @ Rutgers 4/22/08TAXITAXI

Constraints of“valid/invalid”

Two types of constraints canbe identified Valid: values in choices conform

to the specification of theXML Schema

Invalid: values in choices donot conform to the declarationof XML Schema.

preprocessorAnalyzeSpecifications

Identify Functional Units

Identify Sub-Schema Sets

Partition Categories

Identify Types

Selecte ChoicesPartition

Values and Structures

Determine Constraints

Determine “valid/invalid” Constraints

Generate Test Specification

Generate Intermediate

Instances

Generate Test Specification

Generate Final Instances

Mapping from CP to XPT

Page 13: Systematic document generation from XML Schemaprolangs.cs.vt.edu/rutgers/meetings/seday2008/slides/Bertolino/slides.pdfSystematic document generation from XML Schema Antonia Bertolino,

13SE Day @ Rutgers 4/22/08TAXITAXI

Example of “valid/invalid” contraints

Sequence:

The same sequence of element as the specification of XML Schema [Valid]

The sequence of element is differnt from the XML Schema specification [Invalid]

String:

Any strings confrom to the specification of XML Schema [Valid]

Any strings do not conform to the specification of XML schema. [Invalid]

Numeric:

Any digitals confrom to the specification of XML Schema [Valid]

Any digitals do not conform to the specification of XML schema. [Invalid]

Occurrence:

Occurrence value ∈ (-∞ , minOccurs) [Invalid]

Occurrrence value ∈ [minOccurs, maxOccurs] [Valid]

Occurrence value ∈ (maxOccurs, ∞) [Invalid]

Page 14: Systematic document generation from XML Schemaprolangs.cs.vt.edu/rutgers/meetings/seday2008/slides/Bertolino/slides.pdfSystematic document generation from XML Schema Antonia Bertolino,

14SE Day @ Rutgers 4/22/08TAXITAXI

Intermediate InstancesGenerate intermediate instance by combining the values of“minOccurs” and “maxOccurs”.

We apply the conventional Boundary Condition test approachto reduce the combinations

sub-Schema

minOccurs=0

maxOccurs=3

minOccurs=2

maxOccurs=4

A

B

Intermediate Instance

B occurs=2

Intermediate Instance

Intermediate Instance

Intermediate Instance

A occurs=0 A occurs=3

B occurs=2

A occurs=0

B occurs=4

A occurs=3

B occurs=4

preprocessorAnalyzeSpecifications

Identify Functional Units

Identify Sub-Schema Sets

Partition Categories

Identify Types

Selecte ChoicesPartition

Values and Structures

Determine Constraints

Determine “valid/invalid” Constraints

Generate Test Specification

Generate Intermediate

Instances

Generate Test Specification

Generate Final Instances

Mapping from CP to XPT

Page 15: Systematic document generation from XML Schemaprolangs.cs.vt.edu/rutgers/meetings/seday2008/slides/Bertolino/slides.pdfSystematic document generation from XML Schema Antonia Bertolino,

15SE Day @ Rutgers 4/22/08TAXITAXI

Instance Derivation

The set of final instancesis generated by giving theproper value to eachelement. The values are selected

from the choices accordingto the restrictionsexpressed in the XMLSchema.

preprocessorAnalyzeSpecifications

Identify Functional Units

Identify Sub-Schema Sets

Partition Categories

Identify Types

Selecte ChoicesPartition

Values and Structures

Determine Constraints

Determine “valid/invalid” Constraints

Generate Test Specification

Generate Intermediate

Instances

Generate Test Specification

Generate Final Instances

Mapping from CP to XPT

Page 16: Systematic document generation from XML Schemaprolangs.cs.vt.edu/rutgers/meetings/seday2008/slides/Bertolino/slides.pdfSystematic document generation from XML Schema Antonia Bertolino,

16SE Day @ Rutgers 4/22/08TAXITAXI

Instance Derivation(2)

The problem of CP method: Too manygenerations!

Our solution: Apply Pair-wise testing during the

occurrence generation Weighted Test Strategies

preprocessorAnalyzeSpecifications

Identify Functional Units

Identify Sub-Schema Sets

Partition Categories

Identify Types

Selecte ChoicesPartition

Values and Structures

Determine Constraints

Determine “valid/invalid” Constraints

Generate Test Specification

Generate Intermediate

Instances

Generate Test Specification

Generate Final Instances

Mapping from CP to XPT

Page 17: Systematic document generation from XML Schemaprolangs.cs.vt.edu/rutgers/meetings/seday2008/slides/Bertolino/slides.pdfSystematic document generation from XML Schema Antonia Bertolino,

17SE Day @ Rutgers 4/22/08TAXITAXI

Main Interface of TAXI

Page 18: Systematic document generation from XML Schemaprolangs.cs.vt.edu/rutgers/meetings/seday2008/slides/Bertolino/slides.pdfSystematic document generation from XML Schema Antonia Bertolino,

18SE Day @ Rutgers 4/22/08TAXITAXI

TAXI

The mapping from the CP to the XML Schema Partition Testinghas been partially implemented in a proof-of-concept tool calledTAXI : Testing by Automatically generated XML Instances

TAXI includes four components Schema Analyzer (XSA)

Expands and preprocesses the XML Schema, Prepares the intermediate instance frames Provides a set of final instances

Test Strategy Selector (TSS) Implements a set of test strategies. Manages the weight assignment for the elements in the

identified functional units Values Storage (VS)

Manages a database for occurrences and values assignment User Interface (UI)

Page 19: Systematic document generation from XML Schemaprolangs.cs.vt.edu/rutgers/meetings/seday2008/slides/Bertolino/slides.pdfSystematic document generation from XML Schema Antonia Bertolino,

19SE Day @ Rutgers 4/22/08TAXITAXI

Potential Applications

For validating database management systems automatically generate valid XML instances for populating

database in a systematic evaluate the performance and the quality of the associated

management systems For testing the inter-operability between applications and

for enabling the correct interactions among the interfacesused by remote components in distributed systems. Automatic and controlled generation of valid and invalid

instances enables the automated testing of I/O behavior For verifying the proper communication protocols between

web-services. SOAP-based interaction between services can be reconducted

to the corresponding XML Schemas

Page 20: Systematic document generation from XML Schemaprolangs.cs.vt.edu/rutgers/meetings/seday2008/slides/Bertolino/slides.pdfSystematic document generation from XML Schema Antonia Bertolino,

20SE Day @ Rutgers 4/22/08TAXITAXI

Black-box testing

Page 21: Systematic document generation from XML Schemaprolangs.cs.vt.edu/rutgers/meetings/seday2008/slides/Bertolino/slides.pdfSystematic document generation from XML Schema Antonia Bertolino,

21SE Day @ Rutgers 4/22/08TAXITAXI

100% automaticXSLT testing

Page 22: Systematic document generation from XML Schemaprolangs.cs.vt.edu/rutgers/meetings/seday2008/slides/Bertolino/slides.pdfSystematic document generation from XML Schema Antonia Bertolino,

22SE Day @ Rutgers 4/22/08TAXITAXI

Conclusions

TAXI tool can automatically derive a set ofinstances that systematically covers a XSD

It can be applied for interoperability validation,database benchmarking, black-box testing, …

Future work Invalid instance generation: Robustness testing Tool refinement Experimental validation

Page 23: Systematic document generation from XML Schemaprolangs.cs.vt.edu/rutgers/meetings/seday2008/slides/Bertolino/slides.pdfSystematic document generation from XML Schema Antonia Bertolino,

23SE Day @ Rutgers 4/22/08TAXITAXI

Thank you!To get TAXI, or for joint experimentation

• Beta Version of TAXI on line at http://labse.isti.cnr.it/Or

• send an email to [email protected]