Class Visibility Errors Slide 1

<Insert Picture Here>

Class Visibility ErrorsBryan AtsattFebruary 2009

Class Visibility Errors Slide 3

Class Visibility Errors

Not Enough Visibility (not-found)• ClassNotFoundException• NoClassDefFoundError

Split Visibility (split-packages)• IllegalAccessError (package-private, different loaders)

Too Much Visibility (duplicates)• LinkageError (loader constraint violation)• ClassCastException (same name, different loaders)

Most developers have not seen duplicate errors… yet.

Class Visibility Errors Slide 4

Duplicate Class DefinitionsTicking time bombs

• A duplicate exists when there are two or more Class instances with the same name

• Duplicates always have different defining loaders• Frequently result in runtime errors

• LinkageError• The JVM enforces "Loading Constraints" (5.3.4) to avoid

collisions: "It is essential that any type name N mentioned in the field or method descriptor denote the same class or interface when loaded by L1 and when loaded by L2.“

• ClassCastException• Well known in the wild (e.g."child-first“ loading from Servlet

specification, OSGi). Very confusing since type names are same. Cause and effect widely separated in time.

Class Visibility Errors Slide 5

Duplication ErrorsOracle’s experience (>= 2003)

• Modules (“shared-libraries”) in application server• Named, versioned loader instances• Versioned imports complex (acyclic) graph• Rich reflective api and diagnostic machinery

• Highly successful at solving the versioning problem, but…

• Duplication errors suddenly common• No prevention model in place• Fancy diagnostics and education averted disaster

Class Visibility Errors Slide 6

Duplication ErrorsA support nightmare

• Subtle conditions, which few developers understand• Class loading is like plumbing: ignored until it breaks, messy

when it does and often requiring special expertise to fix

• Extremely hard to diagnose• Error messages rarely contain enough information• Loading often deferred until first use, stack trace confusing• Loading can be highly recursive, stack often very deep• Stack trace does not provide instance data, cannot determine

which loader(s) are involved• No useful reflective api on ClassLoader• Most loader code pathways are non-debug or native so

mostly useless in a debugger

Class Visibility Errors Slide 7

How Are Duplicates Created?

• Today • Copies of jars in different loaders• EE implementations that support child-first• Poorly behaved custom loaders

• Tomorrow• Multiple versions of a module• Copies of classes in a module that are available in another• Wrong import choice when multiple candidates are available

Class Visibility Errors Slide 8

Modules Change The Balance

• Modules should reduce frequency of not-found errors at runtime, but...

• Will increase frequency of them at compile time, initially, since dependency declarations must be precise.

• Tools can certainly help.

• Without intervention, modules will significantly increase frequency of duplication errors at runtime.

Class Visibility Errors Slide 9

Modules == More LoadersMore loaders == more opportunities for duplicates

• Finer grained class partitioning• A small, simple tree of loaders becomes a potentially large

(even cyclic) graph.• Tendency to bundle classes defensively or for convenience

• Multiple simultaneous module versions• Unsophisticated use of version requirements

• Compiler can catch many error cases, but only for a specific environment.

Class Visibility Errors Slide 10

Error CasesConflicting imports, one module

• These cases use an example “animal” module system, with a simple property syntax. Multiple versions of “akita” are present in the repository.

pika: Module-Imports=akita, version=1.0; newt; akita, version=0.9

• Module “pika” imports module “akita” twice, “newt” once; silly, but possible.

• If not prevented, duplicates from akita will be visible to pika.

See AnimalModuleTest.conflictingImportsFail()

Class Visibility Errors Slide 11

Error CasesConflicting module imports == LinkageError

vole: Module-Imports=akita, version=0.9 wombat: Module-Imports=vole; akita

• Vole ctor takes Akita: public Vole(Akita akita)• Wombat instantiates Vole: new Vole(new Akita());

See AnimalModuleTest.moduleImportsCauseLinkageError()

Class Visibility Errors Slide 12

Error CasesConflicting package imports == LinkageError

raven: Imports=akita, version=0.9

shrew: Imports=raven; akita

• Raven ctor takes Akita: public Raven(Akita akita)• Shrew instantiates Raven: new Raven(new Akita());

See AnimalModuleTest.packageImportsCauseLinkageError()

Class Visibility Errors Slide 13

Error CasesConflicting imports == ClassCastException

newt: Module-Imports=akita, version=0.5

oryx: Module-Imports=akita; newt

• Newt ctor publishes Akita instance to static List<Akita>.• Oryx.toString() tries to return loader name for instance in

static list.• Unspecified version in oryx selects newer version.• Compiler generated downcast to Akita in Oryx fails.• The same modules do not fail when only the 0.5 version of akita

is present.

See AnimalModuleTest.moduleImportsCauseClassCast() and AnimalModuleTest.moduleImportsDoNotCauseClassCast()

Class Visibility Errors Slide 14

Error CasesEmbedded copy == ClassCastException

newt: Module-Imports=akita, version=0.5 quetzal: Imports=newt

• Quetzal contains a copy of classes from akita, same generics downcast results in error when it tries to reference Akita instance from the list in newt.

See AnimalModuleTest.embeddedCopyCausesClassCast()

Class Visibility Errors Slide 15

Duplicates Acceptable if Not Shared

• If C does not expose classes from B2, either by re-exporting them or by sharing instances, then A‘s import of C will not result in failures.

• Duplicates that are hidden implementation details are fine.

• If not allowed, A is forced to move to B2, or to find a version of C that does not conflict, neither of which is required here; in a complex system, this behavior would be a serious usability issue.

x y == x imports y

Class Visibility Errors Slide 16

So… rule out all visible duplicates?

• Maybe, but first, to detect them, we need to be able to determine what classes or packages are "exposed" by each module.

• So assume that we can at least ask each module what packages it exports.

• Now, we can collect transitive dependencies, discover all exported packages, and ensure that modules share a common provider where packages intersect.

Class Visibility Errors Slide 17

But… We Can Do Better

• If package level imports are possible (ever), then we want a slight refinement.

• Given that a single module will likely export many packages, an importer may select a subset of them; if classes from that subset don’t expose the duplicate, that importer is shielded from errors.

• If C exports packages x, y and z, and only y refers to classes from B, then importers of x or z don't care about the duplicate.

Class Visibility Errors Slide 18

Maximize Resolution FlexibilityImplied dependencies

• To support this refinement, we need to know what package imports are “implied” by a given package export.

• That is, if an exported class uses a class from another package (e.g. in a method signature), as long as we can determine that fact at runtime, we can enforce the no duplicates rule.

• And we enforce it at a granularity that maximizes resolution flexibility.

Class Visibility Errors Slide 19

Error Avoided

• Identical scenario as raven & shrew, but declare 'implied' dependency:

toad: Exports=toad, depends=akita Imports=akita, version=0.9 uakari: Imports=toad; akita

• Toad ctor takes Akita: public Toad(Akita akita)• Uakari instantiates Toad: new Toad(new Akita());

• System ensures that toad and uakari both select akita 0.9.

See AnimalModuleTest.packageImportsAndDependsDoesNotCauseLinkageError()

Class Visibility Errors Slide 20

Summary

• Many failures occur only when multiple versions are present and will therefore manifest very late in the cycle.

• Duplication is a double-edged sword:• Useful to avoid forced split of dependency graph when

duplicates are hidden implementation details.• Dangerous when duplicate types are surfaced.• Implied dependency declarations enable disambiguation.

Class Visibility Errors Slide 21

Summary, continued.

• Early detection is critical to avoid support nightmare:• compile time (one "model" environment)• resolution time (actual environment)

• Enumeration of package level exports is required for duplicate detection and warning/failure at resolution time.

• Class space "consistency" model is required, but enforcement must be controlled declaratively; tools should generate this information.

Class Visibility Errors Slide 22

Resolution Constraints

1. A module must fail to resolve if a required import cannot be satisfied.

2. A failed module must not be used to satisfy imports.3. Importing a module M is equivalent to importing all packages

exported by M.4. A module may declare that it both exports and imports package

p; once resolved it may either export or import p, not both.5. If multiple providers exist for an import, resolved providers are

preferred over unresolved, and higher versions over lower.6. Modules that share a dependency must select a common

provider or fail; a shared dependency on package s exists between modules M1 and M2 when all of the following are true:• M1 imports package p from M2

• Package p depends on package s• M1 either imports or exports package s

Class Visibility Errors Slide 23

AppendixOSGi ‘uses’ example from R4 specification

• GivenA: Import-Package: q; version=”[1.0,1.0]”

Export-Package: p; uses:=”q,r”

B: Export-Package: q; version=1.0

C: Export-Package: q; version=2.0

• Shared dependency on q means D fails to resolve

D: Import-Package: p, q; version=2.0

Class Visibility Errors Slide 24