modernizing scala - oracle · scala macro primer experimental feature of scala 2.{10,11,12} client...

1

ModernizingScalaTheJourneytoJava8

JasonZaugg

2

3

OverviewScala2.12TraitEncodinginvokedynamicLambdaEncoding

4 .1

Scala2.12

4 .2

Scala2.12DropJava6/7supportExploitnewfeaturesinJVMImproveJavainteropNewcodegeneratorand-optimiseimplementation

NonGoals:newlanguagefeatures,majorlibrarychanges

4 .3

JavaSupport/TargetMatrixJava6 Java8 Java9

Scala2.11 ST S

Scala2.12.0 ST

Scala2.12.N ST S

4 .4

CodeGeneratorpreviously:AST->IR->optimizedIR->bytecodenow:AST->bytecode->optimizedbytecodeAOToptimization(ifenabled):

standardlocaloptimizations:jumpchain,dce,nullness,push-pop,redundantstoreetc.Scalaawarebox/unboxelimination(tuples,options)inlining(w/ScalaawareCHA)

4 .5

InteropScalalambdasnowfulfilanyfunctionalinterface(FI)ScalatraitswithconcretemethodscanbeimplementedbyJavaclasses

hence,FunctionNarenowFIs

4 .6

ConsumeJavaFunctionalAPIsUsefunctionalAPIsfromJavalibraries,e.g.j.u.streamscala> import java.util.stream._import java.util.stream._

scala> Stream.of("a", "b", "c").map[String](_.toUpperCase).findFirst.getres0: String = A

4 .7

ConsumeJavaFunctionalAPIsUsesitevarianceinStreamAPIdoesn'tplaynicelywithinference.scala> Stream.of("a", "b", "c").map(_.toUpperCase).findFirst.getres8: ?0 = A

Retrofitdeclarationsitevariance?scala> :powerscala> val List(i, o) = symbolOf[Function[_, _]].typeParamsi: $r.intp.global.Symbol = type To: $r.intp.global.Symbol = type R

scala> i.setFlag(Flags.CONTRAVARIANT); o.setFlag(Flags.COVARIANT)res6: o.type = type R

scala> Stream.of("a", "b", "c").map(_.toUpperCase).findFirst.getres8: String = A

4 .8

ExposeScalafunctionalAPIsJavalambdascanfulfilscala.FunctionNjshell> new scala.Some<String>("42").map(x -> x + "!!!").get()$1 ==> "42!!!"

4 .9

SmallerJARslambdaloversprofitthemostYMMV

5 .1

EncodingTraitswithoutthebloat

5 .2

TraitsCheatSheetclassesmayextendoneclass+manytraitstraitsmaycontainanythingthataclassdoes:

initialization,fields,methodbodiesnoconstructorparams(yet)

5 .3

2.11traitsTrait:

interfaceTincludingabstractaccessorsforfields

classT$classwithmethodbodies

Subclass:

implementinterfacemethodsbyforwardingtoimplclassfields+concreteaccessors

5 .4

2.11traitstrait T { def foo = 42; var v = 1 }class C extends T

Compileto:public interface T { int foo(); int v(); void v_$eq(int); }class T$class { static int foo(T $this) { return 42; } static void $init$(T) { T.v_eq(1); }}class C implements T { C() { T.$init$(this); } private int v; int v() { return v; } void v_eq(int v) { this.v = v } int foo() { return T$class.foo(this); }}

5 .5

2.12traits(naive)fieldencodingunchangedleavetraitmethodbodiesintheinterfacetraitconstructorasastaticmethodomitforwardersinclasses

5 .6

2.12traits(naive)trait T { def foo = 42 }class C extends T

Compileto:interface T { default int foo() { return 42; } }class C implements T

5 .7

Challenge:runtimedispatchclass C { def foo = "C" }trait T extends C { override def foo = "T" }class D extends Tnew D().foo // prints?

5 .8

Challenge:runtimedispatchJVMandScaladisagreeondispatchhereWeneedtoadda"traitforwarder"toDclass D extends T {/*synthetic*/ def foo = super[T].foo}

onlyaddincaseofdisagreement

5 .9

TraitforwarderstraitforwardersneedtonamearbitraryancestortraitJVMonlyallowsnamedparentsAddredundantparents?

5 .10

theparentofmyparentismyparentAddallancestorstoparentinterfacelistWhatcouldpossiblygowrong?!

5 .11

theparentofmyparentismyparentStartuptimeballoonedHierarchyVisitorinvm/classfile/defaultMethods.cppuncooperative

5 .12

Challenge:supercallswhataboutexplicitsupercalls?supercallsinclassesfollowJava'srestrictionsbutintraits,supercallscantargetanyancestor

5 .13

Challenge:supercallsSimplecase:trait T { def foo = 42 }trait U extends T { override def foo = -super[T].foo }class C extends U

Compileto:interface T { default int foo() { return 42; }}interface U extends T { default int foo() { return super.foo(); }}class C implements U

5 .14

superdispatchScalasuperrequiresstaticdispatchinvokespecialdispatchesvirtuallyLatentbuginScala2.11SD-143

5 .15

class A { def m = 1}

class B extends A { override def m = 2}

trait T extends A

class C extends B with T { override def m = super[T].m}

5 .16

class A { def m = 1}

class B extends A { override def m = 2}

trait T extends A

class C extends B with T { override def m = invokespecial T.m}

5 .17

class A { def m = 1}

class B extends A { override def m = 2}

trait T extends A

class C extends B with T { override def m = invokeplease A.m // }

5 .18

class A { def m = 1}

class B extends A { override def m = 2}

trait T extends A

class C extends B with T { override def m = invokeactually B.m // }

5 .19

super-dilemmaIndytotherescue?

bootstrapcouldunreflectSpecialwouldrequireelevatedpermissions

Or,addstaticaccessorsforalltraitmethods

wecanthenuseinvokestaticasbefore

can'ttargetJavadefaultsorclassmethods

introducelanguagerestrictionforsuchcases

5 .20

Cleanerstacks2.11:scala> trait T { def foo = throw null }; new T {}.foojava.lang.NullPointerException at T$class.foo(<console>:11) at $anon$1.foo(<console>:13) ... 32 elided

2.12:scala> trait T { def foo = throw null }; new T {}.foojava.lang.NullPointerException at T.foo(<console>:11) ... 30 elided

5 .21

TraitsRecapwe'veshedtraitimplclassesandmosttraitforwardersshallower,lesscruftycallstacksnon-privatemethodsstillrequirestaticaccessorsreconsidertheindy-supersolutiondownthetrack

6 .1

arriving(fashionablylate)at

theindyparty

6 .2

@PolymorphicSignaturescala> import java.lang.invoke._, MethodType._scala> val lookup = MethodHandles.lookup(); import lookup._

scala> findVirtual(classOf[Object], "hashCode", methodType(classOf[Int]))res0: java.lang.invoke.MethodHandle = MethodHandle(Object)int

scala> val hc: Int = res0.invoke(new Object)hc: Int = 1251285265

supportaddedin2.11series

6 .3

SymbolLiterals'foo // shorthand for scala.Symbol("foo")def x = f('blah)x eq q

Weusedtocacheinsyntheticstaticfieldprivatestaticsnotsupportedininterfacespublicise?indify!

6 .4

Structuralcalls(c: {def close(): Unit}) => c.close()

Compiledtoreflectivecalls+reflectioncacheagain,indifythecacheintothecallsitefuturework:dynalink

first,needtoteachitaboutInt.+etc

6 .5

indyliberationscalacbackendemitsindyfornewASTshape

notspecialcasedtousecasesabovemacros:transformapplication=>arbitraryAST(undocumented,experimental)librarylevelsupportforindy

6 .6

ScalamacroprimerExperimentalfeatureofScala2.{10,11,12}Clientcodelooks,feels,typecheckslikeanmethodcallMacroauthorwritesapairofmethods:

adeclarationwiththetypesignatureanimplthattransformstheAST

stdlibmacros:stringformatting,quasiquotesquasiquotemacroshorthandsASTde-/con-struction

6 .7

Example:linktimepatterncompilation

6 .8

patterns:ClientCode// pattern is compiled and cached when call site is linkedcompilePattern("foo.bar").matcher("foo!bar").matches

// bonus static error: Unclosed character classcompilePattern("abc[..")

6 .9

patterns:Macroclass impl(val c: Context) extends IndySupport { import ...

val bootstrapSym = typeOf[test.Bootstrap].companion .member(TermName("bootstrap")) val invokedType = NullaryMethodType(typeOf[Pattern])

def compilePattern(s: Tree): Tree = s match { case l@Literal(Constant(s: String)) => checkPattern(l.pos, s) Indy(bootstrapSym, List(l), invokedType) case _ => q"_root_.java.util.regex.Pattern.compile($s)" }}def compilePattern(s: String): Pattern = macro impl.compilePattern

6 .10

patterns:BootstrapCallSite bootstrap(Lookup lookup, String invokedName, MethodType invokedType, String value) { return new ConstantCallSite( MethodHandles.constant(Pattern.class, Pattern.compile(value)));}

6 .11

indyliberationSeeJohnRose'sideas:Scala'smacrosystemcanhelpexploretheseideas

mlvm-dev/2016-July/006661.html

6 .12

Example:Java8/9shimsCouldweusethistohelppeoplecrosstargetJava8/9?Link-timechoiceofUnsafevsVarHandle

6 .13

shims:ClientCodeclass Foo { var blah = 42 }

import Macro._

val f = new Foo()compareAndSetInt(classOf[Foo], "blah", f, -1, 0)assert(!compareAndSetInt(classOf[Foo], "blah", f, -1, 0))assert(compareAndSetInt(classOf[Foo], "blah", f, 42, 0))assert(f.blah == 0)

6 .14

autostaging? def foo(x: String) = { partiallyEvaluate { List("1", "2", "3").contains(x)) } }

Idea:macropartiallyEvaluatecouldmechanicallyturnthisinto:def foo(x: String) = { invokedynamic[LinkTimeInterpreter, List("1", "2", "3")].contains(x)}

6 .15

autostaging?LotsofrelatedresearchinScalaworld,although

"LightweightModularStaging",Rompf,Odersky"Yin-Yang:ConcealingtheDeepEmbeddingofDSLs",Jovanovicetal

integrationwithlinktimeevaldeservesmoreresearch

7 .1

indyλ

7 .2

Lambdas:BeforeandafterEssentiallythesameasJava7=>8

7 .3

Challenge:LMFassumptionsCan'tdefinetoStringUnboxingofnullsinScalais0,notNPEScalaValueClassboxingvoidreturntypeneedstobeboxed

7 .4

Challenge:LMFassumptionsscala> (() => ()).toStringres1: String = [function0]

scala> ((x: Int) => x).asInstanceOf[Any => Int].apply(null)res2: Int = 0

scala> ((() => ()) : (() => Any)).apply().getClassres3: Class[_] = class scala.runtime.BoxedUnit

7 .5

Challenge:LMFassumptionsSolution:

livewiththenewtoStringselectivelyuseadaptermethodsatthelambdatarget

7 .6

Challenge:ScalaSpecializationSpecializedtraitsleavethewrongmethodabstractShouldchangespecializationtofixthisbutitisacomplexbeast,andwe'vestartedoutwithintermediateFIstoflipthemethods.

7 .7

Challenge:Serializationjavactreatsserializablefuncinterfacesastheexceptionscalalambdasaretypicallyserializablejavac-style$deserializeLambda$wouldgrowlarge

hardlimitonnumberoflambdas(JDK-8008413)useageneric insteadlambdadeserializer

7 .8

Challenge:SerializationSolution:genericlambdadeserializer

bodyof$deserializeLambda$isanindycallsiteforacachepasseslookupalongtogenericcodethatcallsLMFmanuallytradeoff:can'tfailfastforsubtleserializationincompatibilties

7 .9

Challenge:lockscopinglazyvalsinlambdastranslateintothis.synchronizedthisusedtobetheanonclassnowitisthelambdahost,whichcontainsthelambdabodyinasyntheticmethod

7 .10

Challenge:unwantedcapturelocalmethodsinlambdastranslatetoprivatemethods......intheanonclass(before)...inthelambdahostclass(now)lambdanowretainsareferencetothelambdahostsolution:transitiveanalysistomakeliftedmethodsstatic

8

What'snext?2.12.0-RC1endofthemonthDeliverJava9supportincrementallyInvestinbenchmarking,performanceofcompiler/library

9

Thanks!Questions?