Writing DSL with Applicative Functors

Download Writing DSL with Applicative Functors

Post on 26-May-2015

1.081 views

Category:

Technology

0 download

Embed Size (px)

DESCRIPTION

Prsentation au Paris Scala User Group le 21/08/2014

TRANSCRIPT

<ul><li> 1. Writing DSL withApplicative FunctorsDavid GalichetFreelance functional programmer!twitter: @dgalichet</li></ul><p> 2. Content normalization We want to parse heterogeneous data formats(CSV, XML ) and transform them to a pivot format(Scala object in our case) The transformation should be described as a DSL This DSL must be simple to use, and enable anykinds of data transformations or verifications 3. Expected DSL formatval reader = ( !Pick(0).as[String].map(_.capitalize) and !Pick(1).as[Date].check(_.after(now())) !).reduce(FutureEvent)!!reader("PSUG; 21/08/2014") // returns Success(FutureEvent("PSUG",date))Inspired by Play2 Json API 4. tag: step1Conventions &amp; material Code is available on Github : https://github.com/dgalichet/PsugDSLWritingWithApplicative Code revision is define on the top of any slidesincluding source code (just checkout the specifiedtag) 5. tag: step1Reading single entry We have a CSV line and we want to read one column We will introduce several abstractions: Picker: fetch a data from CSV or XML Result: either a Success or a Failure Converter: convert value from Picker to Reader Reader: container with methods to process itscontent 6. tag: step1Introducing Pickercase class Picker(p: String =&gt; Result[String]) {!def as[T](implicit c: Converter[T]): Reader[T] =c.convert(p)!}!!object CsvPicker {!def apply[T](i: Int)(implicit separator: Char): Picker =Picker { s: String =&gt;!val elems = s.trim.split(separator)!if (i &gt; 0 &amp;&amp; elems.size &gt; i) Success(elems(i).trim)!else Failure(s"No column ${i} for ${s}")!}!} 7. tag: step1Introducing Pickercase class Picker(p: String =&gt; Result[String]) {!def as[T](implicit c: Converter[T]): Reader[T] =c.convert(p)!}!!object CsvPicker {!def apply[T](i: Int)(implicit separator: Char): Picker =Picker { s: String =&gt;!val elems = s.trim.split(separator)!if (i &gt; 0 &amp;&amp; elems.size &gt; i) Success(elems(i).trim)!else Failure(s"No column ${i} for ${s}")!}!}Picker wraps a function from String to Result 8. The Resulttag: step1sealed trait Result[+T]case class Success[T](t: T) extends Result[T]case class Failure(error: String) extends Result[Nothing]! 9. The Convertertag: step1trait Converter[T] {def convert(p: String =&gt; Result[String]): Reader[T]}!!object Converter {!implicit val string2StringConverter = new Converter[String] {!override def convert(p: String =&gt; Result[String]) = Reader[String](p)!// See code on Github for more converters!} 10. The Convertertag: step1trait Converter[T] {def convert(p: String =&gt; Result[String]): Reader[T]}!!Convert the content of the Picker to a Reader!object Converter {!implicit val string2StringConverter = new Converter[String] {!override def convert(p: String =&gt; Result[String]) = Reader[String](p)!// See code on Github for more converters!} 11. The Readertag: step1case class Reader[O](p: String =&gt; Result[O]) {def apply(s: String): Result[O] = p(s)}A Reader doesnt contain a value but a process totransform original data (CSV line or XML) to aResult 12. Usage sampletag: step1import Converter._ // import implicit converters!implicit val separator = ;!!CsvPicker(1).as[String].apply("foo;bar") === "bar" 13. tag: step2Enhancing the Reader The first defines a very simple Reader. We mustadd a method to combine two instances of Reader We will also enhance Failure to store multipleerror messages 14. tag: step2Enhancing the Readercase class Reader[O](p: String =&gt; Result[O]) {!def apply(s: String): Result[O] = p(s)!!def and[O2](r2: Reader[O2]): Reader[(O, O2)] = Reader { s: String =&gt;!(p(s), r2.p(s)) match {!case (Success(s1), Success(s2)) =&gt; Success((s1, s2))!case (Success(_), Failure(f)) =&gt; Failure(f)!case (Failure(f), Success(_)) =&gt; Failure(f)!case (Failure(f1), Failure(f2)) =&gt; Failure(f1 ++ f2)!}!}!def map[T](f: O =&gt; T): Reader[T] = Reader { s: String =&gt;!p(s) match {!case Success(o) =&gt; Success(f(o))!case f: Failure =&gt; f!}!}!def reduce[T] = map[T] _ // alias for map!} 15. tag: step2Enhancing Result typesealed trait Result[+T]!case class Success[T](t: T) extends Result[T]!case class Failure(error: NonEmptyList[String]) extendsResult[Nothing]!!object Failure {!def apply(s: String): Failure = Failure(NEL(s))!}!!case class NonEmptyList[T](head: T, tail: List[T]) {def toList = head::taildef ++(l2: NonEmptyList[T]): NonEmptyList[T] = NonEmptyList(head,tail ++ l2.toList)}object NEL {def apply[T](h: T, t: T*) = NonEmptyList(h, t.toList)} 16. Usage sampletag: step2implicit val separator = ';'implicit val dtFormatter = new SimpleDateFormat("dd/MM/yyyy")import Converter.string2StringConverterimport Converter.string2DateConverter!!val reader = (CsvPicker(1).as[String] andCsvPicker(2).as[Date]).reduce { case (n, d) =&gt; FutureEvent(n, d) }!reader("foo;bar;12/10/2014") === Success(FutureEvent("bar",dtFormatter.parse("12/10/2014")))!!case class FutureEvent(name: String, dt: Date) 17. tag: step2Usability problem The use of reduce (or map) method to transform aReader[(0, 02)] into an instance ofReader[FutureEvent] for example is quiteverbose This will be even more verbose for instances ofReader[(0, (02, 03))] We want the API to automatically bind tupleelements to a constructor as we can encounter inPlay2 Json API 18. tag: step3Applicative functors To tackle our problem, we will use ApplicativeFunctors and play2 functional library (andespecially FunctionalBuilder) This approach is inspired by @sadache (SadekDrobi) article https://gist.github.com/sadache/3646092 An Applicative Functor is a Type Class relying onad-hoc polymorphism to extends a Class with someproperties Play2 functional library (or Scalaz) providesmechanism to compose Applicatives in a smart way 19. tag: step3Applicative functorsM is an Applicative Functor if there exists the following methods :def pure[A](a: A): M[A]def map[A, B](m: M[A], f: A =&gt; B): M[B]def apply[A, B](mf: M[A =&gt; B], ma: M[A]): M[B]!with the following Laws : Identity: apply(pure(identity), ma) === ma where ma is an Applicative M[A] Homomorphism: apply(pure(f), pure(a)) === pure(f(a)) where f: A =&gt;B and a an instance of A Interchange: mf if an instance of M[A =&gt; B]apply(mf, pure(a)) === apply(pure {(g: A =&gt; B) =&gt; g(a)}, mf)! Composition: map(ma, f) === apply(pure(f), ma) 20. tag: step3Applicative functorstrait Applicative[M[_]] {def pure[A](a: A): M[A]def map[A, B](m: M[A], f: A =&gt; B): M[B]def apply[A, B](mf: M[A =&gt; B], ma: M[A]): M[B]} Applicative is an Higher Kinded type(parameterized with M that take a single type parameter) 21. tag: step3Reader is an Applicativecase class Reader[O](p: String =&gt; Result[O]) {def apply(s: String): Result[O] = p(s)!def map[T](f: O =&gt; T): Reader[T] = Reader { s: String =&gt;p(s) match {case Success(o) =&gt; Success(f(o))case f: Failure =&gt; f}}!}!object Reader {def map2[O, O1, O2](r1: Reader[O1], r2: Reader[O2])(f: (O1, O2) =&gt;O): Reader[O] = Reader { s: String =&gt;(r1.p(s), r2.p(s)) match {case (Success(s1), Success(s2)) =&gt; Success(f(s1, s2))case (Success(_), Failure(e)) =&gt; Failure(e)case (Failure(e), Success(_)) =&gt; Failure(e)case (Failure(e1), Failure(e2)) =&gt; Failure(e1 ++ e2)}} !} 22. tag: step3Reader is an Applicativeobject Reader { // map2implicit val readerIsAFunctor: Functor[Reader] = newFunctor[Reader] {override def fmap[A, B](m: Reader[A], f: (A) =&gt; B) = m.map(f)}implicit val readerIsAnApplicative: Applicative[Reader] = newApplicative[Reader] {override def pure[A](a: A) = Reader { _ =&gt; Success(a) }override def apply[A, B](mf: Reader[A =&gt; B], ma: Reader[A]) =map2(mf, ma)((f, a) =&gt; f(a))override def map[A, B](m: Reader[A], f: A =&gt; B) = m.map(f)}} 23. Usage sampletag: step3import Converter.string2StringConverterimport Converter.string2DateConverter!import play.api.libs.functional.syntax._import Reader.readerIsAnApplicative!!implicit val separator = ';'implicit val dtFormatter = new SimpleDateFormat("dd/MM/yyyy")!!val reader = (CsvPicker(1).as[String] andCsvPicker(2).as[Date])(FutureEvent) // here we use CanBuild2.applyreader("foo;bar;12/10/2014") === Success(FutureEvent("bar",dtFormatter.parse("12/10/2014")))! 24. Usage sampletag: step3(errors accumulation)import Converter.string2StringConverterimport Converter.string2DateConverter!import play.api.libs.functional.syntax._import Reader.readerIsAnApplicative!!implicit val separator = ';'implicit val dtFormatter = new SimpleDateFormat("dd/MM/yyyy")!!val reader = (CsvPicker(1).as[Int] andCsvPicker(2).as[Date])((_, _))reader(List("foo", "not a number", "not a date")) === Failure(NEL(!"Unable to format 'not a number' as Int", !"Unable to format 'not a date' as Date"))! 25. Benefitstag: step3 Making Reader an Applicative Functor give abilityto combine efficiently instances of Reader Due to Applicative properties, we still accumulateerrors Play2 functional builder give us a clean syntax todefine our DSL 26. tag: step4Introducing XML Pickercase class Picker(p: String =&gt; Result[String]) {def as[T](implicit c: Converter[T]): Reader[T] = c.convert(p)}!!object XmlPicker {def apply[T](query: Elem =&gt; NodeSeq): Picker = Picker { s: String =&gt;try {val xml = XML.loadString(s)Success(query(xml).text)} catch {case e: Exception =&gt; Failure(e.getMessage)}}}! 27. Usage sampletag: step4import play.api.libs.functional.syntax._import Reader.readerIsAnApplicative!import Converter._implicit val dF = new SimpleDateFormat("dd/MM/yyyy")!val xml = """"""val r = (XmlPicker(_"person""@firstname").as[String] andXmlPicker(_"person""@lastname").as[String] andXmlPicker(_"person""@birthdate").as[Date])(Person)!r(xml) === Success(Person("jean","dupont",dF.parse("11/03/1987")))case class Person(firstname: String, lastname: String, birthDt: Date) 28. tag: step4Implementation problem The Reader[O] takes a type argument for theoutput. The input is always a String With this implementation, an XML content will beparsed (with XML.load) as many times as we useXmlPicker. This will cause unnecessary overhead We will have the same issue (with lower overhead)with our CsvPicker 29. tag: step5Introducing Reader[I, 0]To resolve this problem, we will modify Reader totake a type parameter for the input 30. tag: step5Introducing Reader[I, 0]case class Reader[I, O](p: I =&gt; Result[O]) {def apply(s: I): Result[O] = p(s)def map[T](f: O =&gt; T): Reader[I, T] = Reader { s: I =&gt;p(s) match {case Success(o) =&gt; Success(f(o))case f: Failure =&gt; f}}!}!object Reader {def map2[I, O, O1, O2](r1: Reader[I, O1], r2: Reader[I, O2])(f: (O1,O2) =&gt; O): Reader[I, O] = Reader { s: I =&gt;(r1.p(s), r2.p(s)) match {case (Success(s1), Success(s2)) =&gt; Success(f(s1, s2))case (Success(_), Failure(e)) =&gt; Failure(e)case (Failure(e), Success(_)) =&gt; Failure(e)case (Failure(e1), Failure(e2)) =&gt; Failure(e1 ++ e2)}} 31. tag: step5Introducing Reader[I, 0]object Reader {!implicit def readerIsAFunctor[I] = new Functor[({type [A] =Reader[I, A]})#] {override def fmap[A, B](m: Reader[I, A], f: (A) =&gt; B) = m.map(f)}implicit def readerIsAnApplicative[I] = new Applicative[({type [A] =Reader[I, A]})#] {override def pure[A](a: A) = Reader { _ =&gt; Success(a) }override def apply[A, B](mf: Reader[I, A =&gt; B], ma: Reader[I, A]) =map2(mf, ma)((f, a) =&gt; f(a))override def map[A, B](m: Reader[I, A], f: (A) =&gt; B) = m.map(f)} 32. tag: step5What are Type lambdas ?If we go back to Applicative definition, we can see that itsan Higher Kinded type (same with Functor) :!trait Applicative[M[_]] { } // Applicative acceptparameter M that take itself any type as parameter!!Our problem is that Reader[I, 0] takes two parametersbut Applicative[M[_]] accept types M with only oneparameter. We use Type Lambdas to resolve this issue:!new Applicative[({type [A] = Reader[I, A]})#]! 33. tag: step5Go back to Reader[I, 0]object Reader {!implicit def readerIsAFunctor[I] = new Functor[({type [A] = Reader[I,A]})#] {override def fmap[A, B](m: Reader[I, A], f: A =&gt; B) = m.map(f)}implicit def readerIsAnApplicative[I] = new Applicative[({type [A] =Reader[I, A]})#] {override def pure[A](a: A) = Reader { _ =&gt; Success(a) }override def apply[A, B](mf: Reader[I, A =&gt; B], ma: Reader[I, A]) =map2(mf, ma)((f, a) =&gt; f(a))override def map[A, B](m: Reader[I, A], f: A =&gt; B) = m.map(f)} 34. tag: step5Go back to Reader[I, 0]object Reader {!import scala.language.implicitConversions// Here we help the compiler a bit. Thanks @skaalf (Julien Tournay) !// and https://github.com/jto/validationimplicit def fcbReads[I] = functionalCanBuildApplicative[({type [A] =Reader[I, A]})#]implicit def fboReads[I, A](a: Reader[I, A])(implicit fcb:FunctionalCanBuild[({type [x] = Reader[I, x]})#]) = newFunctionalBuilderOps[({type [x] = Reader[I, x]})#, A](a)(fcb) 35. Converter[I, T]tag: step5trait Converter[I, T] {def convert(p: I =&gt; Result[String]): Reader[I, T]}!object Converter {implicit def stringConverter[I] = new Converter[I, String] {override def convert(p: I =&gt; Result[String]) = Reader[I, String](p)}!!implicit def dateConverter[I](implicit dtFormat: DateFormat) = newConverter[I, Date] {override def convert(p: I =&gt; Result[String]) = Reader[I, Date] { s:I =&gt;p(s) match {case Success(dt) =&gt; try { !Success(dtFormat.parse(dt))} catch { case e: ParseException =&gt; Failure(s"...") }case f: Failure =&gt; f}}} 36. Picker[I]tag: step5case class Picker[I](p: I =&gt; Result[String]) {def as[T](implicit c: Converter[I, T]): Reader[I, T] = c.convert(p)}object CsvPicker {def apply[T](i: Int): Picker[List[String]] = Picker { elems:List[String] =&gt;if (i &gt; 0 &amp;&amp; elems.size &gt; i) Success(elems(i).trim)else Failure(s"No column ${i} found in ${elems.mkString(";")}")}}object XmlPicker {def apply[T](query: Elem =&gt; NodeSeq): Picker[Elem] = Picker { elem:Elem =&gt;try {Success(query(elem).text)} catch {case e: Exception =&gt; Failure(e.getMessage)}}}! 37. Usage sampletag: step5import play.api.libs.functional.syntax._import Reader._!import Converter._!implicit val dF = new SimpleDateFormat("dd/MM/yyyy")!val xml = XML.loadString("""""")!!val r = (XmlPicker(_"person""@firstname").as[String] andXmlPicker(_"person""@lastname").as[String] andXmlPicker(_"person""@birthdate").as[Date])(Person)!r(xml) === Success(Person("jean", "dupont", dF.parse("11/03/1987"))) 38. tag: step6Adding combinators We now add new abilities to Reader We especially want a method to validate content 39. tag: step6Adding combinatorscase class Reader[I, O](p: I =&gt; Result[O]) {!!def flatMap[T](f: O =&gt; Reader[I, T]): Reader[I, T] = Reader { s: I =&gt;p(s) match {case Success(o) =&gt; f(o)(s)case f: Failure =&gt; f}}def verify(f: O =&gt; Result[O]): Reader[I, O] = flatMap { o: O =&gt;Reader( _ =&gt; f(o)) }!} 40. Usage sampletag: step6val r: Reader[String, String] = Reader { Success(_) }r.verify { x =&gt; if (x == "OK") Success(x) else Failure("KO") }("OK")=== Success("OK") 41. Conclusion We have created a simple and powerful DSL forprocessing CSV and XML content This DSL give us ability to Pick data, transform andverify it and also accumulate encountered errors We have seen that making Reader an instance of theApplicative Functor Type Class add it new capabilities Using ad-hoc polymorphism using Type Classes givesus ability to extends Reader w...</p>