Интерпретирование языков с помощью free-монад

Интерпретирование языков с помощью Free-монад

https://twitter.com/ZhekaKozlov

Пример 1: Console I/O

echo ∷ IO () echo = do s ← getLine putStrLn s

Проблемы

Как убедиться, что echo вызывает только функции работы с консолью?

Мы не можем «заглянуть внутрь» IO и

проконтролировать, что не происходят нежелательные эффекты!

Нежелательный эффект

echo ∷ IO () echo = do s ← getLine launchMissile putStrLn s launchMissile ∷ IO () launchMissile = …

Калькулятор

calculator ∷ IO () calculator = do x ← getLine y ← getLine putStrLn (show (read x + read y))

Проблемы

Как протестировать calculator? сalculator возвращает IO() – мы не

можем протестировать ()!

Легко допустить ошибку

calculator ∷ IO () calculator = do x ← getLine y ← getLine putStrLn (show (read x + read x))

Решение?

В этой книге!

1994 год

1994 год

RIP

Шаблон Интепретатор

«… определяет грамматику простого языка, представляет предложения на этом языке и интерпретирует их»

Диаграмма классов

RegularExpression

interpret()

LiteralExpression

interpret()

literal

SequenceExpression

interpret()

expression1

expression2

RepetitionExpression

interpret()

expression

AlternateExpression

interpret()

expression1

expression2

Диаграмма классов

Expr

interpret()

Literal

interpret()

literal

And

interpret()

expression1

expression2

Many

interpret()

expression

Or

interpret()

expression1

expression2

ADT!

data Expr = Literal String | And Expr Expr | Or Expr Expr | Many Expr

Интерпретатор

interpret ∷ String → Expr → (Bool, String) interpret s (Literal lit) | lit `isPrefixOf` s = (True, drop (length lit) s) interpret s (And e1 e2) | (True, tail) ← interpret s e1 = interpret tail e2 interpret s (Or e1 e2) | (ok, tail) ← interpret s e1 = if ok then (True, tail) else interpret s e2 interpret s (Many e) | (ok, tail) ← interpret s e = if ok then interpret tail (Many e) else (True, s) interpret s _ = (False, s)

raining & (dogs | cats)*

And

expression1 expression2

Literal

‘raining’

Or

expression1 expression2

Many

expression

Literal

‘dogs’

Literal

‘cats’


ex = And (Literal "raining") (Many (Or (Literal "dogs") (Literal "cats")))


ex = And (Literal "raining") (Many (Or (Literal "dogs") (Literal "cats"))) > interpret "rainingdogscats1" ex (True,"1") > interpret "abc" ex (False,"abc")

Таким образом

• Шаблон Интерпретатор определяет грамматику:

data Expr = Literal String | And Expr Expr | Or Expr Expr | Many Expr

And (Literal "raining") (Many (Or (Literal "dogs") (Literal "cats")))

• … представляет предложения на этом языке:

• … и интерпретирует их:

interpret = …

Console I/O

Можно ли применить шаблон Интерпретатор для языка консольного ввода-вывода?

Console I/O

Можно ли применить шаблон Интерпретатор для языка консольного ввода-вывода?

Можно

Диаграмма классов ADT

Command

PrintLine

String

ReadLine

data Command = PrintLine String | ReadLine


Command

PrintLine

String Command

ReadLine

data Command = PrintLine String Command | ReadLine (String → Command)

String → Command


Command a

PrintLine

String a

ReadLine

data Command a = PrintLine String a | ReadLine (String → a)

String → a

echo

data Command a = PrintLine String a | ReadLine (String → a) echo = ReadLine (\s → PrintLine s ())

calculator

data Command a = PrintLine String a | ReadLine (String → a) calculator = ReadLine (\x → ReadLine (\y → PrintLine (show (read x + read y)) ()))

Проблема

> :type echo echo ∷ Command (Command ()) > :type calculator calculator ∷ Command (Command (Command ()))

Проблема

> :type echo echo ∷ Command (Command ()) > :type calculator calculator ∷ Command (Command (Command ()))

Как написать интерпретатор для Command (Command (Command (…)))?

Free

data Free f a = Return a | Free (f (Free f a))

echo

data Free f a = Return a | Free (f (Free f a)) echo = Free (ReadLine (\s → Free (PrintLine s (Return ()))))

calculator

data Free f a = Return a | Free (f (Free f a)) calculator = Free (ReadLine (\x -> Free (ReadLine (\y -> Free (PrintLine (show (read x + read y)) (Return ()))))))

Типы

> :type echo echo ∷ Free Command () > :type calculator calculator ∷ Free Command ()

Громоздко

calculator = Free (ReadLine (\x -> Free (ReadLine (\y -> Free (PrintLine (show (read x + read y)) (Return ()))))))

Free f является монадой!

Утверждение: если f является функтором, то Free f является монадой.

data Free f a = Return a | Free (f (Free f a))

Free f является монадой!

Утверждение: если f является функтором, то Free f является монадой.

data Free f a = Return a | Free (f (Free f a)) instance Functor f ⇒ Monad (Free f) where return = Return (Return a) >>= g = g a (Free f) >>= g = Free (fmap (>>= g) f)

Command является функтором

data Command a = PrintLine String a | ReadLine (String → a) deriving Functor


data Command a = PrintLine String a | ReadLine (String → a) deriving Functor type Console = Free Command


data Command a = PrintLine String a | ReadLine (String → a) deriving Functor type Console = Free Command printLine ∷ String → Console () printLine s = Free (PrintLine s (Return ())) readLine ∷ Console String readLine = Free (ReadLine (\s → Return s))

echo

echo ∷ Console () echo = do s ← readLine printLine s

echo

echo ∷ Console () echo = do s ← readLine launchMissile printLine s

Couldn't match type `IO' with `Console'

calculator

calculator ∷ Console () calculator = do x ← readLine y ← readLine printLine (show (read x + read y))

Пишем интерпретатор

runConsole ∷ Console a → IO a runConsole (Return a) = return a runConsole (Free command) = case command of PrintLine s a → do putStrLn s runConsole a ReadLine after → do s ← getLine runConsole (after s)

Запускаем интерпретатор

> runConsole echo Hello Hello > runConsole calculator 3 4 7

2-й интерпретатор (для теста)

Console a

runConsole testConsole

Тест

testConsole ∷ Console a → [String] → [String] testConsole (Return _) _ = [] testConsole (Free command) inputs = case command of PrintLine s a → s : testConsole a inputs ReadLine after → testConsole (after (head inputs)) (tail inputs)

Запускаем тест

> :m + Test.QuickCheck > quickCheck (\s → testConsole echo [s] == [s]) +++ OK, passed 100 tests

Запускаем тест

> quickCheck (\x y → testConsole calculator [show x, show y] == [show (x + y)]) +++ OK, passed 100 tests

3-й интерпретатор (вывод в файл)

Console a

runConsole testConsole

runConsoleFile

Таким образом

• Шаблон Интепретатор определяет грамматику:

data Command a = PrintLine String a | ReadLine (String → a) deriving Functor

do s ← readLine printLine s

• … представляет предложения на этом языке:

• … и интерпретирует их:

runConsole = … testConsole = …

Пример 2: Key-Value Store

data KVSCommand a = Put String String a | Get String (String → a) | Delete String a deriving Functor type KVS = Free KVSCommand


put ∷ String → String → KVS () put k v = Free (Put k v (Return ())) get ∷ String → KVS String get k = Free (Get k Return) del ∷ String → KVS () del k = Free (Delete k (Return ()))


put ∷ String → String → KVS () put k v = Free (Put k v (Return ())) get ∷ String → KVS String get k = Free (Get k Return) del ∷ String → KVS () del k = Free (Delete k (Return ())) modify ∷ String → (String → String) → KVS () modify k f = do v ← get k put k (f v)

Интерпретатор KVS (NoSQL)

runNoSQL ∷ KVS a → Connection → IO a runNoSQL = …

Интерпретатор KVS (NoSQL)

runNoSQL ∷ KVS a → Connection → IO a runNoSQL = … kvs ∷ KVS String kvs = do put "firstName" "John" put "lastName" "Doe" modify "age" (show.(+1).read) get "salary" main = do conn ← getConnection runNoSQL kvs conn

Интерпретатор KVS (Map)

import Data.Map runMap ∷ KVS a → Map String String → Map String String runMap (Return _) map = map runMap (Free f) map = case f of Put k v a → runMap a (insert k v map) Get k after → runMap (after (map ! k)) map Delete k a → runMap a (delete k map)

Эффективный интерпретатор

do put "key1" "value1" put "key2" "value2" put "key3" "value3"

"key1" ,"value1"

"key2" ,"value2"

"key3" ,"value3"

Эффективный интерпретатор

do put "key1" "value1" put "key2" "value2" put "key3" "value3"

"key1" ,"value1"

"key2" ,"value2"

"key3" ,"value3"

["key1" ,"value1", "key2" ,"value2", "key3" ,"value3"]

Пример 3: загрузка твитов

import Data.ByteString data Tweet = Tweet { id ∷ Integer, userName ∷ String, text ∷ String } deriving Show data User = User { name ∷ String, photo ∷ ByteString } deriving Show fetchTweets ∷ String → Int → IO [Tweet] fetchUser ∷ String → IO User

Twitter API

fetchTweets ∷ String → Int → IO [Tweet] fetchTweets "odersky" _ = do print ("Fetching tweets for @odersky") return [Tweet 1 "odersky" "Scala!", Tweet 2 "bos31337" "Haskell!", Tweet 3 "odersky" "Good morning"]

fetchUser ∷ String → IO User fetchUser name = do print ("Fetching user profile for @" ++ name) return $ case name of

"odersky" → User "odersky" (pack [1,2,3]) "bos31337" → User "bos31337" (pack [4,5])

Хотим достать твиты с фото

fetchTweetsWithPhotos ∷ String → Int → IO [(Tweet, ByteString)] fetchTweetsWithPhotos name limit = do tweets ← fetchTweets name limit forM tweets (\t → do user ← fetchUser (userName t) return (t, image user))


fetchTweetsWithPhotos ∷ String → Int → IO [(Tweet, ByteString)] fetchTweetsWithPhotos name limit = do tweets ← fetchTweets name limit forM tweets (\t → do user ← fetchUser (userName t) return (t, image user)) > fetchTweetsWithPhotos "odersky" 10 "Fetching tweets for odersky" "Fetching user profile for odersky" "Fetching user profile for bos31337" "Fetching user profile for odersky"


fetchTweetsWithPhotos ∷ String → Int → IO [(Tweet, ByteString)] fetchTweetsWithPhotos name limit = do tweets ← fetchTweets name limit forM tweets (\t → do user ← fetchUser (userName t) return (t, image user)) > fetchTweetsWithPhotos "odersky" 10 "Fetching tweets for odersky" "Fetching user profile for odersky" "Fetching user profile for bos31337" "Fetching user profile for odersky"

2 запроса!

Решение

data TwitterCommand a = GetTweets String Int ([Tweet] → a) | GetUser String (User → a) deriving Functor type Twitter = Free TwitterCommand

Решение

data TwitterCommand a = GetTweets String Int ([Tweet] → a) | GetUser String (User → a) deriving Functor type Twitter = Free TwitterCommand getTweets ∷ String → Int → Twitter [Tweet] getTweets name limit = Free (GetTweets name limit Return) getUser ∷ String → Twitter User getUser name = Free (GetUser name Return)

Кэширующий интерпретатор

runTwitter ∷ Twitter a → Map String User → IO a runTwitter (Return a) _ = return a runTwitter (Free f) cache = case f of GetTweets s limit after → do tweets ← fetchTweets s limit runTwitter (after tweets) cache GetUser name after → do user ← if member name cache then return (cache ! name) else fetchUser name runTwitter (after user) (insert name user cache)

Кэш фото


getTweetsWithPhotos ∷ String → Int → Twitter [(Tweet, ByteString)] getTweetsWithPhotos name limit = do tweets ← getTweets name limit forM tweets (\t → do user ← getUser (userName t) return (t, photo user)) > runTwitter (getTweetsWithPhotos "odersky" 10) empty "Fetching tweets for odersky" "Fetching user profile for odersky" "Fetching user profile for bos31337"

1 запрос

Ещё

При желании можно написать интерпретатор, который распараллеливает запросы.

Существующие библиотеки

Haskell: • https://github.com/facebook/Haxl • http://hackage.haskell.org/package/CouchDB Scala: • Twitter Stitch (Scala, not open sourced) • https://github.com/MonsantoCo/stoop

https://github.com/facebook/Haxl

https://github.com/facebook/Haxl

http://hackage.haskell.org/package/CouchDB

https://github.com/MonsantoCo/stoop

Вопросы?

Интерпретирование языков с помощью free-монад

Science