tech days paris intoduction f# and collective intelligence

48
F# on the Road to Visual Studio 2010 With thanks to Don Syme, Leon “secretGeek” Bambrick, Chris Smith and the puppies Robert Pickering – LexiFi (ALTi)

Upload: robert-pickering

Post on 10-May-2015

1.987 views

Category:

Technology


0 download

TRANSCRIPT

Page 1: Tech Days Paris Intoduction F# and Collective Intelligence

F# on the Road to Visual Studio 2010

With thanks to Don Syme, Leon “secretGeek” Bambrick, Chris Smith

and the puppies

Robert Pickering – LexiFi (ALTi)

Page 2: Tech Days Paris Intoduction F# and Collective Intelligence

Session Overview

• Why was F# created? Why learn F#?

• A taste of the F# language– Especially the functional side!

• See a complete working F# “Collective Intelligence” application

Page 3: Tech Days Paris Intoduction F# and Collective Intelligence

F#...

Do not be afraid.

Part 1

Page 4: Tech Days Paris Intoduction F# and Collective Intelligence

hello

Page 5: Tech Days Paris Intoduction F# and Collective Intelligence

I’m F#

Page 6: Tech Days Paris Intoduction F# and Collective Intelligence

Friendly, Approachable.

Page 7: Tech Days Paris Intoduction F# and Collective Intelligence
Page 8: Tech Days Paris Intoduction F# and Collective Intelligence

I'll let you in on a secret: I'm doing F# simply because it's lots and lots of fun. In a very broad sense of the word: functional programming is fun, OO programming with F# is fun, watching people use F# is fun.

One of the wonderful things about F# is that you can actually end up working in your domain. In the zone. With F#, you're not necessarily "just" a programmer! You're likely to also be a probabilistic modeller, or an AutoCAD engineer, or a finance engineer, or a symbolic programmer, or one of many other things.

- Don Syme, F#’s creator

Page 9: Tech Days Paris Intoduction F# and Collective Intelligence

F# is unique amongst both imperative and declarative languages in that it is the golden middle road where these two extremes converge. F# takes the best features of both paradigms and tastefully combines them in a highly productive and elegant language that both scientists and developers identify with. F# makes programmers better mathematicians and mathematicians better programmers.

- Eric Meijer, Forward to Expert F#

Page 10: Tech Days Paris Intoduction F# and Collective Intelligence

Julien LaugelHead of Research & DevelopmentEuroPerformance

A Few Words From …

Page 11: Tech Days Paris Intoduction F# and Collective Intelligence

F#: Combining ParadigmsI've been coding in F# lately, for a production task.

F# allows you to move smoothly in your programming style... I start with pure functional code, shift slightly towards an object-oriented style, and in production code, I sometimes have to do some imperative programming.

I can start with a pure idea, and still finish my project with realistic code. You're never disappointed in any phase of the project!

Julien Laugel, Head of Research & Development, EuroPerformance

Page 12: Tech Days Paris Intoduction F# and Collective Intelligence

F#: Influences

OCa

ml

C#/.NETF#

Similar core language

Similar objectmodel

Page 13: Tech Days Paris Intoduction F# and Collective Intelligence

• Current Release: September CTP for VS 2008

• Next Release: as part of VS 2010 beta 1

• Plugin for VS Shell and CodePlex release in VS 2010 timeframe

• More:http://blogs.msdn.com/dsyme/archive/2008/12/10/fsharp-to-ship-as-part-of-visual-studio-2010.aspx

Road Map

Page 14: Tech Days Paris Intoduction F# and Collective Intelligence

Part 2

F# the Language ...

... and the Functional Point of View

Page 15: Tech Days Paris Intoduction F# and Collective Intelligence

Code!//F#open Systemlet a = 2Console.WriteLine a

//C#using System;

namespace ConsoleApplication1{ class Program { static int a() { return 2; } static void Main(string[] args) { Console.WriteLine(a); } }}

Page 16: Tech Days Paris Intoduction F# and Collective Intelligence

More Code!//F#open Systemlet a = 2Console.WriteLine a

//C#using System;

namespace ConsoleApplication1{ class Program { static nt a() { return 2; }

static void Main(string[] args) { Console.WriteLine(a); } }}

More Noise Than Signal!

Page 17: Tech Days Paris Intoduction F# and Collective Intelligence

Let Bindings & Literals

let aString = "Stringy" // a string

Declaration of a binding

Like “var” in C#

An identifier

A literal

A comment

Page 18: Tech Days Paris Intoduction F# and Collective Intelligence

ListsLet binding

The empty list

The empty list

ConcatenateLiteral

// the empty listlet emptyList = []

// adding to a listlet conact = "One" :: []

Page 19: Tech Days Paris Intoduction F# and Collective Intelligence

Lists

let list = 1 :: 2 :: 3 :: []

let list = [1; 2; 3]

These are the same

Page 20: Tech Days Paris Intoduction F# and Collective Intelligence

// a functionlet addTen = fun x -> x + 10

Functions

Let binding

Parameter

Result

Page 21: Tech Days Paris Intoduction F# and Collective Intelligence

Functions

Let binding

Parameter

Result

// shorter versionlet addTen x = x + 10

Page 22: Tech Days Paris Intoduction F# and Collective Intelligence

FunctionsLet binding

Result

Parameters

WhitespaceSensitive

IntermediateValue

// multi parameters and // intermediate resultslet addThenTimesTwo x y = let temp = x + y temp * 2

Page 23: Tech Days Paris Intoduction F# and Collective Intelligence

Pattern Matching

// simple pattern matchinglet matching n = match n with | 1 -> "One" | 2 -> "Two" | _ -> "Other"

Pattern Result

Page 24: Tech Days Paris Intoduction F# and Collective Intelligence

Pattern Matching

// more pattern matching// over listslet rec subOne list = match list with | head :: tail -> head - 1 :: subOne tail | [] -> []

Pattern

Result

Recursive function

Recursive call

// pattern matching// over listslet rec addOne list = match list with | head :: tail -> head + 1 :: addOne tail | [] -> []

Page 25: Tech Days Paris Intoduction F# and Collective Intelligence

DRY!

DRY

(ML is 23 years older than Ruby)

on’teapt

ourself

Page 26: Tech Days Paris Intoduction F# and Collective Intelligence

Functions as Parameters// pattern matching// over listslet rec map func list = match list with | head :: tail -> func head :: map tail | [] -> []

let addOne list = List.map (fun x -> x + 1) list

let subOne list = List.map (fun x -> x + 1) list

Function parameter

Call function

Page 27: Tech Days Paris Intoduction F# and Collective Intelligence

Part 3

A “Collective Intelligence” Example Application

Classifying the Bloggers

Page 28: Tech Days Paris Intoduction F# and Collective Intelligence

Representing a Blog

• A blog can be represented by the frequency that words appear in it

• Most and least common words are cut – words like “the” are ignored– words like “woah” are ignored

Phone Email Book People Affiliate

Heckler Spray 0 0 1 2 0

Shiny Shiny 3 0 0 5 0

Chris Garret on New Media 1 75 12 60 24

Page 29: Tech Days Paris Intoduction F# and Collective Intelligence

Measuring “Closeness” of Blogs

Email Book

People

Heckler Spray

Chris Garret on New Media

Page 30: Tech Days Paris Intoduction F# and Collective Intelligence

Measuring “Closeness” of Blogs

• Pervious slide shows “Euclidean Distance”• While this could be use we use “Pearson

correlation”– “Pearson correlation” corrects for the problem

that some blogs are longer than others

where

Page 31: Tech Days Paris Intoduction F# and Collective Intelligence

Pearson correlation in F#let pearson (wc1: seq<float>) (wc2: seq<float>) = let sum = PSeq.reduce (+) let sum1 = sum wc1 let sum2 = sum wc2 let sumSq1 = sum (Seq.map (fun x -> x * x) wc1) let sumSq2 = sum (Seq.map (fun x -> x * x) wc2) let pSum = sum (Seq.map2 (fun x y -> x * y) wc1 wc2) let len = float (Seq.length wc1) let num = pSum - ((sum1 * sum2) / len) let den = sqrt ((sumSq1 - (sum1 * sum1) / len) * (sumSq2 - (sum2 * sum2) / len)) if den = 0. then 0. else num / den

Page 32: Tech Days Paris Intoduction F# and Collective Intelligence

Hierarchical Clustering

A B

C

D E

A B

C

D E

A B

C

D E

B

E

C

A B

C

D E

A

D

Page 33: Tech Days Paris Intoduction F# and Collective Intelligence

Viewing the Hierarchical Clustering

A

B

C

D

E

Page 34: Tech Days Paris Intoduction F# and Collective Intelligence

THE ALGORITHMHow we implement hierarchical clustering efficiently in F#

Page 35: Tech Days Paris Intoduction F# and Collective Intelligence

Map

Fetch Data Process Data Build Tree

EndStart

Make HTTP Request

Clean DataSplit DataCount WordCreate Master Word List

Calculate Words UsedBuild Tree

Reduce

Page 36: Tech Days Paris Intoduction F# and Collective Intelligence

Map

Fetch Data Process Data Build Tree

EndStart

Reduce

Parallelized by Asynchronous WorkflowsParallelized by the Parallel Task Library

Page 37: Tech Days Paris Intoduction F# and Collective Intelligence

3 Layer Architecture

Data Access – dataAccess.fs

Business Logic – algo.fs

UI – Program.fs

Helpersextentions.fs

Tests – test.fsx

Page 38: Tech Days Paris Intoduction F# and Collective Intelligence

DEMOHierarchical clustering application in action

Page 39: Tech Days Paris Intoduction F# and Collective Intelligence

Application Metrics

• Helper/Extension Code: ~60 lines

• Data Access Code: ~30 lines

• Main algorithm: ~220 lines

• UI code: ~150 lines (F#) / ~40 lines (XAML)

Page 40: Tech Days Paris Intoduction F# and Collective Intelligence

Other Things To Cluster

• Any other kind of text:– Books from project Gutenberg– Twitter!– Messages boards or other web pages

• People in social networks

• Reviewers on Amazon

• Companies based on metrics

Page 41: Tech Days Paris Intoduction F# and Collective Intelligence

Collective IntelligenceBy Toby Segaran

Page 42: Tech Days Paris Intoduction F# and Collective Intelligence

Part 3

End Bit

Page 43: Tech Days Paris Intoduction F# and Collective Intelligence

msdn.microsoft.com/fsharp/

Page 44: Tech Days Paris Intoduction F# and Collective Intelligence

F# Resources• MDSN Resource center: http://msdn.microsoft.com/fsharp/

• User forums: http://cs.hubfs.net/forums

• Blogs (there are lots of others!): • http://blogs.msnd.com/dsyme• http://strangelights.com/blog

• Samples on the web:• http://code.msdn.microsoft.com/fsharpsamples • http://code.google.com/hosting/search?q=label:fsharp• http://codeplex.com/Project/ProjectDirectory.aspx?TagName=F%23

• Source available with the distribution: %ProgramFiles%\FSharp-1.9.6.2\source

Page 45: Tech Days Paris Intoduction F# and Collective Intelligence

Current Books about F#

Page 47: Tech Days Paris Intoduction F# and Collective Intelligence

alt.net.france

Program:• 18 February: Aspectize – Frédéric Fadel –

Winwise, 16 rue Gaillon 75002 Paris• 18 March: TDD – Djamel Zouaoui – chez OCTO

Technology, 50 avenue des champs Elysées• 22 April: Entity Framework – Matthieu MEZIL• More info: http://

groups.google.com/group/parisaltnet

Page 48: Tech Days Paris Intoduction F# and Collective Intelligence

Questions?