CS 294-5: Statistical Natural Language Processing

PEAS: Medical diagnosis systemPerformance measurePatient health, cost, reputationEnvironmentPatients, medical staff, insurers, courtsActuatorsScreen display, emailSensorsKeyboard/mouse

Environment typesPacmanBackgammonDiagnosisTaxiFully or partially observableSingle-agent or multiagentDeterministic or stochasticStatic or dynamicDiscrete or continuousKnown or unknownAgent designThe environment type largely determines the agent designPartially observable => agent requires memory (internal state), takes actions to obtain informationStochastic => agent may have to prepare for contingencies, must pay attention while executing plansMulti-agent => agent may need to behave randomlyStatic => agent has enough time to compute a rational decisionContinuous time => continuously operating controller

Agent types

In order of increasing generality and complexitySimple reflex agentsReflex agents with stateGoal-based agentsUtility-based agentsSimple reflex agents

A reflex Pacman agent in Pythonclass TurnLeftAgent(Agent): def getAction(self, percept): legal = percept.getLegalPacmanActions() current = percept.getPacmanState().configuration.direction if current == Directions.STOP: current = Directions.NORTH left = Directions.LEFT[current] if left in legal: return left if current in legal: return current if Directions.RIGHT[current] in legal: return Directions.RIGHT[current] if Directions.LEFT[left] in legal: return Directions.LEFT[left] return Directions.STOP

Pacman agent contd.Can we (in principle) extend this reflex agent to behave well in all standard Pacman environments?Handling complexityWriting behavioral rules or environment models more difficult for more complex environmentsE.g., rules of chess (32 pieces, 64 squares, ~100 moves) ~100 000 000 000 000 000 000 000 000 000 000 000 000 pages as a state-to-state transition matrix (cf HMMs, automata) R.B.KB.RPPP..PPP..N..N..PP.q.pp..Q..n..n..ppp..pppr.b.kb.r~100 000 pages in propositional logic (cf circuits, graphical models) WhiteKingOnC4@Move12 1 page in first-order logic x,y,t,color,piece On(color,piece,x,y,t)

Reflex agents with state

Goal-based agents

Utility-based agents

SummaryAn agent interacts with an environment through sensors and actuatorsThe agent function, implemented by an agent program running on a machine, describes what the agent does in all circumstances PEAS descriptions define task environments; precise PEAS specifications are essential More difficult environments require more complex agent designs and more sophisticated representations

CS 188: Artificial Intelligence

SearchInstructor: Stuart Russell]Please retain proper attribution and the reference to ai.berkeley.edu. Thanks!14

TodayAgents that Plan Ahead

Search Problems

Uninformed Search Methods

Depth-First Search

Breadth-First Search

Uniform-Cost Search

General theme in the class:

A goal we have in mind.

A mathematical abstraction to formalize this

Algorithms that operate on these abstractions15Agents that plan aheadPlanning agents:Decisions based on predicted consequences of actionsMust have a transition model: how the world evolves in response to actionsMust formulate a goal

Spectrum of deliberativeness:Generate complete, optimal plan offline, then executeGenerate a simple, greedy plan, start executing, replan when something goes wrong

Video of Demo Replanning

Video of Demo Mastermind

Search Problems

Search ProblemsA search problem consists of:

A state space

For each state, a set Actions(s) of allowable actions

A transition model Result(s,a)

A step cost function c(s,a,s)

A start state and a goal test

A solution is a sequence of actions (a plan) which transforms the start state to a goal state

NE

{N, E}11Often comes back on exams

Goal test sometimes more than one state that satisfies having achieved the goal, for example, eat all the dots

Abstraction20Search Problems Are Models

Example: Travelling in RomaniaState space:CitiesActions:Go to adjacent cityTransition modelResult(Go(B),A) = BStep costDistance along road link Start state:AradGoal test:Is state == Bucharest?Solution?

Infinitely many solutions! Some better than others22Whats in a State Space?Problem: PathingStates: (x,y) locationActions: NSEWTransition model: update locationGoal test: is (x,y)=ENDProblem: Eat-All-DotsStates: {(x,y), dot booleans}Actions: NSEWTransition model: update location and possibly a dot booleanGoal test: dots all falseThe real world state includes every last detail of the environmentA search state abstracts away details not needed to solve the problem

MN statesMN2MN statesWrong example for eat-all-dots: (x, y, dot count)23Quiz: Safe PassageProblem: eat all dots while keeping the ghosts perma-scaredWhat does the state space have to specify?(agent position, dot booleans, power pellet booleans, remaining scared time)

State Space Graphs and Search Trees

State Space GraphsState space graph: A mathematical representation of a search problemNodes are (abstracted) world configurationsArcs represent transitions resulting from actionsThe goal test is a set of goal nodes (maybe only one)

In a state space graph, each state occurs only once!

We can rarely build this full graph in memory (its too big), but its a useful idea

More examples

Typical finite graph explicitly defined (theoretical CS)27More examples

Typical exponential (or infinite) graph implicitly defined (AI)28Search TreesA search tree:A what if tree of plans and their outcomesThe start state is the root nodeChildren correspond to possible action outcomesNodes show states, but correspond to PLANS that achieve those statesFor most problems, we can never actually build the whole tree

E, 1.0N, 1.0This is now / startPossible futuresDifferent plans that achieve the same state, will be different nodes in the tree.29State Space Graphs vs. Search Trees

Quiz: State Space Graphs vs. Search TreesSGbaConsider this 4-state graph: Important: Lots of repeated structure in the search tree!How big is its search tree (from S)?

SabGGabGaGbTree Search

Search Example: Romania

Searching with a Search TreeSearch:Expand out potential plans (tree nodes)Maintain a frontier of partial plans under considerationTry to expand as few tree nodes as possible

General Tree SearchImportant ideas:FrontierExpansionExploration strategy

Main question: which frontier nodes to explore?function TREE-SEARCH(problem) returns a solution, or failure initialize the frontier using the initial state of problem loop do if the frontier is empty then return failure choose a leaf node and remove it from the frontier if the node contains a goal state then return the corresponding solution expand the chosen node, adding the resulting nodes to the frontier

Depth-First SearchDepth-First SearchSabdpacephfrqqcGaqephfrqqcGaSGdbpqcehafrqphfdbacerStrategy: expand a deepest node firstImplementation: Frontier is a LIFO stackSearch Algorithm Properties

Search Algorithm PropertiesComplete: Guaranteed to find a solution if one exists?Optimal: Guaranteed to find the least cost path?Time complexity?Space complexity?

Cartoon of search tree:b is the branching factorm is the maximum depthsolutions at various depths

Number of nodes in entire tree?1 + b + b2 + . bm = O(bm)

b1 nodeb nodesb2 nodesbm nodesm tiersDepth-First Search (DFS) Propertiesb1 nodeb nodesb2 nodesbm nodesm tiersWhat nodes does DFS expand?Some left prefix of the tree.Could process the whole tree!If m is finite, takes time O(bm)

How much space does the frontier take?Only has siblings on path to root, so O(bm)

Is it complete?m could be infinite, so only if we prevent cycles (more later)

Is it optimal?No, it finds the leftmost solution, regardless of depth or cost

Breadth-First Search

Breadth-First SearchSabdpacephfrqqcGaqephfrqqcGaSGdbpqcehafrSearchTiersStrategy: expand a shallowest node firstImplementation: Frontier is a FIFO queueBreadth-First Search (BFS) PropertiesWhat nodes does BFS expand?Processes all nodes above shallowest solutionLet depth of shallowest solution be sSearch takes time O(bs)

How much space does the frontier take?Has roughly the last tier, so O(bs)

Is it complete?s must be finite if a solution exists, so yes!

Is it optimal?Only if costs are all 1 (more on costs later)

b1 nodeb nodesb2 nodesbm nodess tiersbs nodesQuiz: DFS vs BFS

Quiz: DFS vs BFSWhen will BFS outperform DFS?

When will DFS outperform BFS?[Demo: dfs/bfs maze water (L2D6)]Video of Demo Maze Water DFS/BFS (part 1)

Video of Demo Maze Water DFS/BFS (part 2)

Iterative DeepeningbIdea: get DFSs space advantage with BFSs time / shallow-solution advantagesRun a DFS with depth limit 1. If no solutionRun a DFS with depth limit 2. If no solutionRun a DFS with depth limit 3. ..

Isnt that wastefully redundant?Generally most work happens in the lowest level searched, so not so bad!Finding a least-cost pathBFS finds the shortest path in terms of number of actions.It does not find the least-cost path. We will now covera similar algorithm which does find the least-cost path. STARTGOALdbpqcehafr29281823244151322Uniform Cost Search

Uniform Cost SearchSabdpacephfrqqcGaqephfrqqcGaStrategy: expand a cheapest node first:Frontier is a priority queue (priority: cumulative cost)SGdbpqcehafr39116411571381011171106391128821512Cost contours2Uniform Cost Search (UCS) PropertiesWhat nodes does UCS expand?Processes all nodes with cost less than cheapest solution!If that solution costs C* and arcs cost at least , then the effective depth is roughly C*/Takes time O(bC*/) (exponential in effective depth)

How much space does the frontier take?Has roughly the last tier, so O(bC*/)

Is it complete?Assuming best solution has a finite cost and minimum arc cost is positive, yes!

Is it optimal?Yes! (Proof next lecture via A*)

bC*/ tiersc 3c 2c 1Uniform Cost IssuesRemember: UCS explores increasing cost contours

The good: UCS is complete and optimal!

The bad:Explores options in every directionNo information about goal location

Well fix that soon!

StartGoalc 3c 2c 153Video of Demo Empty UCS

Video of Demo Maze with Deep/Shallow Water --- DFS, BFS, or UCS? (part 1)

Let students guess which one of the three it is. (This is BFS.)55Video of Demo Maze with Deep/Shallow Water --- DFS, BFS, or UCS? (part 2)

Let students guess which one of the three it is. (This is UCS.)

56Video of Demo Maze with Deep/Shallow Water --- DFS, BFS, or UCS? (part 3)

Let students guess which one of the three it is. (This is DFS.)

57Search Gone Wrong?