1 midterm review. 2 topic 1 - agents agents and environments rationality peas (performance...
TRANSCRIPT
1
Midterm Review
2
Topic 1 - Agents
Agents and environments Rationality PEAS (Performance measure, Environment,
Actuators, Sensors) Environment types Agent types
3
PEAS: Specifying an automated taxi driver
Performance measure: • safe, fast, legal, comfortable, maximize profits
Environment: • roads, other traffic, pedestrians, customers
Actuators: • steering, accelerator, brake, signal, horn
Sensors: • cameras, sonar, speedometer, GPS
4
Environment types: Definitions I Fully observable (vs. partially observable): An agent's sensors
give it access to the complete state of the environment at each point in time.
Deterministic (vs. stochastic): The next state of the environment is completely determined by the current state and the action executed by the agent.
• (If the environment is deterministic except for the actions of other agents, then the environment is strategic)
Static (vs. dynamic): The environment is unchanged while an agent is deliberating. • The environment is semidynamic if the environment itself does not
change with the passage of time but the agent's performance score does.
Discrete (vs. continuous): A limited number of distinct, clearly defined percepts and actions.
Single agent (vs. multiagent): An agent operating by itself in an environment.
5
Agent types
Four basic types in order of increasing generality: Simple reflex agents Model-based reflex agents Goal-based agents Utility-based agents
Learning agents incorporate one of the above
6
Topic 2 - Python
Reference Semantics, Names & Assignment Containers: Lists, Tuples, Dictionaries Mutability List Comprehensions Flow of Control: For loops Importing & Modules Defining new classes & methods; inheritance
7
Topic 3 – Uninformed Search
Problem formulation Basic uninformed search algorithms
• Breadth-first search• Depth-first search• Iterative-deepening search
8
Problem formulation A problem is defined by:
• An initial state, e.g. Arad• Successor function S(X)= set of action-state pairs
—e.g. S(Arad)={<Arad Zerind, Zerind>,…}
—initial state + successor function* = state space
• Goal test, can be—Explicit, e.g. x=‘at bucharest’—Implicit, e.g. checkmate(x)
• Path cost (additive)—e.g. sum of distances, number of actions executed, …—c(x,a,y) is the step cost, assumed to be >= 0
A solution is a sequence of actions from initial to goal state.
Optimal solution has the lowest path cost.
9
Formulating a Search Problem
Decide:
Which properties matter & how to represent• Initial State, Goal State, Possible Intermediate States
Which actions are possible & how to represent• Operator Set
Which action is next• Path Cost Function
10
Basic search algorithms: Tree Search How do we find the solutions of previous problems?
• Search the state space through explicit tree generation— ROOT= initial state.— Nodes and leafs generated through successor function.
function TREE-SEARCH(problem,fringe) return a solution or failurefringe INSERT(MAKE-NODE(INITIAL-STATE[problem]), fringe)loop do
if EMPTY?(fringe) then return failurenode PICK-NEXT-NODE(fringe, STRATEGY(problem))if GOAL-TEST[problem] applied to STATE[node] succeeds
then return SOLUTION(node)fringe INSERT-ALL(EXPAND(node, problem), fringe)
A strategy is defined by picking the order of node expansion
11
Evaluating Search Strategies Strategy = order of expansion
Dimensions for evaluation• Completeness- always find the solution?
• Time complexity - # of nodes generated
• Space complexity - # of nodes in memory
• Optimality - find least cost solution?
Time/space complexity measurements• b, maximum branching factor of search tree
• d, depth of the least cost solution
• m, maximum depth of the state space ()
12
Breadth-first search
Idea: • Expand shallowest unexpanded node
Implementation:• Fringe is FIFO Queue:
—Put successors at the end of fringe successor list.
13
Properties of breadth-first search
Complete? Yes (if b is finite) Time? 1+b+b2+b3+… +bd + b(bd-1) = O(bd+1) Space? O(bd+1) (keeps every node in memory) Optimal? Yes (if cost = 1 per step)
(not optimal in general)
b: maximum branching factor of search tree
d: depth of the least cost solution
m: maximum depth of the state space ()
14
Depth-first search
Idea: • Expand deepest unexpanded node
Implementation:• Fringe is LIFO Queue:
—Put successors at the front of fringe successor list.
15
Properties of depth-first search
Complete? No: fails in infinite-depth spaces, spaces with loops• Modify to avoid repeated states along path
complete in finite spaces
Time? O(bm): terrible if m is much larger than d• but if solutions are dense, may be much faster than breadth-first
Space? O(bm), i.e., linear space!
Optimal? No
b: maximum branching factor of search treed: depth of the least cost solution
m: maximum depth of the state space ()
16
Depth-first vs Breadth-first Use depth-first if
• Space is restricted
• There are many possible solutions with long paths and wrong paths can be detected quickly
• Search can be fine-tuned quickly
Use breadth-first if• Possible infinite paths
• Some solutions have short paths
• Can quickly discard unlikely paths
17
Iterative deepening search
A general strategy to find best depth limit l.• Complete: Goal is always found at depth d, the
depth of the shallowest goal-node.
• Often used in combination with DF-search
Combines benefits of DF-search & BF-search
18
ID search, Evaluation III
Complete: YES (no infinite paths)
Time complexity:
Space complexity:
Optimal: YES if step cost is 1.
O(bd )
O(bd)
19
Calculating path cost, g(N) All paths are often not equal
Simplest cost: All moves equal in cost• Cost = # of nodes in path-1• g(N) = depth(N)
Expand lowest cost node first = breadth first
Assign unique cost to each step• N0, N1, N2, N3 = nodes visited• C(i,j): Cost of going from Ni to Nj
• g(N3)=C(0,1)+C(1,2)+C(2,3)
20
Uniform-cost search (UCS) Extension of BF-search:
• Expand node with lowest path cost
Really (generalized) breadth first• AIMA calls this (strangely) uniform cost search
— And so will we….
Implementation: fringe = queue ordered by path cost.
Again: UC-search is the same as BF-search when all step-costs are equal.
21
Summary of algorithms (for notes)
Criterion Breadth-First
Uniform-cost
Depth-First
Depth-limited
Iterative deepening
Bidirectional search
Complete?
YES* YES* NO YES, if l
d
YES YES*
Time bd+1 bC*/e bm bl bd bd/2
Space bd+1 bC*/e bm bl bd bd/2
Optimal? YES* YES* NO NO YES YES
22
Topic 4 - Informed SearchPART I Informed = use problem-specific knowledge Best-first search and its variants A* - Optimal Search using Knowledge
Proof of Optimality of A*
Heuristic functions?• How to invent them
PART II Local search and optimization
• Hill climbing, local beam search, genetic algorithms,… Local search in continuous spaces Online search agents
23
Heuristic functions & Greedy best-first search
Heuristic Functions
Let evaluation function f(n) = h(n) (heuristic)• h(n) = estimated cost of the cheapest path from
node n to goal node.• If n is goal then h(n)=0
Greedy Best-First Search
Expands the node that appears to be closest to goal Expands nodes based on f(n) = h(n)
• Some estimate of cost from n to goal For example, h(n) = hSLD(n) = straight-line distance from n to
goal
24
Properties of greedy best-first search
Complete? • No – can get stuck in loops,
Time? • O(bm) – worst case (like Depth First Search)• But a good heuristic can give dramatic improvement
Space? • O(bm) – keeps all nodes in memory
Optimal? No
25
A* search
Best-known form of best-first search. Idea: avoid expanding paths that are already
expensive. Evaluation function f(n)=g(n) + h(n)
• g(n) the cost (so far) to reach the node
• h(n) estimated cost to get from the node to the goal
• f(n) estimated total cost of path through n to goal
Implementation: Expand the node n with minimum f(n)
26
Admissible heuristics
For A* to be optimal,h(n) must be an admissible heuristic • A heuristic is admissible if it never overestimates
the cost to reach the goal; i.e. it is optimistic• Formally, n, where n is a node:
1. h(n) <= h*(n) where h*(n) is the true cost from n
2. h(n) >= 0 so h(G)=0 for any goal G.
• Example:
hSLD(n) never overestimates the actual road distance
Theorem: If h(n) is admissible, A* using Tree Search is optimal
27
A* search, evaluation Completeness: YES Time complexity: (exponential with path length) Space complexity:(all nodes are stored) Optimality: YES, if h(n) is admissible
• Cannot expand fi+1 until fi is finished.
• A* expands all nodes with f(n)< f(G)
• A* expands one node with f(n)=f(G)
• A* expands no nodes with f(n)>f(G)
Also optimally efficient (not including ties)
28
Optimality of A* using Tree-Search I (proof)
Suppose some suboptimal goal G2 has been generated and is in the fringe. Let n be an unexpanded node in the fringe such that n is on a shortest path to an optimal goal G.
1. g(G2) > g(G) since G2 is suboptimal
2. f(G2) = g(G2) since h(G2) = 0 3. f(G) = g(G) since h(G) = 0
4. f(G2) > f(G) from 1,2,3
29
Optimality of A* using Tree-Search II (proof)
Suppose some suboptimal goal G2 has been generated and is in thefringe. Let n be an unexpanded node in the fringe such that n is on ashortest path to an optimal goal G.
4. f(G2) > f(G) repeated 5. h*(n) h(n) since h is admissible, and therefore optimistic6. g(n) + h*(n) g(n) + h(n) from 57. g(n) + h*(n) f(n) substituting definition of f8. f(G) f(n) from definitions of f and h* (think about it!)
9. f(G2) >f(n) from 4 & 8
So A* will never select G2 for expansion
30
Defining Heuristics & Relaxed Problems
Cost of an exact solution to a relaxed problem (fewer restrictions on operator)
A tile can move from square A to square B if A is adjacent to B and B is blank.• A tile can move from square A to square B if A is adjacent to B.
(hmd)
• A tile can move from square A to square B if B is blank.
• A tile can move from square A to square B. (hoop)
31
Hill-climbing searchI. While ( uphill points):
• Move in the direction of increasing value, lessening distance to goal Properties:
• Terminates when a peak is reached.• Does not look ahead of the immediate neighbors of the current state.• Chooses randomly among the set of best successors, if there is more than one.• Doesn’t backtrack, since it doesn’t remember where it’s been
a.k.a. greedy local search"Like climbing Everest in thick fog with amnesia“
Problems:• Local Maxima (foothills)• Plateaus• Ridges
Remedies:• Random restart• Problem reformulation• In the end: Some problem spaces are great for hill climbing and others
are terrible.
32
Topic 5: Adversarial Search & Games
Part I Motivation Game Trees Evaluation Functions
Part II The Minimax Rule Alpha-Beta Pruning Game-playing AI successes
33
Game setup Two players: A and B A moves first and they take turns until the game is over.
Winner gets award, loser gets penalty. Games as search:
• Initial state: e.g. board configuration of chess• Successor function: list of (move,state) pairs specifying legal
moves.• Terminal test: Is the game finished?• Utility function: Gives numerical value of terminal states. E.g.
win (+1), lose (-1) and draw (0) in tic-tac-toe A uses search tree to determine next move.
34
How to Play a Game by Searching
General Scheme• Consider all legal moves, each of which will lead to some new state
of the environment (‘board position’)
• Evaluate each possible resulting board position
• Pick the move which leads to the best board position.
• Wait for your opponent’s move, then repeat.
Key problems• Representing the ‘board’
• Representing legal next boards
• Evaluating positions
• Looking ahead
35
Game Trees Represent the problem space for a game by a tree Nodes represent ‘board positions’; edges represent legal
moves. Root node is the position in which a decision must be
made. Evaluation function f assigns real-number scores to `board
positions.’ Terminal nodes represent ways the game could end,
labeled with the desirability of that ending (e.g. win/lose/draw or a numerical score)
36
MAX & MIN Nodes When I move, I attempt to MAXimize my performance. When my opponent moves, he or she attempts to MINimize
my performance.
TO REPRESENT THIS:
If we move first, label the root MAX; if our opponent does, label it MIN.
Alternate labels for each successive tree level.• if the root (level 0) is our turn (MAX), all even levels will
represent turns for us (MAX), and all odd ones turns for our opponent (MIN).
37
Evaluation functions
Evaluations how good a ‘board position’ is
• Based on static features of that board alone
Zero-sum assumption lets us use one function to
describe goodness for both players.
• f(n)>0 if we are winning in position n
• f(n)=0 if position n is tied
• f(n)<0 if our opponent is winning in position n
Build using expert knowledge, • Tic-tac-toe: f(n)=(# of 3 lengths open for me)
- (# open for you)
38
The Minimax Procedure
1. Start with the current position as a MAX node.
2. Expand the game tree a fixed number of ply (half-moves).
3. Apply the evaluation function to the leaf positions.
4. Calculate back-up up values bottom-up.
5. Pick the move which was chosen to give the MAX value at the root.
39
Alpha-Beta Pruning Traverse the search tree in depth-first order For each MAX node n, α(n)=maximum child value found so
far• Starts with –• Increases if a child returns a value greater than the current α(n)• Lower-bound on the final value
For each MIN node n, β(n)=minimum child value found so far• Starts with +• Decreases if a child returns a value less than the current β(n)• Upper-bound on the final value
MAX cutoff rule: At a MAX node n, cut off search if α(n)>=β(n)
MIN cutoff rule: At a MIN node n, cut off search if β(n)<=α(n) Carry α and β values down in search
40
Effectiveness of Alpha-Beta Pruning
Guaranteed to compute same root value as Minimax
Worst case: no pruning, same as Minimax (O(bd)) Best case: when each player’s best move is the
first option examined, you examine only O(bd/2) nodes, allowing you to search twice as deep!
For Deep Blue, alpha-beta pruning reduced the average branching factor from 35-40 to 6.
41CIS 391 - Intro to AI
41
Topic 6:Interpreting Line Drawings(An Introduction to Constraint Satisfaction)
42 42
Convexity Labeling ConventionsEach edge in an image can be interpreted to be either a convex
edge, a concave edge or an occluding edge:
+ labels a convex edge (angled toward the viewer); - labels a concave edge (angled away from the viewer); labels an occluding edge. To its right is the body for
which the arrow line provides an edge. On its left is space.
convex concaveoccluding
43CIS 391 - Intro to AI
43
“Arrow” Junctions have 3 Interpretations “L” Junctions have 6
44CIS 391 - Intro to AI
44
“T” Junctions Have 4 Interpretations, “Y” Junctions Have 5
+ -
45CIS 391 - Intro to AI
45
The Edge Consistency Constraint
Any consistent assignment of labels to the junctions in a picture must assign the same line label to any given line.
L4
+
+
L3 ...
...
+
-
L3 ...
...
L5
46CIS 391 - Intro to AI
46
Search Trees for Line Labelling
Implementation by search tree:1. Select some junction as the root.2. Label children at the 1st level with all possible
interpretations of that junction. 3. Label their children with possible consistent
interpretations of some junction adjacent to that junction.
4. Each level of the tree adds one more labeled node to the growing interpretation.
5. Leaves represent either futile interpretations that cannot be continued or full interpretations of the line drawing.
47CIS 391 - Intro to AI
47
The Top Three Levels of a Search Tree
A1 A3A2
A1L1 A1L5 A2L3 A3L6
48
Constraint Propagation
Waltz’s insight: Pairs of adjacent junctions (junctions connected
by a line) constrain each other’s interpretations! These constraints can propagate along the
connected edges of the graph.
Waltz Filtering: Suppose junctions i and j are connected by an
edge. Remove any labeling from i that is inconsistent with every labeling in the set assigned in j.
By iterating over the graph, the constraints will propagate.
49
The Waltz/Mackworth Constraint Propagation Algorithm
1. Associate with each junction in the picture a set of all Huffman/Clowes junction labels appropriate for that junction type;
2. Repeat until there is no change in the set of labels associate with any junction:
3. For each junction i in the picture:
4. For each neighboring junction j in the picture:
5. Remove any junction label from i for which there is no edge-consistent junction label on j.
50
An Example of Constraint Propagation
A1,A2,A3
L1,L2,L3,L4,L5,
L6
A1,A2, A3
A1,A2, A3
L1,L3,L5,L6
A1,A3
A1,A2, A3
L1,L3,L5,L6
A1,A2,A3
A1,A2,A3
L1, L5,L6
A1,A3
A1, A3L1, L5,L6
A1,A3
…
Check
Given
51
Arcs
Given this notion of (i,j) junction pairs, The crucial notion is an arc, where an arc is a
directed pair of neighboring junctions. For a neighboring pair of junctions i and j, there
are two arcs that connect them: (i,j) and (j,i).
i j
(i,j)
(j,i)
52
Arc ConsistencyWe will define the algorithm for line labeling with respect to
arc consistency as opposed to edge consistency:
An arc (i,j) is arc consistent if and only if every label on junction i is consistent with some label on junction j.
To make an arc (i,j) arc consistent, for each label l on junction i,
if there is no label on j consistent with l then
remove l from i
(Any consistent assignment of labels to the junctions in a picture must assign the same line label to an arc (i,j) as to the inverse arc (j,i) ).
53
The AC-3 Algorithm (for Line Labeling)
1. Associate with each junction in the picture a set of all Huffman/Clowes junction labels for that junction type;
2. Make a queue of all the arcs (i,j) in the picture;
3. Repeat until the queue is empty:a. Remove an arc (i,j) from the queue;
b. For each label l on junction i, if there is no label on j consistent with l then
remove l from i
c. If any label is removed from i then for all arcs (k,i) in the picture except (j,i),
put (k,i) on the queue if it isn’t already there.
54
AC-3: Worst Case Complexity Analysis All junctions can be connected to every other junction,
• so each of n junctions must be compared against n-1 other junctions,
• so total # of arcs is n*(n-1), i.e. O(n2)
If there are d labels, checking arc (i,j) takes O(d2) time Each arc (i,j) can only be inserted into the queue d times Worst case complexity: O(n2d3)
For planar pictures, where lines can never cross, the number of arcs can only be linear in N, so for our pictures, the time complexity is only O(nd3)
55
Topic 7:Constraint Satisfaction Problems
56
Constraint satisfaction problems
What is a CSP?• Finite set of variables X1, X2, …, Xn
• Nonempty domain of possible values for each variable D1, D2, … Dd
• Finite set of constraints C1, C2, …, Cm
• Each constraint Ci limits the values that variables can take, e.g., X1 ≠ X2
A state is defined as an assignment of values to some or all variables.
Consistent assignment: assignment does not violate the constraints.
57
Constraint satisfaction problems
An assignment is complete when every variable is assigned a value.
A solution to a CSP is a complete assignment that satisfies all constraints.
Some CSPs require a solution that maximizes an objective function.
Applications: • Scheduling problems
—Job shop scheduling —Scheduling the Hubble Space Telescope
• Floor planning for VLSI • Map coloring
58
Constraint graph
Binary CSP: each constraint relates two variables
Constraint graph: • nodes are variables• arcs are constraints
CSP benefits• Standard representation pattern: variables with
values• Generic goal and successor functions• Generic heuristics (no domain specific expertise).
Graph can be used to simplify search.—e.g. Tasmania is an independent subproblem.
WA NT
WA SA
NT
S
ANT Q
59
CSP as a standard search problem
A CSP can easily expressed as a standard search problem, as we have seen• Initial State: the empty assignment {}.• Successor function: Assign value to unassigned
variable provided that there is not conflict.• Goal test: the current assignment is complete.• Path cost: a constant cost for every step.
Solution is found at depth n , for n variables• Hence depth first search can be used
60
Backtracking search Note that variable assignments are commutative
• Eg [ step 1: WA = red; step 2: NT = green ] equivalent to [ step 1: NT = green; step 2: WA = red ]
Only need to consider assignments to a single variable at each node b = d and there are dn leaves
Depth-first search for CSPs with single-variable assignments is called backtracking search
Backtracking search is the basic uninformed algorithm for CSPs
Can solve n-queens for n ≈ 25
•
61
Improving backtracking efficiency
General-purpose methods can give huge gains in speed:• Which variable should be assigned next?• In what order should its values be tried?• Can we detect inevitable failure early?
Heuristics:1. Most constrained variable
2. Most constraining variable
3. Least constraining value
4. Forward checking
62
Constraint propagation Forward checking propagates information from assigned to
unassigned variables, but doesn't provide early detection for all failures:
NT and SA cannot both be blue! Constraint propagation repeatedly enforces constraints locally
63
Arc consistency: The General Case Simplest form of propagation makes each arc consistent X Y is consistent iff
for every value x of X there is some allowed y
64
function AC-3(csp) return the CSP, possibly with reduced domains
inputs: csp, a binary csp with variables {X1, X2, …, Xn}
local variables: queue, a queue of arcs initially the arcs in csp
while queue is not empty do
(Xi, Xj) REMOVE-FIRST(queue)
if REMOVE-INCONSISTENT-VALUES(Xi, Xj) then
for each Xk in NEIGHBORS[Xi ] do
add (Xk, Xj) to queue
function REMOVE-INCONSISTENT-VALUES(Xi, Xj) return true iff we remove a value
removed false
for each x in DOMAIN[Xi] do
if no value y in DOMAIN[Xi] allows (x,y) to satisfy the constraints between Xi and Xj
then delete x from DOMAIN[Xi]; removed true
return removed
General version of AC-3
Time complexity: O(n2d3)
Typo on your version fixed
65
Local search for CSPs Hill-climbing, simulated annealing typically work
with "complete" states, i.e., all variables assigned
To apply to CSPs:• allow states with unsatisfied constraints• operators reassign variable values
Variable selection: randomly select any conflicted variable
Value selection by min-conflicts heuristic:• choose value that violates the fewest constraints• i.e., hill-climb with h(n) = total number of violated constraints
66
Example: n-queens States: 4 queens in 4 columns (44 = 256 states) Actions: move queen in column Goal test: no attacks Evaluation: h(n) = number of attacks
Given random initial state, min-conflicts can solve n-queens in almost constant time for arbitrary n with high probability (e.g., n = 10,000,000)