AI - Introduction to Dynamic Programming

AI - Introduction to Dynamic Programming

• 1.
  Introduction to DynamicProgramming Portland Data Science Group Created by Andrew Ferlitsch Community Outreach Officer June, 2017
• 2.
  What is it? •A method of solving complex problems. • Steps: • Solve sub-problems. • Store (Remember) solutions to sub-problems. • Reuse Stored Solutions. Complex Problem Sub-Problem X Sub-Problem Y Sub-Problem X Solution X Solution Y Solution Storage Solve problem by decomposing into sub-problems X, Y and X again. Store solution to X Reuse solution X when sub-problem X Is encountered again.
• 3.
  How is thisdifferent from Functions? • In Traditional Programming the solutions to sub-problems is known. • Steps: • Programmer decomposes the problem into sub-problems. • Designs and Codes solution to sub-problem. • Encapsulates solution to sub-problem in a function. In Dynamic Programming, a Solution is Discovered (not Coded!).
• 4.
  Traditional Design &Coded Program Complex Problem Sub-Problem X Sub-Problem Y Sub-Problem X Function X Function Y Solutions to sub-problems were pre-designed and coded, and stored as re-usable functions (components). Pre-designed solution reused. Run-time program execution Flow. The solutions are already known.
• 5.
  Least Coin Problem– Traditional Solution 1 5 10 25 Problem: For any money amount, calculate the least number of coins to carry in your pocket. $.08 = 5 + 1 + 1 + 1 = 4 coins $.23 = 10 + 10 + 1 + 1 + 1 = 5 coins sum = 0 number_of_coins = 0 while sum < amount: coin = largest_coin(amount – sum) sum = sum + coin number_of_coins = number_of_coins + 1 Solution can be pre-designed and code because the coins are multiples of each other.
• 6.
  Least Coin Problem– Coins not Multiples 1 4 5 15 $.08 = 4 + 4 = 2 coins (not 5 + 1 + 1 +1 = 4 coins) $.23 = 15 + 4 + 4 = 3 coins (not 15 + 5 + 1 + 1 + 1 = 5 coins) If we had used the previous pre-designed/coded solution, we would have gotten the wrong answer!
• 7.
  Discover Solution usingBFS Search Tree 1 4 5 Root of Search Tree sum = 4 First Level of Search Tree, Possible Coin choices <= $.08 coins = BFS( $.08, [ 15, 5, 4, 1 ] ) No node for 15 cent coin because > .08
• 8.
  Discover Solution usingBFS Search Tree 1 4 5 Root of Search Tree sum = 4 coins = BFS( $.08, [ 15, 5, 4, 1 ] ) 1 4 5 sum = 5 1 4 No node for 15 cent coin because > .07 No node for 15 and 5 cent coin because > .04 Not expanded Because goal was found Found Goal Node: $.08 = 4 + 4 ( 2 coins) BFS will found solution at shallowest node, which is the least number of coins.
• 9.
  Put Solution inSolution Space Dollar Amount Coins .08 [ 4, 4 ] Store the solution in a solution space Now let’s solve: coins = BFS( $.23, [ 15, 5, 4, 1 ] )
• 10.
  Discover Solution usingBFS Search Tree 1 4 5 Root of Search Tree coins = BFS( $.23, [ 15, 5, 4, 1 ] ) 15 First Level of Search Tree, Possible Coin choices <= $.23
• 11.
  Discover Solution usingBFS Search Tree 1 4 5 Root of Search Tree coins = BFS( $.23, [ 15, 5, 4, 1 ] ) 15 4 4 Dollar Amount Coins .08 [ 4, 4 ] .23 [15, 4, 4] Solve sub-problem $.22 Solve sub-problem $.08 Solve sub-problem $.19 Solve sub-problem $.18 Reuse Solved Solution for .08
• 12.
  Advanced – InitialSolution Space 1 4 5 coins = BFS( $.08, [ 15, 5, 4, 1 ] ) sum = 4 Dollar Amount Coins .01 [ 1 ] .04 [ 4 ] .05 [ 5 ] Initial Optimal Solutions
• 13.
  Advanced– Prune LessOptimal Solution 1 1 4 5 sum = 5 coins = BFS( $.08, [ 15, 5, 4, 1 ] ) Dollar Amount Coins .01 [ 1 ] .02 [ 1, 1 ] .04 [ 4 ] .05 [ 5 ] .06 [ 5, 1 ] Add new solutions Prune Node since current solution is less optimal than solution in solution space. Solve for sub-problem $.07
• 14.
  Advanced – PruneNodes when Solution Exists 4 sum = 4 coins = BFS( $.08, [ 15, 5, 4, 1 ] ) 4 sum = 8 Dollar Amount Coins .01 [ 1 ] .02 [ 1, 1 ] .04 [ 4 ] .05 [ 5 ] .06 [ 5, 1 ] .08 [ 4, 4 ] Solve for sub-problem $.04 Add solution for $.08 No node for 1 cent coin because solution exists.
• 15.
  Advanced – InitialSolutions for $.23 1 4 5 coins = BFS( $.23, [ 15, 5, 4, 1 ] ) 15 Dollar Amount Coins .01 [ 1 ] .02 [ 1, 1 ] .04 [ 4 ] .05 [ 5 ] .06 [ 5, 1 ] .08 [ 4, 4 ] .15 [ 15 ] Add solution for $.15
• 16.
  Advanced – ApplySub-Solution $.08 1 4 5 coins = BFS( $.23, [ 15, 5, 4, 1 ] ) 15 Dollar Amount Coins .01 [ 1 ] .02 [ 1, 1 ] .04 [ 4 ] .05 [ 5 ] .06 [ 5, 1 ] .08 [ 4, 4 ] .15 [ 15 ] Solve for $.22 Solve for $.08 Solve for $.19 4 4 Solve for $.18 Reuse solution for $.08
• 17.
  Advanced – PruneN-1 candidate solutions 1 4 5 coins = BFS( $.23, [ 15, 5, 4, 1 ] ) 15 Solve for $.08 Solve for $.15 4 4 N = 2 4 Solve for $.18 If sub-solution is N deep, then only search n-1 levels deeper for alternate solution. If not found at N-1 depth, then alternate solution must be at least N deep and therefore not a better solution. 1 Prune at N-1
• 18.
  Search Solution Spacefor Prior Solutions That’s Dynamic Programming!