From 63e90d4d8c08c23db66e901e57ff8b0bc8432f3c Mon Sep 17 00:00:00 2001 From: Adam Djellouli <37275728+djeada@users.noreply.github.com> Date: Tue, 2 Sep 2025 14:04:48 +0200 Subject: [PATCH 1/3] Revise dynamic programming considerations and pitfalls --- notes/dynamic_programming.md | 27 +++++++++++++++------------ 1 file changed, 15 insertions(+), 12 deletions(-) diff --git a/notes/dynamic_programming.md b/notes/dynamic_programming.md index 9233e1c..e54eed8 100644 --- a/notes/dynamic_programming.md +++ b/notes/dynamic_programming.md @@ -288,30 +288,33 @@ We build a two-dimensional table $L[0..m][0..n]$ using the above recurrence. **Time Complexity**: $O(mn)$ -### Practical Considerations in Dynamic Programming +### Practical Considerations #### Identifying DP Problems -Not all problems are amenable to dynamic programming. To determine if DP is appropriate: - -- Can the problem's optimal solution be constructed from optimal solutions to its subproblems? -- Are the same subproblems being solved multiple times? +* If the problem asks for the number of *ways* to do something, DP usually works because smaller counts combine into larger ones; without it, counting paths in a grid would require enumerating every route. +* If the task is to find the *minimum* or *maximum* value under constraints, DP is useful because it compares partial solutions; without it, knapsack would require checking every subset of items. +* If the same *inputs* appear again during recursion, DP saves time by storing answers; without it, Fibonacci numbers would be recomputed many times. +* If the solution depends on both the *current step* and *remaining resources* (time, weight, money, length), DP fits naturally; without it, scheduling tasks within a time limit would require brute force. +* If the problem works with *prefixes, substrings, or subsequences*, DP is often a match because these can be built step by step; without it, longest common subsequence would need exponential checking. +* If choices at each step must be explored and combined carefully, DP provides structure; without it, coin change with mixed denominations cannot guarantee the fewest coins. +* If the state space can be stored in a *table or array*, DP is feasible; without this, problems with infinitely many possibilities (like arbitrary real numbers) cannot be handled. #### State Design and Transition -- Choose variables that capture the essence of subproblems. -- Clearly define how to move from one state to another. +* A well-chosen *state* defines what each subproblem represents, while a poorly chosen one leaves the formulation incomplete; for example, `dp[i][w]` in the knapsack problem captures value using `i` items and capacity `w`. +* A correct *transition* connects states consistently, while skipping this leads to undefined progress; in knapsack, the choice to include or exclude an item gives the formula for moving between states. #### Complexity Optimization -- Reduce the storage requirements by identifying and storing only necessary states. -- Prune unnecessary computations, possibly using techniques like memoization with pruning. +* Reducing *memory usage* by discarding unnecessary states makes solutions efficient, while failing to do so can waste resources; for example, knapsack space can shrink from `O(nW)` to `O(W)` with a one-dimensional array. +* Using *pruning* to skip impossible paths speeds up computation, while omitting it allows redundant work; in recursive search with memoization, branches exceeding a current best value can be safely ignored. #### Common Pitfalls -- Leads to missing subproblems or incorrect dependencies. -- Can cause incorrect results or infinite recursion. -- Failing to handle special inputs can result in errors. +* Missing *base cases* causes results to fail, while including them ensures correct foundations; in grid path counting, setting `dp[0][0] = 1` allows all later counts to build properly. +* Updating *dependencies* in the wrong order leads to invalid reuse, while correct order avoids errors; in knapsack with a 1D array, iterating weights backward prevents an item from being counted twice. +* Ignoring *edge inputs* results in crashes or incorrect answers, while handling them ensures robustness; for example, knapsack with zero capacity must return a value of zero instead of failing. ### List of Problems From 1b1578189ba2a03182fd9843b735b6b26cdd8dfc Mon Sep 17 00:00:00 2001 From: Adam Djellouli <37275728+djeada@users.noreply.github.com> Date: Tue, 2 Sep 2025 14:07:38 +0200 Subject: [PATCH 2/3] Update notes/dynamic_programming.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- notes/dynamic_programming.md | 28 +++++++++++++++++++++------- 1 file changed, 21 insertions(+), 7 deletions(-) diff --git a/notes/dynamic_programming.md b/notes/dynamic_programming.md index e54eed8..92217f9 100644 --- a/notes/dynamic_programming.md +++ b/notes/dynamic_programming.md @@ -292,13 +292,27 @@ We build a two-dimensional table $L[0..m][0..n]$ using the above recurrence. #### Identifying DP Problems -* If the problem asks for the number of *ways* to do something, DP usually works because smaller counts combine into larger ones; without it, counting paths in a grid would require enumerating every route. -* If the task is to find the *minimum* or *maximum* value under constraints, DP is useful because it compares partial solutions; without it, knapsack would require checking every subset of items. -* If the same *inputs* appear again during recursion, DP saves time by storing answers; without it, Fibonacci numbers would be recomputed many times. -* If the solution depends on both the *current step* and *remaining resources* (time, weight, money, length), DP fits naturally; without it, scheduling tasks within a time limit would require brute force. -* If the problem works with *prefixes, substrings, or subsequences*, DP is often a match because these can be built step by step; without it, longest common subsequence would need exponential checking. -* If choices at each step must be explored and combined carefully, DP provides structure; without it, coin change with mixed denominations cannot guarantee the fewest coins. -* If the state space can be stored in a *table or array*, DP is feasible; without this, problems with infinitely many possibilities (like arbitrary real numbers) cannot be handled. +* If the problem asks for the number of *ways* to do something: + * Example: Counting paths in a grid. + * Consequence: Without DP, you would need to enumerate every route. +* If the task is to find the *minimum* or *maximum* value under constraints: + * Example: Knapsack problem. + * Consequence: Without DP, you would need to check every subset of items. +* If the same *inputs* appear again during recursion: + * Example: Fibonacci numbers. + * Consequence: Without DP, Fibonacci numbers would be recomputed many times. +* If the solution depends on both the *current step* and *remaining resources* (time, weight, money, length): + * Example: Scheduling tasks within a time limit. + * Consequence: Without DP, brute force would be required. +* If the problem works with *prefixes, substrings, or subsequences*: + * Example: Longest common subsequence. + * Consequence: Without DP, exponential checking would be needed. +* If choices at each step must be explored and combined carefully: + * Example: Coin change with mixed denominations. + * Consequence: Without DP, you cannot guarantee the fewest coins. +* If the state space can be stored in a *table or array*: + * Example: Problems with discrete states. + * Consequence: Without this, problems with infinitely many possibilities (like arbitrary real numbers) cannot be handled. #### State Design and Transition From 4bc68899bc1dfe2d24b75f656494b3ddd66d7f99 Mon Sep 17 00:00:00 2001 From: Adam Djellouli <37275728+djeada@users.noreply.github.com> Date: Tue, 2 Sep 2025 14:12:00 +0200 Subject: [PATCH 3/3] Update dynamic_programming.md --- notes/dynamic_programming.md | 72 ++++++++++++++++++++++++------------ 1 file changed, 48 insertions(+), 24 deletions(-) diff --git a/notes/dynamic_programming.md b/notes/dynamic_programming.md index 92217f9..b51e372 100644 --- a/notes/dynamic_programming.md +++ b/notes/dynamic_programming.md @@ -292,27 +292,40 @@ We build a two-dimensional table $L[0..m][0..n]$ using the above recurrence. #### Identifying DP Problems -* If the problem asks for the number of *ways* to do something: - * Example: Counting paths in a grid. - * Consequence: Without DP, you would need to enumerate every route. -* If the task is to find the *minimum* or *maximum* value under constraints: - * Example: Knapsack problem. - * Consequence: Without DP, you would need to check every subset of items. -* If the same *inputs* appear again during recursion: - * Example: Fibonacci numbers. - * Consequence: Without DP, Fibonacci numbers would be recomputed many times. -* If the solution depends on both the *current step* and *remaining resources* (time, weight, money, length): - * Example: Scheduling tasks within a time limit. - * Consequence: Without DP, brute force would be required. -* If the problem works with *prefixes, substrings, or subsequences*: - * Example: Longest common subsequence. - * Consequence: Without DP, exponential checking would be needed. -* If choices at each step must be explored and combined carefully: - * Example: Coin change with mixed denominations. - * Consequence: Without DP, you cannot guarantee the fewest coins. -* If the state space can be stored in a *table or array*: - * Example: Problems with discrete states. - * Consequence: Without this, problems with infinitely many possibilities (like arbitrary real numbers) cannot be handled. +I. If the problem asks for the number of *ways* to do something: + +* *Example:* Counting paths in a grid. +* *Consequence:* Without DP, you would need to enumerate every route. + +II. If the task is to find the *minimum* or *maximum* value under constraints: + +* *Example:* Knapsack problem. +* *Consequence:* Without DP, you would need to check every subset of items. + +III. If the same *inputs* appear again during recursion: + +* *Example:* Fibonacci numbers. +* *Consequence:* Without DP, Fibonacci numbers would be recomputed many times. + +IV. If the solution depends on both the *current step* and *remaining resources* (time, weight, money, length): + +* *Example:* Scheduling tasks within a time limit. +* *Consequence:* Without DP, brute force would be required. + +V. If the problem works with *prefixes, substrings, or subsequences*: + +* *Example:* Longest common subsequence. +* *Consequence:* Without DP, exponential checking would be needed. + +VI. If choices at each step must be explored and combined carefully: + +* *Example:* Coin change with mixed denominations. +* *Consequence:* Without DP, you cannot guarantee the fewest coins. + +VII. If the state space can be stored in a *table or array*: + +* *Example:* Problems with discrete states. +* *Consequence:* Without this, problems with infinitely many possibilities (like arbitrary real numbers) cannot be handled. #### State Design and Transition @@ -326,9 +339,20 @@ We build a two-dimensional table $L[0..m][0..n]$ using the above recurrence. #### Common Pitfalls -* Missing *base cases* causes results to fail, while including them ensures correct foundations; in grid path counting, setting `dp[0][0] = 1` allows all later counts to build properly. -* Updating *dependencies* in the wrong order leads to invalid reuse, while correct order avoids errors; in knapsack with a 1D array, iterating weights backward prevents an item from being counted twice. -* Ignoring *edge inputs* results in crashes or incorrect answers, while handling them ensures robustness; for example, knapsack with zero capacity must return a value of zero instead of failing. +I. Failure to Define Proper Base Cases + +* *Example*: In grid path counting, omitting `dp[0][0] = 1` prevents any valid paths from being constructed. +* *Consequence*: Without correct starting values, the DP table propagates errors and produces incorrect results. + +II. Updating States in the Wrong Dependency Order + +* *Example*: In knapsack with a 1D array, iterating weights from low to high causes items to be reused multiple times. +* *Consequence*: Using the wrong order inflates computed values and leads to invalid or impossible solutions. + +III. Ignoring Special or Edge Case Inputs + +* *Example*: In knapsack, a zero-capacity input should return zero value rather than throwing an error. +* *Consequence*: Overlooking edge inputs causes crashes or incorrect answers in boundary conditions. ### List of Problems