Dynamic Programming: Fibonacci Sequence

What is Dynamic Programming?

Dynamic Programming (DP) is an algorithmic technique for solving complex problems by breaking them down into simpler, overlapping subproblems and solving each subproblem only once. The results of these subproblems are stored (usually in an array or hash map) to avoid redundant computations.

DP is typically applicable when a problem exhibits two key characteristics:

• Overlapping Subproblems: The problem can be broken down into subproblems that are reused several times. For example, calculating fib(5) requires calculating fib(4) and fib(3). Calculating fib(4) also requires calculating fib(3). Notice fib(3) is needed multiple times.
• Optimal Substructure: The optimal solution to the overall problem can be constructed from the optimal solutions of its subproblems. The Fibonacci sequence naturally has this property: fib(n) is directly built from fib(n-1) and fib(n-2).

Avoiding Redundant Calculations

Consider calculating fib(5) recursively:

fib(5)
├── fib(4)
│   ├── fib(3)
│   │   ├── fib(2)
│   │   │   ├── fib(1)
│   │   │   └── fib(0)
│   │   └── fib(1)
│   └── fib(2)
│       ├── fib(1)
│       └── fib(0)
└── fib(3)
    ├── fib(2)
    │   ├── fib(1)
    │   └── fib(0)
    └── fib(1)
            

Notice how many times fib(3), fib(2), fib(1), and fib(0) are calculated. DP eliminates this repetition.

Approaches to Dynamic Programming

1. Memoization (Top-Down):

   This approach follows the natural recursive structure but stores the result of each subproblem the first time it's computed. Before computing a subproblem, it checks if the result is already stored. If so, it returns the stored result directly, avoiding re-computation.

2. Tabulation (Bottom-Up):

   This approach solves the problem "bottom-up" by first solving the smallest subproblems and then using those results to build up solutions to larger subproblems. It typically involves filling a table (e.g., an array) iteratively, starting from the base cases.

Enter n: Run Naive Recursive Run DP (Memoization) Run DP (Tabulation)

Results Comparison

Fibonacci Number (fib()):

Naive Recursive Calls:

Memoization Calls (Function Entries):

Tabulation Steps (Iterations):

Approximate Timings (ms):

Naive: ms

Memoization: ms

Tabulation: ms

Time and Space Complexity

Approach	Time Complexity	Space Complexity
Naive Recursive	O(2n) - Exponential	O(n) - Due to recursion stack depth
DP (Memoization)	O(n) - Each subproblem fib(k) is computed only once	O(n) - For the memoization table and recursion stack
DP (Tabulation)	O(n) - Simple loop from 2 to n	O(n) - For the tabulation table. (Can be optimized to O(1) space)

Note: The Tabulation approach for Fibonacci can be further optimized to use only O(1) space by storing only the previous two values instead of the entire table.