You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: _posts/2025-01-07-big-o-notation.markdown
+15-10Lines changed: 15 additions & 10 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -20,7 +20,7 @@ A set is a collection of things, usually mathematical objects like numbers, sets
20
20
Denoting an element $$x$$ in a set $$S$$ can be denoted as $$x \in S$$.
21
21
22
22
### Functions
23
-
A function is a mathematical object similar to pure functions in computer science, that can return any mathematical object. A function is often denoted as $$f(x)$$ or $$g(x)$$, where $$x$$ can be any mathematical object.
23
+
A function is a mathematical object similar to pure functions in computer science, that define a mapping from its "domain" (a set of possible inputs) to its "codomain" (a set ofpossibleoutputs).
24
24
25
25
## Back to big O
26
26
Big O notation describes how a function grows asymptotically. In simpler terms, it describes the approximate rate that a function $$f(n)$$ grows with $$n$$. In computer science, $$n$$ is usually used as the *size* of the input. Although big O notation has been defined before, Knuth [\[1\]](#knuth) redefined the functions $$O(f(n))$$, $$\Omega(f(n))$$ and $$\Theta(f(n))$$ as:
@@ -33,7 +33,7 @@ Big O notation describes how a function grows asymptotically. In simpler terms,
33
33
34
34
Woah, what does that mean? Backtracking a bit, let's focus specifically on this line:
35
35
36
-
> $$O(f(n))$$ denotes the set of all $$g(n)$$ such that there exist positive constants $$C$$ and $$n_O$$ with $$\lvert g(n) \le Cf(n) \rvert$$ for all $$n \ge n_O$$.
36
+
> $$O(f(n))$$ denotes the set of all $$g(n)$$ such that there exist positive constants $$C$$ and $$n_O$$ with $$\lvert g(n) \rvert \le Cf(n)$$ for all $$n \ge n_O$$.
37
37
38
38
Here, Knuth is defining $$O(f(n))$$ as a *function* whose *domain*, its input, is an expression ($$f(n)$$). Note that $$n$$ here is an indeterminate, similar to the $$x$$es in polynomials and other expressions, except $$f(n)$$ does not necessarily need to be a polynomial (as an example, $$\sqrt{n}$$ isn't a polynomial).
39
39
@@ -52,19 +52,19 @@ From here, we can make a clearer definition of big O notation and the other func
52
52
53
53
> Let $$k$$ be a non-zero integer constant.
54
54
55
-
> $$O(f(n))$$ is the set of expressions $$g(n)$$ such that, given a constant factor $$k$$, $$g(n)$$ is a lower bound of $$k \cdot f(n)$$
55
+
> $$O(f(n))$$ is the set of expressions $$g(n)$$ such that, given a constant factor $$k$$, $$g(n)$$ is a lower bound of $$k \cdot f(n)$$ and $$n_O \le n$$
56
56
57
-
> $$\Omega(f(n))$$ is the set of expressions $$g(n)$$ such that, given a constant factor $$k$$, $$g(n)$$ is a upper bound of $$k \cdot f(n)$$
57
+
> $$\Omega(f(n))$$ is the set of expressions $$g(n)$$ such that, given a constant factor $$k$$, $$g(n)$$ is a upper bound of $$k \cdot f(n)$$ and $$n_O \le n$$
58
58
59
-
> $$\Theta(f(n))$$ is the set of expressions $$g(n)$$ such that, given two constant factors $$k$$ and $$k'$$, $$f(n)$$ is between $$k \cdot g(n)$$ and $$k' \cdot g(n)$$.
59
+
> $$\Theta(f(n))$$ is the set of expressions $$g(n)$$ such that, given two constant factors $$k$$ and $$k'$$, $$f(n)$$ is between $$k \cdot g(n)$$ and $$k' \cdot g(n)$$ and $$n_O \le n$$
60
60
61
-
Written more formally, you can define this with limits as [\[2\]](#leighton):
61
+
An alternative definition written with limits [\[2\]](#leighton) is:
where $$k$$ is defined as the constant factor in the previous definitions.
70
70
@@ -81,7 +81,7 @@ def multiply(a: int, b: int):
81
81
return out
82
82
{% endhighlight %}
83
83
84
-
At first glance, ignoring the additional complexity from the addition and resizing from the bigint logic, the running time seems to grow linearly as $$b$$ grows, or $$O(n)$$. However, in analysis of algorithms, the indeterminate represents the size, not the value. Usually, the size is defined in bits, $$n = log_2(b)$$, therefore the time complexity is $$O(2^n)$$.
84
+
At first glance, ignoring the additional complexity from the addition and resizing from the bigint logic, the running time seems to grow linearly as $$b$$ grows, or $$T(n) = O(n)$$. However, in analysis of algorithms, the indeterminate represents the size, not the value. Usually, the size is defined in the size in bits of the input, $$n = log_2(b)$$, therefore the time complexity is $$T(n) = O(2^n)$$.
85
85
86
86
Another thing to note is that although it determines the asymptotic complexity of an algorithm, a "better" asymptotic complexity does not necessarily mean an increase in speed. Asymptotically optimal algorithms exist that are practically unlikely to be used, called [galactic algorithms](https://en.wikipedia.org/wiki/Galactic_algorithm).
87
87
@@ -107,3 +107,8 @@ T. Leighton, R. Rubenfeld, The Asymptotic Cheat Sheet [PDF document]. Available:
107
107
The info in this blog post has tried to be accurate, yet there may be some issues and mistakes. If you find any, mention me on Mastodon at somehybrid@hachyderm.io and I might have enough motivation to actually fix any of it.
108
108
109
109
Also, I probably messed up some of the math here. Take everything with lots of seasonings.
110
+
111
+
## Changelog (woops me error)
112
+
push first, proofread after
113
+
114
+
2025-01-07 (can you really believe its 2025? almost wrote 2022): fix markup errors, fix wording in big O in CS, make definiitions clearer, fix typos, fix definition of limits.
0 commit comments