#328 - Lowest-cost Search
We are trying to find a hidden number selected from the set of integers \(\{1, 2, \dots, n\}\) by asking questions.
Each number (question) we ask, has a cost equal to the number asked and we get one of three possible answers:
- "Your guess is lower than the hidden number", or
- "Yes, that's it!", or
- "Your guess is higher than the hidden number".
Given the value of \(n\), an optimal strategy minimizes the total cost (i.e. the sum of all the questions asked) for the worst possible case. E.g.
If \(n=3\), the best we can do is obviously to ask the number "2". The answer will immediately lead us to find the hidden number (at a total cost \(= 2\)).
If \(n=8\), we might decide to use a "binary search" type of strategy: Our first question would be "\(\mathbf 4\)" and if the hidden number is higher than \(4\) we will need one or two additional questions.
Let our second question be "\(\mathbf 6\)". If the hidden number is still higher than \(6\), we will need a third question in order to discriminate between \(7\) and \(8\).
Thus, our third question will be "\(\mathbf 7\)" and the total cost for this worst-case scenario will be \(4+6+7={\color{red}\mathbf{17}}\).
We can improve considerably the worst-case cost for \(n=8\), by asking "\(\mathbf 5\)" as our first question.
If we are told that the hidden number is higher than \(5\), our second question will be "\(\mathbf 7\)", then we'll know for certain what the hidden number is (for a total cost of \(5+7={\color{blue}\mathbf{12}}\)).
If we are told that the hidden number is lower than \(5\), our second question will be "\(\mathbf 3\)" and if the hidden number is lower than \(3\) our third question will be "\(\mathbf 1\)", giving a total cost of \(5+3+1={\color{blue}\mathbf 9}\).
Since \({\color{blue}\mathbf{12}} \gt {\color{blue}\mathbf 9}\), the worst-case cost for this strategy is \({\color{red}\mathbf{12}}\). That's better than what we achieved previously with the "binary search" strategy; it is also better than or equal to any other strategy.
So, in fact, we have just described an optimal strategy for \(n=8\).
Let \(C(n)\) be the worst-case cost achieved by an optimal strategy for \(n\), as described above.
Thus \(C(1) = 0\), \(C(2) = 1\), \(C(3) = 2\) and \(C(8) = 12\).
Similarly, \(C(100) = 400\) and \(\sum \limits_{n = 1}^{100} C(n) = 17575\).
Find \(\sum \limits_{n = 1}^{200000} C(n)\).
Problem text © Project Euler, licensed under CC BY-NC-SA 4.0. Original: projecteuler.net/problem=328. Published Saturday, 12th March 2011, 10:00 pm. Solved by 539 members at time of mirroring.
Why this is useful
Algorithmic Development. Optimal substructure and state-space reasoning are exactly how American-option pricing and optimal execution are solved (Phases 13, 16).
We classify relevance honestly - not every Euler problem is a trading application.
Prerequisites
Lessons that prepare you:
10.4 Algorithms: Gradient Descent and Newton's Method · 10.1 Convex Sets and Convex Functions · 17.1 Python for Quants: NumPy, pandas, and Vectorization · 19.14 Computational Complexity, Feasibility Estimation, and Proving Algorithms Correct · 19.7 Dynamic Programming: Memoization and Tabulation · 19.9 Graph Algorithms: BFS, DFS, Dijkstra, and Minimum Spanning Trees · 19.6 Recurrence Relations and Generating Functions
Recommended stepping-stone problems: #950 · #339 · #732
Concepts: dynamic-programming game-theory optimization brute-force-reduction
Likely techniques: bfs-dfs hashing
Learning mode
Pick how much scaffolding you want. Your choice is remembered per problem.
Understand the problem
- What exactly is the input to problem 328? Is it a bound (200000), a supplied dataset, or a definition you must generate from?
- What is the required output - restate it precisely: a single sum.
- Which objects exactly are in scope, and which are excluded by the wording (strict vs non-strict inequality, 'distinct', 'proper', 'below' vs 'up to')?
- What do the arguments of C(n), C(1) mean, and what is the value's type (count, sum, probability)?
- What are the edge cases: the smallest legal object, zero/one, ties, and the boundary at exactly 200000?
- Why is brute force hard HERE specifically? Estimate the number of candidates implied by 200000 and the cost of testing one.
- Which optimization fact would, if true, collapse the search - and can you state it as a testable claim before you look for a proof?
Predict & plan (before you code)
- Predict the strategy: in one sentence, what will your solution do? (The classification says optimization / bfs-dfs - do you agree, and why?)
- Predict the complexity of your intended method in terms of N = 200000, and the wall-clock time you expect. Write both down now.
- Predict the key data structure: what is stored, keyed by what, and how large will it get at full scale?
- Predict the failure mode: what is most likely to break - an off-by-one on the bound, a definition misread, precision, or memory?
- Predict the output of the small case from rung 3 BEFORE running it (the statement says: "we get one of three possible answers: "Your guess is lower than the hidden number", or "Yes, that's it!", or "Your guess is higher than the hidden number".") - then run it. A surprise here is worth more than an hour of debugging later.
Scratchpad
Mathematical notes, formulas, pseudocode, hypotheses, complexity notes. Saved automatically with your progress.
Python workbench
Real Python (Pyodide) in a sandboxed Web Worker - no network, no filesystem, no DOM access. Ctrl/Cmd+Enter runs. Escape leaves the editor. Stop terminates the worker.
Check your answer
Answers are checked against a salted hash held in a separate file - not printed in this page. This prevents accidental spoilers; it is not cryptographic protection (see the build notes).
Progressive hints
Optimization
You have a correct answer. That is the start of the learning, not the end.
- Reduce the time complexity. What is the bottleneck, and what mathematical fact removes it?
- Reduce memory. Can you stream, or keep only the last k states?
- Replace brute force with a closed form, a sieve, a recurrence, or a symmetry argument.
- Prove the optimized version computes the same thing.
- Compare two implementations and time them.
Explain it
Which step of your solution were you least confident about, and what evidence would settle it?
What did you try first, and what specifically made you abandon it - a proof, a timing, or a wrong small-case answer?
Where did the optimization structure do the real work? Name the single observation that collapsed the search space.
Could you have reached the bfs-dfs idea faster? Which words in the statement were pointing at it, and did you notice them?
What was the bug that cost you the most time, and what CLASS of bug was it (off-by-one, definition misread, precision, state under-specified)?
How would your solution change if the bound 200000 were multiplied by 1000? Does it survive, or does it need a different idea?
What is the honest complexity of what you wrote (not what you intended), and where is the remaining slack?
Which problem you have already solved is this most similar to, and what is the shared skeleton - is it really 'bfs-dfs' underneath?
State the transferable technique in one sentence, without mentioning this problem's story at all.
Self-assess (mastery is not a correct number)
You reach Mastered only when you have solved it, rated yourself at least Solid across the dimensions, and written a real explanation.
Confidence
Low confidence schedules this problem for spaced review, even if you solved it.
Mastery check
- Variation: change the bound (or a rule) in the statement. Does your method still work? What breaks first?
- Constraints: if the limit were 10× larger, which step fails, and what would you replace it with?
- Related problem: #950 · #339 · #732
- Transfer: where else does this technique appear? Name a lesson and a real computational setting.
- Spaced re-attempt: come back after the review interval and re-solve it with no hints.