Entropy Always Wins
In the previous parts, we pretended that your program owns the machine.
We optimized cache, aligned arrays, and minimized branches. We assumed that if we wrote good code, it would run fast.
The illusion ends here.
Your program is not running. It is being allowed to run.
The Adversary
The Operating System (OS) is not your friend. It is a Landlord.
It slices time into tiny chunks (Quantums). Every few milliseconds, it pauses your program, saves your registers, flushes your pipeline, and gives the CPU to someone else.
This is a Context Switch.
You don't see it in your code. You only feel it in the latency.
1. Run Code: Watch the "Green" bars. Low latency. Warm cache.
2. Forced Switch: Click it. The OS ("Red") takes over. It fills the cache with its own data.
3. Return: When your code runs again, it hits "Orange" spikes. Your data is gone. You must pay the latency tax to bring it back.
Ecological Performance
Performance is not just about your algorithm. It is about your capability to survive in a hostile ecosystem.
When you share a CPU, you share the Cache. If another process (like a backup daemon or a browser tab) is running, it destroys your cache locality every time it wakes up.
Your code didn't change. But your Hit Rate collapsed.
Can we prevent context switches?
Mostly no. You can pin threads to cores (CPU Affinity) or use Real-Time OS kernels, but standard systems (Linux, Windows, macOS) will always preempt you to handle interrupts, networking, and UI.
Does having more Cores help?
Yes and No. More cores mean you might not share execution time, but you still share L3 Cache and Memory Bandwidth. The neighbor on the next core can still evict your cache lines.
Performance is not Local.
We have learned that we are guests in a shared house.
We have seen what happens when one adversary steals our time.
But modern servers don't just have one adversary. They have thousands.
What happens when everyone tries to run at once?