Resource requests are load-bearing

Adding a node creates fresh scheduling capacity, but it does not move existing running pods. New pods, including replacements created during rollouts or after eviction, can land there. If their containers declare no resource requests, the scheduler has an incomplete picture of the capacity they need.

That was the risk I found while reviewing one cluster: fifteen pods declared neither requests nor limits, despite consuming roughly 1.4 GB of memory between them.

The scheduler does not look at usage

This is the part that surprises people, and it surprised me for longer than I would like to admit. When Kubernetes decides where a pod goes, it does not measure how much memory or CPU that pod actually consumes. It reads spec.containers[].resources.requests and treats that number as the truth.

For scheduling feasibility, a container with no effective request reserves no capacity for that resource. Check the admitted Pod specification: admission policies can supply defaults, and a limit can become the request when no request is specified. Actual usage still matters to node health, even when it is absent from scheduling accounting.

Node scoring combines several configured plugins and constraints. Resource allocation is one input, so a node with spare capacity on paper can be attractive even when workloads use more than their requests suggest. It does not necessarily win every scheduling decision.

The practical risk is overcommitment: replacement or newly created pods can fit according to requests while leaving too little real memory headroom. Adding capacity does not correct missing requests on existing workloads.

What it costs

The count is usually higher than expected. On one cluster, fifteen pods had no requests declared at all. Their actual combined usage was roughly 1.4 GB of memory. That was 1.4 GB the scheduler did not know about, distributed across nodes according to a model that said it did not exist.

None of it had caused an outage, which is what makes it easy to leave alone. The cluster looks balanced in every dashboard that reports requests, and is quietly not balanced at all.

The second problem: QoS class

Requests and limits also determine Pod quality of service. For memory-pressure eviction, kubelet considers whether usage exceeds requests, Pod priority, and usage relative to requests. QoS is a useful guide to likely behaviour, not a strict eviction queue.

• Guaranteed: every container sets CPU and memory requests equal to its limits for both resources.
• Burstable: the Pod has a CPU or memory request or limit, but does not meet the Guaranteed criteria.
• BestEffort: no container sets CPU or memory requests or limits.

Every one of those fifteen pods was BestEffort. With no memory request, any memory use puts a pod above its request, making it vulnerable to eviction under memory pressure. Missing requests hurt both placement accounting and resilience.

Sizing requests without guessing

The advice to “just set requests” is easy to give and annoying to act on, because the obvious question is what number to use. Two rules got me most of the way.

Size requests from observed usage, not from fear. Look at what the workload actually consumes in steady state and set the request near that, with a little headroom. Requests are a reservation. Every megabyte you request is a megabyte no other pod can be scheduled against, whether or not you ever touch it. Inflating requests to feel safe is how you end up buying a node you did not need.

kubectl top pods -A --sort-by=memory

# and the inverse: everything that declares nothing
kubectl get pods -A -o json | jq -r '
  .items[] | select(
    [.spec.containers[].resources.requests // {}] | map(length) | add == 0
  ) | "\(.metadata.namespace)/\(.metadata.name)"'

Size limits separately from requests. With explicit requests, raising a limit does not reserve additional scheduling capacity. It does allow more runtime consumption: generous memory limits across many pods can still exhaust a node, while CPU limits can throttle work. Choose limits from workload measurements and the headroom available, rather than a universal multiplier.

There is one important exception. If a process has its own internal memory ceiling, setting the container limit equal to that ceiling will get it killed before its own eviction logic ever runs. That is a specific enough trap that it deserves its own post, and I have written one.

The rule I settled on

Every deployment declares requests. Not because every workload is important, but because a workload that declares nothing is invisible to the only model the scheduler has. An unimportant pod that lies about costing zero does more damage to placement decisions than an important one that tells the truth.

Include init containers in the review. Their requests contribute to the Pod’s effective scheduling requirement even though ordinary init containers run sequentially. The command above checks application containers only; inspect spec.initContainers separately.

If you want one thing to check after reading this, run the second command above. The output is usually shorter than you expect and more interesting than you want.

References: Kubernetes resource management, scheduling, and node-pressure eviction.