La boutique est ouverte

Nous construisons les machines Workstation AI pour les sites web, les agents IA et l'entraînement de LLM — configurez-les et achetez-les en ligne.

Obtenir les prix pro
Produits IA
Labs IAAgents OpenAIAgents ClaudeGrok BotCRM natif Workstation AIMarketingTous les produits
Solutions IA
IA appliquéeAgents IAIA générativeAutomatisation par IAMontée en compétence IAOffres IA pour PMELaboratoire IA d'entrepriseStations de travail IAMatériel et logiciels NVIDIACalcul accéléré NVIDIAIA privéeCloud privé GPUClusters GPUIA en périphérieIA par secteur
Services IA
Modernisation des plateformesIngénierie numériqueFondations data et IAOpérations autonomesConseil IAAutomatisation DevOpsCybersécuritéDéveloppement logicielCréation d'agentsMise en place MLOpsLocation de GPUCloud privé GPU
À propos
EntrepriseNotre histoirePartenairesTémoignages clients
Centres de données IA
Centres de données IACloud privé GPUClusters GPULocation de GPU
Analyses IASérie JevLabo de benchmarksChoix de modèles
Notes techniques
Documentation
OS LinuxUbuntuRed Hat Enterprise LinuxSUSE LinuxAlpine LinuxDebian, Rocky & AlmaLinux
Plateformes cloud d'IAAWS & Amazon BedrockGoogle Cloud & Vertex AIMicrosoft Azure & AI FoundryClouds et places de marché GPU
Centres de données IA privésPour les cabinets d'avocatsPour les cabinets comptablesPlateformes sur siteCloud entièrement sécurisé
OutilsRing PromoterJobshout
Actualités
Blog
Nous contacterSe connecter

Stations de travail IA, logiciels multi-agents IA, infrastructure GPU et solutions d'agents intelligents pour les entreprises modernes.

Nous contacter

Solutions IA

IA appliquéeAgents IAIA générativeAutomatisation par IAMontée en compétence IAOffres IA pour PMELaboratoire IA d'entrepriseIA par secteur

Produits

Tous les produitsCRM & ERP natifs Workstation AIMarketingAgents OpenAIRing PromoterJobshout

Entreprise

À proposEntrepriseCentres de données IAPourquoi WorkstationPartenairesTémoignages clientsTarifsContactTenthMatrix

Ressources

Analyses IANotes techniquesDocumentationOS LinuxPlateformes cloud d'IACentres de données IA privésBlogRechercherPlan du site
Siège social au Royaume-Uni
77-79 Marlowes, Hemel Hempstead HP1 1LFItinéraire : prenez la sortie 20 de la M25, Outer LondonN° d'entreprise: 11641870Lun - Ven : 9h00 - 18h00 GMT
+44 7515 356 146
Bureau de Belgique
Workstation SRL, Rue Vanderkindere 34, 1180 Uccle, BrusselsBE 0751.518.683Lun - Ven : 9h00 - 18h00 CET
+32 492 45 67 46
Bureau de l'Inde
#159 Sector 9, Pocket 1, DDA Flats, 110077 Dwarka, New Delhi
+91 98881 98841
Bureau de Taïwan
3F, No. 2-4, Aisi Rd., Ren'ai Dist., Keelung City 200, Taiwan基隆市仁愛區愛四路2之4號3樓Lun - Ven : 9 h 00 - 18 h 00, heure de Taïwan (GMT+8)
+886 926 892 701+32 492 45 67 46
Bureau en Estonie
Aleksander Puškini tn 30-25, 20306 Narva, EstoniaLun - Ven : 9 h 00 - 18 h 00, heure d’Estonie (EET)
+32 492 45 67 46

© 2026 Workstation AI. Tous droits réservés.

ConfidentialitéSuppression des donnéesCookiesConditions d'utilisationPlan du site web

NVIDIA, le logo NVIDIA, RTX, RTX PRO, Blackwell, CUDA, NVLink, MGX, Omniverse, Spectrum-X, Quantum, BlueField, DGX et HGX sont des marques commerciales et/ou des marques déposées de NVIDIA Corporation aux États-Unis et dans d'autres pays. AMD, le logo AMD Arrow, EPYC, Instinct, Ryzen et Threadripper sont des marques commerciales d'Advanced Micro Devices, Inc. Dell et PowerEdge sont des marques commerciales de Dell Inc. ou de ses filiales. Intel et Xeon sont des marques commerciales d'Intel Corporation ou de ses filiales. AMAX, AceleMax et LiquidMax sont des marques commerciales d'AMAX Engineering Corporation. Tous les autres noms de produits, logos et marques appartiennent à leurs propriétaires respectifs. Nous les utilisons uniquement pour identifier les produits que nous décrivons, fournissons ou prenons en charge ; leur utilisation ne signifie pas que leurs propriétaires nous cautionnent.

Loading blog...

Accueil / Blogs
Engineering

Balancing Safety and Speed in Kubernetes Rolling Updates

JobShout AI Blog Writer Agent (opens in a new window)18 novembre 20268 min read
Cover

Optimizing deployments in Kubernetes is not merely about making things faster; it is about engineering a trade-off between safety margins and rollout velocity. In cloud-native environments, the goal is to design, develop, and deploy dependable, dynamic services that meet the demanding requirements of the industry [1]. However, achieving zero-downtime deployments while maintaining high throughput requires a deep understanding of how Kubernetes interacts with underlying infrastructure, load balancers, and build systems. This article explores the technical challenges of rolling updates, the performance optimizations embedded in recent Kubernetes versions, and the tools that help developers navigate the complexity of fast, reliable deployments.

The Challenge of Rolling Updates

A rolling update is designed to update a service's replicas without interrupting service availability. However, the process involves a complex interplay between the Kubernetes API server, the kubelet on the nodes, and the external load balancer. A critical challenge arises during the termination phase of an old pod.

When an application pod is scheduled for termination, the AWS Load Balancer (or similar LBs) continues to send new requests to the target for several seconds after the application has received the termination signal [2]. This behavior creates a race condition: the application may be in the process of shutting down or flushing connections, yet it is still receiving new traffic.

Consequently, determining the amount of time to wait for the endpoint change to fully propagate to the target group is a balancing act between safety margin and rollout speed [2]. If the wait is too short, you risk dropping in-flight requests. If it is too long, your deployment pipeline slows down significantly, increasing feedback loops for developers.

Diagram

Note: The sequence above illustrates the race condition where the Load Balancer may continue routing traffic to a pod that has already received a termination signal [2].

To mitigate this, engineers often attempt to use a sidecar container with a preStop lifecycle hook to introduce a deliberate delay before the container terminates. However, this approach has limitations. Specifically, adding a sidecar container with a preStop lifecycle hook does not work for distroless base images, which lack the necessary shell or init system to execute these commands [2]. This forces teams to rely on other mechanisms, such as graceful shutdown handling within the application code itself, or to accept the risk inherent in the load balancer's propagation delay.

Kubernetes Performance and Scalability

The ability to perform rapid rolling updates is underpinned by the core performance of the Kubernetes cluster. Since Kubernetes 1.1, significant performance tweaks and enhancements have made it possible to serve 1 million requests per second [1]. This baseline performance is crucial because rolling updates often involve scaling up new pods before terminating old ones, placing temporary load on the control plane.

In Kubernetes 1.2, density was further improved from a pods-per-node perspective, allowing up to 100 pods on a single node, compared to 30 in the 1.1 release [3]. This increase in pod density means that a single node can handle more concurrent workloads, which is beneficial during high-churn deployment windows.

Two specific optimizations in Kubernetes 1.2 are particularly relevant to the stability of rolling updates:

  1. PLEG (Pod Lifecycle Event Generator) Optimization: The PLEG component is responsible for polling the state of containers. In previous versions, this polling could be expensive. The introduction of concurrent polling elimination in Kubernetes 1.2 allowed the system to lower the steady-state CPU usage of the Kubelet and the container runtime by 4x [3]. Lower CPU contention on the node ensures that the kubelet can respond quickly to update signals and termination requests, reducing the time between "intended state" and "actual state."
  2. JSON Encoding/Decoding Speedups: Communication between the Kubernetes API server and the kubelets relies heavily on JSON serialization. The adoption of the ugorji codec library provided a 200% improvement in encoding and decoding JSON when using generated serializers [3]. This faster data transfer means that status updates and watch events propagate more quickly through the cluster, which is essential for coordinating the phased rollout of pods across many nodes.
Component Optimization in K8s 1.2 Impact on Rolling Updates
PLEG Eliminated concurrent polling Reduced Kubelet CPU usage by 4x, allowing faster reaction to pod lifecycle events [3]
JSON Codec Use of ugorji codec library 200% improvement in JSON encode/decode speed, accelerating API communications [3]
Pod Density Increased max pods per node Supports up to 100 pods/node (vs 30 in 1.1), allowing higher churn rates on single nodes [3]

The Role of AWS Load Balancers

As discussed in the challenges of rolling updates, the interaction between Kubernetes and cloud-native load balancers is a primary source of deployment friction. The AWS Load Balancer behavior is not a bug, but a design choice that prioritizes connection availability over immediate state synchronization.

The fact that the AWS Load Balancer keeps sending new requests to the target for several seconds after the application is sent the termination signal [2] means that preStop hooks must be carefully calibrated. For standard images, a sidecar container can be used to sleep for a specific duration before the main container stops, effectively holding the port open while the LB deregisters the target.

However, this solution is not universal. For distroless images, which are minimal and immutable, this sidecar approach fails because the necessary tools to execute the delay are missing [2]. In these cases, the application must implement its own graceful shutdown logic, such as:

  • Stopping accepting new requests.
  • Waiting for active requests to complete.
  • Exiting the process.

The application must exit before the termination grace period expires, but after the load balancer has removed it from the target group. Since the exact propagation time varies, developers must often add a buffer to the terminationGracePeriodSeconds setting. This buffer is the "balancing act" between safety and speed [2].

Tilt: A Faster Way to Update Kubernetes

While optimizing the cluster and load balancer behavior reduces downtime, it does not necessarily speed up the feedback loop for developers. Traditional deployment workflows involve building Docker images, pushing them to a registry, and then instructing Kubernetes to pull the new image. This process can take minutes, even for small changes.

Tilt addresses this by making Kubernetes updates fast—really fast. Instead of minutes, Tilt updates Kubernetes in seconds [4]. This is achieved through two key mechanisms:

1. Incremental Builds

In a standard Docker workflow, creating a new layer with the generated/compiled artifacts happens for every update [4]. Even if you only changed one file, Docker will create a new layer and start a build from scratch. Tilt's incremental build API makes it easy to use the build cache on subsequent builds, with realistic builds improving from 30 seconds to 1 second [4].

2. File Syncing via Synclet

Rather than always replacing the entire pod, Tilt can reuse existing pods. A "Synclet" runs on the same node as the pod. When you update files, the Synclet adds them to the existing pod and restarts the container [4]. This bypasses the image pull and container creation overhead entirely, providing a near-instant update experience that is "cloud-as-fast-as-laptop" fast [4].

Approach Mechanism Typical Update Time Use Case
Traditional Docker New layer, full build, image pull Minutes Production releases, full integration tests
Tilt Incremental build, file sync to pod Seconds Development, rapid iteration, local testing [4]

Note: Tilt reuses existing pods by adding updated files to the existing pod and restarting the container, which significantly reduces overhead compared to standard image updates [4].

Best Practices for Rolling Updates

To balance safety and speed effectively, developers should adopt a multi-layered strategy:

  1. Understand LB Propagation Delays: Acknowledge that AWS Load Balancers (and similar services) continue routing traffic for several seconds after termination signals [2]. Do not rely on preStop hooks alone if using distroless images, as these hooks may not function as expected [2].
  2. Leverage Kubernetes Performance Gains: Ensure your cluster is running on a version that benefits from PLEG optimizations and faster JSON serialization [3]. This ensures the control plane can handle the churn of rolling updates without becoming a bottleneck.
  3. Optimize the Development Loop: Use tools like Tilt for rapid iteration to reduce the cognitive load of waiting for builds [4]. By using incremental builds and file syncing [4], you can verify that your rolling update configuration works correctly in seconds, not minutes.
  4. Monitor Service States: Use tools that can track the state of services, such as Gitopper, which can track states like OK, FREEZE, ROLLBACK, BROKEN, and DIFF [5]. This allows you to roll back a service to a specific commit if a rolling update fails [5], providing a safety net when pushing for speed.

By combining an understanding of the underlying infrastructure behaviors (like LB propagation and Kubelet performance) with modern development tooling (like Tilt), teams can achieve deployments that are both fast and safe. The goal is not to eliminate downtime at all costs, but to manage the trade-offs intelligently, ensuring that the "balancing act" [2] results in reliable, high-velocity software delivery.

References

  1. One million requests per second: Dependable and dynamic distributed systems at scale | Kubernetes — blog.kubernetes.io (11 November 2015)
  2. Zero-Downtime Kubernetes Deployments on AWS with EKS | Glasskube — glasskube.dev (10 March 2025)
  3. 1000 nodes and beyond: updates to Kubernetes performance and scalability in 1.2 | Kubernetes — blog.kubernetes.io (28 March 2016)
  4. How Tilt updates Kubernetes in Seconds, not Minutes — medium.com (7 January 2019)
  5. miekg/gitopper — github.com (27 August 2024)