Infrastructure That Adapts: Vision ML Workloads On Low-Bandwidth Kubernetes

This article discusses the challenges and solutions for deploying machine learning workloads in low-bandwidth environments using Kubernetes. It highlights the importance of composability and resilience in infrastructure design.

Infrastructure That Adapts: Deploying Vision ML Workloads on Low-Bandwidth Bare Metal Using Kubernetes

There is a blind spot in how modern infrastructure is built. In most enterprises, the ML lifecycle has matured inside compute-rich, bandwidth-stable cloud environments. But the real world is not the cloud. It is warehouses with metal roofs, clinics with patchy networks, and factory floors with tight latency constraints. It is where bandwidth collapses and GPUs are not infinite. And yet, this is exactly where ML workloads are increasingly being deployed.

"We are past the phase where ML was just a cloud problem," says Aniruddha Maru, VP of Infrastructure at Standard AI and a Senior IEEE member. "The edge has its own physics, limited bandwidth, unreliable connectivity, power constraints, and we need infrastructure that acknowledges that instead of abstracting it away."

Why Low-Bandwidth ML Deployment Is Still Underserved

At first glance, deploying ML at the edge seems like an extension of cloud pipelines. But reality diverges fast. Vision models, for instance, are bandwidth-heavy by nature. Feeding continuous video streams into the cloud from stores or factories is expensive and unreliable. Local inference becomes necessary, but that means optimizing workloads to run on constrained hardware. Standard’s Autonomous Checkout product—deployed in real-world retail environments like Circle K and built by Aniruddha and his team—embodies this challenge. He worked on both the infrastructure and backend that establish and run the system, designed specifically to operate under limited connectivity and constrained hardware.

Deploying vision ML workloads on low-bandwidth bare metal using Kubernetes is central to this challenge. Bare metal orchestration with Kubernetes allows teams to tightly control scheduling and resource usage while avoiding unnecessary overhead. At Standard AI, workloads are containerized and dynamically scheduled based on local compute capacity.

But it is not just about containerization, it is about understanding the entire dependency chain. From model updates to logging and telemetry, every component must be aware of failure domains and be built with resilience in mind.

"We have to treat disconnection as a first-class event," Aniruddha adds. "That changes how you log, monitor, and even update your models."

Composability Over Complexity

In cloud-native circles, abstraction is often equated with sophistication. But in edge deployments, every unnecessary layer becomes a liability. The emphasis, Aniruddha explains, must shift from abstraction to composability.
"Composable systems let you rebuild without starting over. That is a superpower in edge environments, where your assumptions are constantly challenged."

Rather than large, monolithic deployments, Aniruddha advocates for modular microservices that can be independently updated and deployed. The architecture is kept lean, each service is purpose-built and failure-tolerant, stitched together via message buses and declarative APIs. This was core to his work at Automatic Labs, where he designed and maintained a suite of 20 microservices powering everything from developer APIs to crash alerting logic. Each microservice had a single responsibility—from a time-series database tailored to driving data, to an OAuth2 provider engineered from scratch—which made the system easier to debug, evolve, and scale as adoption grew.

This modular philosophy is reflected in how the ML pipeline operates. Model inference, streaming, alerting, and user feedback loops all live as independent components that can scale or fail independently. This drastically improves observability and makes recovery deterministic rather than dependent on fragile orchestration logic.
"The key," Aniruddha says, "is not just resilience, but debuggability. You should not need a PhD in distributed systems to figure out why your inference failed."

DevOps at the Edge: GitOps Meets Reality

Traditional DevOps tooling assumes connectivity, especially when it comes to CI/CD, logging, and real-time observability. But when deploying in low-bandwidth regions, many of these assumptions fail. That is where GitOps has found a new relevance.
"GitOps works beautifully at the edge, because it treats declarative state as truth. Even if you lose the connection, the node knows what it should become," Aniruddha explains.

While some engineering teams in the industry pair GitOps with Kubernetes to enable consistent deployments in disconnected environments, as demonstrated by F5 Networks' approach to managing thousands of Kubernetes clusters using a GitOps model, the specific implementation approaches vary widely. What remains consistent is the value of decoupling configuration from availability.

At Automatic, this decoupling was critical in environments where vehicle connectivity could drop at any time. Backend systems had to remain operational regardless of momentary data gaps. By designing for declarative control—where microservices and data pipelines recovered to a known-good state—Aniruddha enabled stability across millions of miles of vehicle telemetry. His infrastructure ensured data ingestion, driver behavior tracking, and insurance-grade analytics continued to function even when network assumptions failed.

The infrastructure Aniruddha oversees is designed for failure, but engineered for continuity. Even in degraded states, ML workloads continue to operate, and infrastructure self-heals when conditions normalize.
"You cannot stop a store just because your model update failed," Aniruddha says. "Our job is to ensure that models run, even when everything else does not."

Engineering for the World as It Is

As machine learning moves out of the lab and into physical environments, infrastructure must evolve. The cloud-native patterns of the last decade are not obsolete, but they are incomplete.

Aniruddha, a Judge at the Globee Awards for Business, believes the new frontier is not more abstraction, but more awareness. "We need to design for the world as it is, not the world as we wish it were. That means infrastructure that respects constraints, recovers intelligently, and thrives in imperfect conditions."

For engineering leaders facing similar constraints, whether deploying retail ML, IoT intelligence, or logistics optimization, the takeaway is clear: composability, observability, and reality-first design are not edge cases. They are the new baseline.
"The best infrastructure," Aniruddha concludes, "does not fight the environment. It adapts to it."