The AV Stack Demystified: How a Self-Driving Car Really Works

This article examines the role of simulation and generative AI in enhancing the safety of autonomous vehicles. Neha Boloor's innovations are pivotal in building trust through transparency and effective scenario analysis.

Autonomous vehicles have moved from the fringes of research labs into real-world streets, but public understanding of how they work—and how they are kept safe—remains limited. At the heart of the AV revolution are four systems: perception, planning, prediction, and control. Together, these components process billions of data points per second to help a vehicle not only drive but make decisions with human-level reliability.

Neha Boloor has dedicated her career to building that reliability into practice. As a Senior IEEE Panel Reviewer and with expertise in generative AI, scalable machine learning infrastructure, and safety validation, she has focused on ensuring that AV systems are not only functional but trustworthy at scale.

Simulation at Scale: The Future of Safety Validation

In the AV industry, simulation has become the critical proving ground. Physical testing, while important, cannot cover the millions of rare and complex scenarios that self-driving cars must be prepared to handle. Industry leaders increasingly rely on simulation to validate safety, but the challenge is cost and scale: running every scenario to test the AV software on, can be computationally prohibitive.

Boloor’s work has directly addressed this bottleneck. She led the design and development of a smart scenario sampling strategy, an AI-driven solution that allows AV companies to analyze millions of miles of driving data by selectively running only “the most informative” scenarios. “We know not all miles are equal,” she explains. “Our focus was on designing a system that could intelligently prioritize the hardest, riskiest situations without wasting resources.”

This new methodology reduced compute requirements for development runs by 50%—saving millions of dollars annually—without any degradation in safety assurance. More importantly, it doubled the evaluated miles in simulation, giving engineers stronger statistical confidence in collision estimates.

Generative AI and the Next Layer of Autonomy

Generative AI has transformed fields from art to finance, and its influence in AVs is growing rapidly. By generating synthetic driving scenes, engineers can expose AV stacks to situations rarely captured in the real world—like a pedestrian suddenly jaywalking at night or a cyclist swerving across traffic. This allows vehicles to train on edge cases that might otherwise go untested.

Boloor holds two patents in this space. These innovations enable AV systems to learn from a far richer set of scenarios than traditional data collection could ever provide. “Synthetic scenes allow us to simulate the unexpected,” she says. “And in AVs, it’s the unexpected that matters most.”

Trust Through Transparency

Safety in autonomy isn’t just technical—it’s also social. Public acceptance of AVs depends on transparency, and the industry has faced criticism for being opaque about how safety is measured. Increasingly, experts advocate for interpretable AI systems that can explain why a decision was made, especially in safety-critical contexts.

For Boloor, a Silver Stevie Award winner, this means designing not just algorithms but accountability. Her work on smart scenario sampling emphasized interpretability, separating “risky” and “easy” driving scenarios so that engineers—and regulators—could understand how AV software performed at different levels of risk. “Validation isn’t just about passing a test,” she notes. “It’s about being able to explain why the system is safe, in terms that regulators and the public can trust.”

Why It Matters

As self-driving cars inch closer to mainstream deployment, the industry is grappling with how to balance ambition with assurance. AV stacks must prove they can outperform human drivers not in perfect conditions but in complex, messy realities. The frameworks and patents Boloor has pioneered—together with her recognition as a Silver Stevie Award winner—are helping define what that assurance looks like.

“The future of autonomy won’t be decided by flashy demos,” Boloor says. “It will be decided by the systems we put in place to prove safety, reduce uncertainty, and earn public trust.”

Her career shows how technical rigor and creativity can come together to advance one of the most ambitious engineering challenges of our time: making autonomy safe, scalable, and real.