Imagine a place that is something like SimCity, the urban development simulation game, where you can control what happens on the streets. You manage how many cars move along the roads and how fast they go. You allow a certain number of cyclists on a roundabout – and allow some to ignore basic rules of the road. You can even change the weather conditions, with sunny skies one moment and suddenly there’s a heavy downpour.
Such ‘power’ is usually only available in computer simulations and doesn’t exist in real life. However, if you drive out to the middle of Merced County in the American state of California, you’ll find it at Castle, a former air force base used by the Waymo development team to help build ‘The World’s Most Experienced Driver’.
Covering over 450,000 square metres, Castle is a state-of-the-art, closed-course testing facility which has been specially designed and redesigned over the years for the unique testing needs of Waymo, the company handling Google’s autonomous car project and developing the required technologies.
Preparing the Waymo Driver
Waymo’s unique testing needs to prepare the Waymo Driver to handle challenges on the road, evaluate new software before it’s released to the self-driving fleet, and validate system performance in all kinds of conditions. The closed-course testing at Castle has always been a critical component of the development process, unlocking some key technology advancements.
Following some months of lockdown restricted movements due to the pandemic, Waymo’s test vehicles are back on the road and the Waymo Driver’s deployment can proceed with the entire development cycle. This involves gathering real-world data and analyzing it and rigorously testing updates made to the system.
The ‘one in a million’
The testing site on the former base is set up like an adaptable city, including everything from wide avenues and suburban driveways to a railroad crossing and roundabouts. It’s meant to be like the real world and complex and rare scenarios are staged in a safe, controlled environment.
These scenarios can be a person walking across the street without considering oncoming traffic, or garbage falling out of the truck ahead. Over the years, the Waymo team has built up a library of over 40,000 structured testing scenarios — and that does not include all the variations with each situation. Some of the scenarios include are situations never seen on public roads but could happen one day, perhaps once every hundreds of millions of kilometres.
Additionally, there are relatively mundane tests that can also be quite challenging for self-driving vehicles and humans alike. For example, driving behind a large garbage truck on a narrow street — one that keeps stopping to load garbage and sometimes the bins are left on the road – may create a lot of complexity on the road and an autonomous vehicle must know what to do.
One source of scenarios is documented traffic accidents and these situations are recreated. By exposing the Waymo Driver to a wide variety of scenarios and teaching self-driving technology fundamental skills rather than only to handle individual situations, the vehicles can become more equipped to handle any situation they encounter on the road.
Structured testing and simulation
When a scenario is developed to test new software, structured testing, simulation, and public road operations can be used separately or together. For example, after executing a test on the closed track, the researchers create and run hundreds of variations of that scenario in simulation. But as much as simulation can help scale the value of structured testing, structured testing complements simulation.
Waymo’s fifth-generation hardware suite
Over the past decade, Waymo has changed the number of vehicle platforms and sensor suites tested and operated on 5 times — from Lexus 450 SUVs to a custom-built Firefly; from Chrysler Pacifica minivans to 18-wheeler trucks. Today, it uses the all-electric Jaguar I-PACE. Like the platforms that came earlier, today’s custom-built 5th generation sensor suite has been rigorously tested at Castle to help ensure its safety and readiness for public roads.
