Gadgets – TechCrunch Devin Coldewey

A self-driving vehicle made by Uber has struck and killed a pedestrian. It’s the first such incident and will certainly be scrutinized like no other autonomous vehicle interaction in the past. But on the face of it it’s hard to understand how, short of a total system failure, this could happen when the entire car has essentially been designed around preventing exactly this situation from occurring.

Something unexpectedly entering the vehicle’s path is pretty much the first emergency event that autonomous car engineers look at. The situation could be many things — a stopped car, a deer, a pedestrian — and the systems are one and all designed to detect them as early as possible, identify them, and take appropriate action. That could be slowing, stopping, swerving, anything.

Uber’s vehicles are equipped with several different imaging systems which work both ordinary duty (monitoring nearby cars, signs, and lane markings) and extraordinary duty like that just described. No less than four different ones should have picked up the victim in this case.

Top-mounted lidar. The bucket-shaped item on top of these cars is a lidar, or light detection and ranging, system that produces a 3D image of the car’s surroundings multiple times per second. Using infrared laser pulses that bounce off objects and return to the sensor, lidar can detect static and moving objects in considerable detail, day or night.

This is an example of a lidar-created imagery, though not specifically what the Uber vehicle would have seen.

Heavy snow and fog can obscure a lidar’s lasers, and its accuracy decreases with range, but for anything from a few feet to a few hundred feet, it’s an invaluable imaging tool and one that is found on practically every self-driving car.

The lidar unit, if operating correctly, should have been able to make out the person in question, if they were not totally obscured, while they were still more than a hundred feet away, and passed on their presence to the “brain” that collates the imagery.

Front-mounted radar. Radar, like lidar, sends out a signal and waits for it to bounce back, but it uses radio waves instead of light. This makes it more resistant to interference, since radio can pass through snow and fog, but also lowers its resolution and changes its range profile.

Tesla’s Autopilot relies mostly on radar.

Depending on the radar unit Uber employed — likely multiple in both front and back to provide 360 degrees of coverage — the range could differ considerably. If it’s meant to complement the lidar, chances are it overlaps considerably, but is built more to identify other cars and larger obstacles.

The radar signature of a person is not nearly so recognizable, but it’s very likely they would have at least shown up, confirming what the lidar detected.

Short and long-range optical cameras. Lidar and radar are great for locating shapes, but they’re no good for reading signs, figuring out what color something is, and so on. That’s a job for visible-light cameras with sophisticated computer vision algorithms running in real time on their imagery.

The cameras on the Uber vehicle watch for telltale patterns that indicate braking vehicles (sudden red lights), traffic lights, crossing pedestrians, and so on. Especially on the front end of the car, multiple angles and types of camera would be used, so as to get a complete picture of the scene into which the car is driving.

Detecting people is one of the most commonly attempted computer vision problems, and the algorithms that do it have gotten quite good. “Segmenting” an image, as it’s often called, generally also involves identifying things like signs, trees, sidewalks and more.

That said, it can be hard at night. But that’s an obvious problem, the answer to which is the previous two systems, which work night and day. Even in pitch darkness, a person wearing all black would show up on lidar and radar, warning the car that it should perhaps slow and be ready to see that person in the headlights. That’s probably why a night-vision system isn’t commonly found in self-driving vehicles (I can’t be sure there isn’t one on the Uber car, but it seems unlikely).

Safety driver. It may sound cynical to refer to a person as a system, but the safety drivers in these cars are very much acting in the capacity of an all-purpose failsafe. People are very good at detecting things, even though we don’t have lasers coming out of our eyes. And our reaction times aren’t the best, but if it’s clear that the car isn’t going to respond, or has responded wrongly, a trained safety driver will react correctly.

Worth mentioning is that there is also a central computing unit that takes the input from these sources and creates its own more complete representation of the world around the car. A person may disappear behind a car in front of the system’s sensors, for instance, and no longer be visible for a second or two, but that doesn’t mean they ceased existing. This goes beyond simple object recognition and begins to bring in broader concepts of intelligence such as object permanence, predicting actions, and the like.

It’s also arguably the most advance and closely guarded part of any self-driving car system and so is kept well under wraps.

It isn’t clear what the circumstances were under which this tragedy played out, but the car was certainly equipped with technology that was intended to, and should have, detected the person and caused the car to react appropriately. Furthermore, if one system didn’t work, another should have sufficed — multiple failbacks are only practical in high stakes matters like driving on public roads.

We’ll know more as Uber, local law enforcement, federal authorities, and others investigate the accident.

Gadgets – TechCrunch Devin Coldewey

Autonomous cars gather up tons of data about the world around them, but even the best computer vision systems can't see through brick and mortar. But by carefully monitoring the reflected light of a laser bouncing off a nearby surface, they might be able to see around corners — that's the idea behind recently published research from Stanford engineers.

Gadgets – TechCrunch Devin Coldewey

 Autonomous cars gather up tons of data about the world around them, but even the best computer vision systems can’t see through brick and mortar. But by carefully monitoring the reflected light of a laser bouncing off a nearby surface, they might be able to see around corners — that’s the idea behind recently published research from Stanford engineers. Read More

Gadgets – TechCrunch Devin Coldewey

 Just a PSA: If you charge your car regularly at a public charge station, you might want to keep an eye out for fraudulent charges on whatever card you use to pay for it. Researchers have found that some charge stations, specifically those that require a dedicated card, “have not implemented basic security mechanisms” like encryption. Read More

Gadgets – TechCrunch Catherine Shu

 Nexar, a startup that makes an AI-based dashcam app to monitor road safety, announced today that it has raised a $30 million Series B led by Ibex Ventures. Participants included Alibaba Innovation Ventures (the venture capital investment arm of Alibaba Group), Nationwide Insurance and returning investors Aleph, Mosaic Ventures, Slow Ventures, True Ventures and Tusk Ventures. Read More

Gadgets – TechCrunch Darrell Etherington

 It’s rare that I pay much attention to automaker infotainment and multimedia system updates at CES – usually there’s too much going on with autonomy, electrification and mobility services to give it much thought. This year, however, Mercedes-Benz had one of the most interesting announcements at the show with its new MBUX smart multimedia system and in-car voice activated… Read More

Gadgets – TechCrunch Devin Coldewey

 It’s one thing for an autonomous car to strut its stuff on smooth, warm California tarmac, and quite another to do so on the frozen winter mix of northern Finland. Martti, a self-driving vehicle system homegrown in Finland, demonstrated just this in a record-setting drive along a treacherous (to normal drivers) Laplandish road. Martti is one of two cars designed by VTT Technical… Read More

Gadgets – TechCrunch Devin Coldewey

 Cars these days are basically computers with wheels, and as with other computers, you’ll probably want to make a few changes to protect against cyberthreats. Trillium, presenting today at Disrupt Berlin’s Startup Battlefield, is looking to be the security solution for in-car computer systems, adding extra encryption, intrusion detection, and other firewall-like features. Read More

Gadgets – TechCrunch Darrell Etherington

 Want to bring Alexa into your vehicle? You now have a few options, including a new $49.99 accessory from voice tech company Speak Music, which is launching Muse, essentially a Bluetooth-enabled Amazon Echo for the car. The small device allows you to access Alexa skills while on the go, using voice commands and the “Alexa” wake word, while connecting to your in-car stereo system… Read More