The End Of Knowledge - Vault 5/12 - CVPR - 2016 - Three Pillars of Autonomous Driving

graph LR classDef autonomous fill:#f9d4d4, font-weight:bold, font-size:14px classDef sensing fill:#d4f9d4, font-weight:bold, font-size:14px classDef mapping fill:#d4d4f9, font-weight:bold, font-size:14px classDef policy fill:#f9f9d4, font-weight:bold, font-size:14px classDef future fill:#f9d4f9, font-weight:bold, font-size:14px A[Three Pillars of
Autonomous Driving] --> B[Autonomous driving: vehicle utilization,
safety, mobility, content, AI/robotics. 1] A --> C[Three pillars: sensing, mapping, policy. 2] C --> D[Sensing: environmental model,
objects, obstacles, paths, cameras. 3] D --> E[3D object detection,
pedestrian actions needed. 4] D --> F[Free space, path detection
via deep learning context. 5] C --> G[Sensing: present, single-agent, predictable.
Policy: future, multi-agent, less predictable. 6] G --> H[Policy learns human behaviors,
assertiveness via reinforcement learning. 7] C --> I[Mapping: redundancy, foresight, control.
Types: none to hi-def. 8] I --> J[Hi-def maps: real-time updates,
10cm localization, crowdsourced. 9] J --> K[Crowdsourced maps: sparse 3D
landmarks, dense 1D road. 10] I --> L[Localization + map info
enables vehicle control, redundancy. 11] B --> M[Current autopilot: unsafe lateral control.
Real autonomy coming in stages. 12] M --> N[2018-2020: highly autonomous
highway driving, safe stop. 13] M --> O[2021: fully autonomous SAE 4/5
geofenced ride-sharing fleets. 14] M --> P[2023+: autonomy everywhere
if proven safe. 15] P --> Q[Enables shared ownership,
subscriptions, transportation disruption. 15] class A,B,M,N,O,P,Q autonomous class C,D,E,F,G sensing class H policy class I,J,K,L mapping

Resume:

1.- Autonomous driving is important for better vehicle utilization, safety, shared mobility, providing passenger content, and developing AI/robotics with clear business models.

2.- Three pillars enable autonomous driving: sensing, mapping, and driving policy. They should be developed simultaneously as they are intertwined.

3.- Sensing (cameras, radar, lidar) builds an environmental model of moving objects, obstacles, path boundaries, and drivable paths. Cameras provide necessary resolution and appearance information.

4.- Moving object detection requires 3D bounding boxes, not just 2D. Pedestrian detection needs details on actions. This is incremental progress over current tech.

5.- Free space/path delimiter detection is a small leap using deep learning to understand context. Drivable path detection with lane semantics is very complex.

6.- Sensing is about the present, single-agent, predictable. Planning/driving policy is future-focused, multi-agent, less predictable. It requires reinforcement learning.

7.- Driving policy learns human driving behaviors to enable appropriate assertiveness through reinforcement learning in simulated and real environments. It handles negotiations in merging, intersections, etc.

8.- Mapping provides redundancy, foresight, and control ability for autonomy. Map types range from none to navigation maps to hi-def maps to 3D point clouds.

9.- Hi-def maps need real-time updates and 10cm localization, so they must be crowdsourced from production vehicles sending minimal data. This leverages regulatory front-facing cameras.

10.- Crowdsourced maps composed of sparse 3D landmarks (signs, poles, etc.) and dense 1D road model elements, using ego-motion to handle landmark gaps, enabling 10cm localization.

11.- Accurate localization in the hi-def map, combined with map information, enables lateral and longitudinal vehicle control for autonomy, providing sensing redundancy.

12.- Current highway autopilot (e.g. Tesla) is not autonomous - it's unsafe lateral control. Real autonomy is coming in stages:

13.- 2018-2020 will bring highly autonomous highway driving with 360° awareness, safe stop if driver doesn't take over. Real autonomy jump is 2021+.

14.- 2021 will enable fully autonomous driving (SAE Level 4/5) for geofenced ride-sharing fleets, enabling driverless Uber/Lyft, new business models.

15.- 2023+ could allow autonomy everywhere once safety is proven. Enables shared ownership across households, subscription models, transportation disruption.

Knowledge Vault built byDavid Vivancos 2024