End-to-End Autonomous Vehicles

Our Approach

1. End-to-end autonomous driving

   Networks that learn to drive from raw LiDAR, camera, IMU and GNSS data, unifying perception, prediction, planning and control in one differentiable framework.

2. Safety certification and integrity monitoring

   Integrity monitoring quantifies in real time how far the navigation solution can be trusted, so the vehicle can detect unsafe states and trigger fail-safe manoeuvres.

3. Real-world deployment for logistics

   Full-stack vehicle platforms for campus patrol, autonomous delivery and connected fleets, with robust localisation in urban canyons.

Research in Action

Systems we have built and tested with partners. Open a card for the problem, our approach, results and photos.

End-to-End Intelligent Driving System for Logistics

An end-to-end AI driving stack with a trustworthy positioning safety layer, validated on logistics vans with Hong Kong's leading logistics operator.

• 2 MOUs SF Express · Rino.ai
• 2,000+ partner vehicles in 170+ cities
• End-to-end AI driving with a safety layer

Problem, approach & results

ProblemUrban logistics vans must drive safely through dense, GNSS-degraded Hong Kong streets with heavy occlusion and tight kerbside space.

Approach

• End-to-end AI driving stack learned from multi-sensor data
• Trustworthy GNSS/INS/LiDAR positioning acting as a safety layer
• Field validation on a logistics vehicle provided by SF Express (HK) and Rino.ai

Assisted Navigation and Collision Avoidance System using AI and Location-based Services

Lane-level positioning and multi-vehicle collision warnings for Hong Kong roads.

• <0.5 m lane-level error
• Multi-car collision warnings

Problem, approach & results

ProblemTall buildings block satellites and sight lines, so lane-level positioning and blind-spot awareness fail in urban canyons.

Approach

• Lane-level positioning from RTK and 3D city maps
• Multi-vehicle collaborative sensing with AI and RTK for over-the-horizon warnings
• AI dynamic-model optimisation and a multi-vehicle collision-avoidance prototype

ResultsION GNSS+ 2024 paper on factor-graph multi-epoch ambiguity resolution; an AI-based NLOS-mitigation deliverable.

Advanced Smart-Mobility Roadside and Edge System

Roadside sensing and edge computing that share maps and dynamic objects with connected vehicles, with ASTRI.

• 1 car donated by ASTRI
• ITSC 2023 paper
• HKSTP joint field tests

Problem, approach & results

ProblemVehicles alone cannot see around corners or keep HD maps current; roadside units can, if their data are fused correctly.

Approach

• Roadside LiDAR and camera units mapping their surroundings
• Edge computing that shares error maps and dynamic objects with connected vehicles
• Joint field tests with ASTRI vehicles at HKSTP and PolyU

ResultsRoadside error maps shared with connected vehicles (IEEE ITSC 2023).

Multi-Sensory HD Mapping System

A full HD-mapping platform delivered to ASTRI, with vector maps and a digital twin of Hong Kong Science Park.

• Vector map of HKSTP
• Digital twin of Science Park

Problem, approach & results

ProblemAutonomous-driving trials at Science Park needed a centimetre-level HD map and a way to keep it up to date.

Approach

• Vehicle and roadside multi-sensor kit (LiDAR, cameras, GNSS/INS) with time synchronisation
• LiDAR-inertial mapping and lane-level vector-map extraction
• Digital-twin model of HKSTP for simulation and validation

ResultsMapping platform, point-cloud and vector maps of HKSTP, and a digital twin for simulation.

Vehicle–Infrastructure Collaboration for Connected UGVs and UAVs

Collaborative positioning and map update in urban canyons using roadside units, drones and ground vehicles.

• V2I collaborative positioning
• UAV + UGV map update

Problem, approach & results

ProblemSingle-vehicle positioning has hit its limits in urban canyons; roadside units and drones can supply the missing views.

Approach

• Vehicle–infrastructure collaborative positioning using roadside units and smart sensors
• UAV–UGV collaboration for map update
• Validation on TAS Lab vehicles and drones

ResultsSeeded the lab's collision-warning and air–ground map-update research.

Video Demonstrations

Field Demonstrations

Collaborators

We work with industry partners including Huawei, Meituan, Tencent and iDriverplus, the Mechanical Systems Control Lab at UC Berkeley and Chemnitz University of Technology in Germany.

Selected Publications

• Integrated Planning and Control on Manifolds: Factor Graph Representation and Toolkit.
  Yang, P., Wen, W., Yang, R., Zhang, Y., Hu, J., Chen, Y., Xiao, N., Zhao, J.
  IEEE International Conference on Robotics & Automation (ICRA), 2026.
• EIRM-RL: Epistemic Integrity Risk Monitoring Inspired Safe Reinforcement Learning for Trustworthy Autonomous Navigation.
  Zhang, Y., Wang, Y., Wen, W.
  IEEE Internet of Things Journal, 13(2), 3500-3512, 2025. (IF: 8.9, JCR Q1)
• Learning Safe, Optimal, Real-Time Flight Interaction with Deep Confidence-enhanced Reachability Guarantee.
  Zhang, Y., Wang, Y., Yan, P., Wen, W.
  IEEE Transactions on Intelligent Transportation Systems, 2025. (IF: 8.4, JCR Q1)
• Safety-quantifiable Line Feature-based Monocular Visual Localization with 3D Prior Map.
  Zheng, X., Wen, W.*, Hsu, L.T.
  IEEE Transactions on Intelligent Transportation Systems, 2025. (IF: 8.4, JCR Q1, Citations: 3)
• Continuous Error Map Aided Adaptive Multi-Sensor Integration for Connected Autonomous Vehicles in Urban Scenarios.
  Huang, F., Wen, W.*, Zhang, G., Su, D., Huang, Y.
  IEEE Transactions on Instrumentation and Measurement, 2025. (IF: 5.9, JCR Q1, Citations: 4)
• Fault Detection Algorithm for Gaussian Mixture Noises: An Application in Lidar/IMU Integrated Localization Systems.
  Yan, P., Li, Z., Huang, F., Wen, W., Hsu, L.T.
  NAVIGATION: Journal of the Institute of Navigation, 72(1), 2025. (IF: 3.1, JCR Q1, Citations: 6)
• Safety-Quantifiable Planar-Feature-based LiDAR Localization with a Prior Map for Intelligent Vehicles in Urban Scenarios.
  Zhang, J., Liu, X., Wen, W.*, Hsu, L.T.
  IEEE Transactions on Intelligent Vehicles, 2024. (IF: 14.3, JCR Q1, Citations: 2)
• A Novel Consistent-Robust SINS/GNSS/NHC Integrated Navigation Method for Autonomous Vehicles Under Intermittent GNSS Outage.
  Du, S., Huang, Y.*, Wen, W., Zhang, Y.
  IEEE Transactions on Intelligent Vehicles, 2024. (IF: 14.3, JCR Q1, Citations: 13)
• Tightly-coupled Visual/Inertial/Map Integration with Observability Analysis for Reliable Localization of Intelligent Vehicles.
  Zheng, X., Wen, W.*, Hsu, L.T.
  IEEE Transactions on Intelligent Vehicles, 2024. (IF: 14.3, JCR Q1, Citations: 3)
• Integration of Vehicle Dynamic Model and System Identification Model for Extending the Navigation Service Under Sensor Failures.
  Yan, P., Wen, W.*, Hsu, L.T.
  IEEE Transactions on Intelligent Vehicles, 2023. (IF: 14.3, JCR Q1, Citations: 11)
• Dynamic Object-Aware LiDAR Odometry Aided by Joint Weightings Estimation in Urban Areas.
  Huang, F., Wen, W., Zhang, J.*, Wang, C., Hsu, L.T.
  IEEE Transactions on Intelligent Vehicles, 2023. (IF: 14.3, JCR Q1, Citations: 9)
• ECMD: An Event-Centric Multisensory Driving Dataset for SLAM.
  Chen, P., Guan, W., Huang, F., Zhong, Y., Wen, W., Hsu, L.T., Lu, P.*
  IEEE Transactions on Intelligent Vehicles, 2023. (IF: 14.3, JCR Q1, Citations: 31)
• An Improved Inertial Preintegration Model in Factor Graph Optimization for High Accuracy Positioning of Intelligent Vehicles.
  Zhang, L., Wen, W.*, Zhang, T., Hsu, L.T.
  IEEE Transactions on Intelligent Vehicles, 2023. (IF: 14.3, JCR Q1, Citations: 16)
• UrbanLoco: A Full Sensor Suite Dataset for Mapping and Localization in Urban Scenes.
  Wen, W., Zhou, Y., Zhang, G., Fahandezh-Saadi, S., Bai, X., Zhan, W., Tomizuka, M., Hsu, L.T.
  IEEE ICRA 2020, 2310-2316. (Citations: 184)
• UrbanNav: An Open-sourced Multisensory Dataset for Benchmarking Positioning Algorithms Designed for Urban Areas.
  Hsu, L.T., Kubo, N., Wen, W., Chen, W., Liu, Z., Suzuki, T., Meguro, J.
  ION GNSS+ 2021. (Citations: 149)

Full publication list →