Back to Search Start Over

RobotPerf: An Open-Source, Vendor-Agnostic, Benchmarking Suite for Evaluating Robotics Computing System Performance

RobotPerf: An Open-Source, Vendor-Agnostic, Benchmarking Suite for Evaluating Robotics Computing System Performance

Authors :
Mayoral-Vilches, Víctor
Jabbour, Jason
Hsiao, Yu-Shun
Wan, Zishen
Crespo-Álvarez, Martiño
Stewart, Matthew
Reina-Muñoz, Juan Manuel
Nagras, Prateek
Vikhe, Gaurav
Bakhshalipour, Mohammad
Pinzger, Martin
Rass, Stefan
Panigrahi, Smruti
Corradi, Giulio
Roy, Niladri
Gibbons, Phillip B.
Neuman, Sabrina M.
Plancher, Brian
Reddi, Vijay Janapa
Publication Year :
2023

Abstract

We introduce RobotPerf, a vendor-agnostic benchmarking suite designed to evaluate robotics computing performance across a diverse range of hardware platforms using ROS 2 as its common baseline. The suite encompasses ROS 2 packages covering the full robotics pipeline and integrates two distinct benchmarking approaches: black-box testing, which measures performance by eliminating upper layers and replacing them with a test application, and grey-box testing, an application-specific measure that observes internal system states with minimal interference. Our benchmarking framework provides ready-to-use tools and is easily adaptable for the assessment of custom ROS 2 computational graphs. Drawing from the knowledge of leading robot architects and system architecture experts, RobotPerf establishes a standardized approach to robotics benchmarking. As an open-source initiative, RobotPerf remains committed to evolving with community input to advance the future of hardware-accelerated robotics.

Subjects

Subjects :
Computer Science - Robotics

Details

Database :
arXiv
Publication Type :
Report
Accession number :
edsarx.2309.09212
Document Type :
Working Paper