Welcome to ScenicRules!

ScenicRules is an autonomous driving benchmark designed to evaluate systems against complex, multi-objective specifications in diverse driving contexts. It integrates the following key features:

• Multi-Objective Specification: Supports the formalization of conflicting driving objectives and explicit priority relations using the Rulebook framework.

• Abstract Scenario Representation: Leverages the Scenic probabilistic programming language to model driving contexts in an expressive, compact, and interpretable manner.

ScenicRules is designed and implemented by Kevin Kai-Chun Chang, Ekin Beyazit, Alberto Sangiovanni-Vincentelli, Tichakorn Wongpiromsarn, and Sanjit A. Seshia. See our paper for more details. If you have any questions or suggestions, please feel free to open an issue on our GitHub repository.

Table of Contents

Getting Started with ScenicRules

• Overview
• Installation
  • Install Python
  • Set up a Python Virtual Environment
  • Install ScenicRules
• Tutorial

ScenicRules Internals

• Preprocessing of Simulation Results
  • Collection (bench.scenic)
  • Trajectory Abstraction (realization.py)
  • Processing (process_trajectory.py)
• Rules
  • How to Define a Rule
  • Evaluation and Scoring
• Rulebooks
  • Defining a Rulebook Specification
  • The Rulebook Class Implementation (rulebook.py)
• Scenarios
  • Common Scenarios & Automated Scenario Generator
  • Near-Accident Scenarios & LLM-Assisted Scenic Code Generation

ScenicRules as a Benchmark

• Falsification of Driving Systems
  • I. Interfacing with Scenic
  • II. Defining Rulebook Specifications
  • III. Running Falsification
• Evaluation against Human Preferences
  • Overview of Reasonable Crowd
  • The Evaluation Flow
  • Caching and Optimization
  • How to Run the Script
  • Extending to Other Datasets

Citation

If you use ScenicRules in your work, please cite the following paper:

@inproceedings{chang2026scenicrules,
   title={{ScenicRules}: An Autonomous Driving Benchmark with Multi-Objective Specifications and Abstract Scenarios},
   author={Chang, Kevin Kai{-}Chun and Beyazit, Ekin and Sangiovanni-Vincentelli, Alberto and Wongpiromsarn, Tichakorn and Seshia, Sanjit A.},
   booktitle={IEEE Intelligent Vehicles Symposium (IV)},
   year={2026}
}

Liscense

ScenicRules is licensed under the 3-Clause BSD License.