Stanford Seminar - Learning-enabled Adaptation to Evolving Conditions for Robotics

Learning-Enabled Adaptation for Lifelong Deployment of Robots

• Robots can respond to unexpected changes in conditions during deployment through learning-enabled adaptation.
• Distribution shifts, where the input data distribution changes from the training distribution, can lead to incorrect predictions and unsafe outcomes.

Data Life Cycle for Lifelong Deployment

• The data life cycle for lifelong deployment involves input data, model predictions, uncertainty estimation, out-of-distribution detection, data labeling, and model fine-tuning.
• Out-of-distribution detection can be achieved using Bayesian methods such as the Scott algorithm, which estimates functional uncertainty and thresholds it to identify out-of-distribution inputs.
• To adapt to distribution shifts, a subset of out-of-distribution inputs are selected for labeling, and the model is fine-tuned or retrained using these labels.
• The performance-cost trade-off should be considered when selecting data for labeling, as more labeled data improves performance but incurs higher costs.

Diverse Subsampling for Efficient Model Adaptation

• Out-of-distribution (OOD) detection algorithms like SCOTT can identify inputs that are different from the training distribution.
• Subsampling a diverse set of OOD inputs for labeling can improve model performance more efficiently than randomly selecting inputs or selecting based solely on uncertainty.
• The proposed diverse subsampling (DS-SCOTT) algorithm uses information gain as a measure to select diverse inputs for labeling.
• DS-SCOTT outperforms random selection and uncertainty-based selection in terms of the trade-off between labeling cost and model performance improvement.
• DS-SCOTT can be used in a data life cycle to continuously improve model performance by requesting labels for a diverse set of OOD inputs.
• By selecting a diverse subset of 5% or 75% of the data, the model achieves performance comparable to labeling 100% of the data, while only labeling about 50% of it.

Addressing Forgetting and Distribution Shifts

• To address the problem of forgetting original data during continual learning, uncertainty estimates can be used to generate a regularization term that prevents forgetting.
• Runtime monitors can be constructed to distinguish between different classes of distribution shifts.
• Self-supervised methods can be used to find invariant features between in-distribution and out-distribution data, reducing the need for labeled data.

Applications and Implementation

• The method is particularly useful in domains prone to distribution shifts, such as autonomous driving, where factors like rain, lighting, and sensor degradation can affect perception modules.
• The diverse subset selection can be implemented entirely at the edge, and the authors are considering open-sourcing the code.