Stanford Seminar - Leveraging Physics-Based Models To Learn Generalizable Robotic Manipulation

Introduction

• The presentation aims to answer three questions: what is missing in robotic manipulation, what causes this deficiency, and how physics-based models can help, with a focus on learning generalizable robotic manipulation (11s).
• Despite impressive robotic manipulation demos, the field is far from being solved, and current policies learned from methods are not generalizable or robust (1m20s).
• State-of-the-art algorithms, such as diffusion policy and reinforcement learning, have limitations, including poor generalizability, sensitivity to demonstration data set size, and the need for expensive and tedious tuning processes (2m5s).
• The key to understanding the difficulty of manipulation lies in the rich physical constraints that govern it, including collision and robot reachability constraints, contact modes, force closure, and friction cone (3m2s).

Constraints in Robotic Manipulation

• Collision and robot reachability constraints are kinematic and geometric, depending on the object's shape and the scene, and are highly non-convex, making it difficult to obtain a global solution (3m45s).
• Contact mode describes the contact configuration between the robot, object, and scene, directly impacting the system's dynamics due to hybrid dynamics (4m12s).
• In robotic manipulation, there are three types of constraints: environment contact, force disclosure, and friction cone, which are crucial for tasks like grasping and manipulation, but are generally difficult to work with due to their non-convex and differentiable nature (4m28s).
• Environment contact constraints depend on the shape of the object and the environment, and can be detected and enforced using tactile force sensing and classical control tools like compliant or force control (4m30s).
• Force disclosure and friction cone constraints are mostly concerned with robotic grasping, where force closure describes the sum of external forces and torques equal to zero, and friction cone typically uses the Kum friction model in literature (5m8s).
• These constraints are conditioned on the contact points, and there are rich computational tools like quadratic optimization that allow us to leverage these constraints (5m32s).
• The common attributes of these constraints are that they are non-convex, mostly differentiable, and computationally expensive to work with, making it difficult to obtain a global solution when solving an optimization problem with these constraints (6m0s).
• These constraints are extensively covered by path literature, classical control theory tools, and stem from physics, particularly classical mechanics, kinematics, and dynamics (6m26s).

Leveraging Physics-Based Models

• To overcome the challenges in robotic manipulation, physics-based models can be leveraged by moving expensive analytical computations offline, generating datasets or creating heuristics, and solving problems with learning (6m53s).
• Learning can be used to refine a network output with these constraints, essentially solving a local problem, and can be applied to challenging manipulation tasks like dexterous grasping, dexterous pre-grasping, and extrinsic manipulation (7m38s).

Dexterous Grasping

• Dexterous grasping refers to grasping an object using a multi-fingered hand, which is difficult due to the high degrees of freedom compared to a parallel gripper (8m22s).
• Dexter grasping is a powerful tool for grasping objects, allowing for a wide range of different objects to be grabbed, and providing diverse grasping strategies, such as grasping from the top, side, and other configurations (9m9s).
• A fully accurate four-finger hand has over 20 degrees of freedom, compared to approximately seven degrees of freedom for a parallel gripper, and is subject to more constraints, including collision, reachability, force closure, and friction cone (8m36s).
• A pipeline is proposed to generate Dexter grasping by combining learning and optimization, starting with building a grasp data set with an analytical model, training a generative grasp point predictor, and refining the prediction with local optimization (9m31s).
• The generative grasp point predictor is a conditional variational autoencoder that predicts where to place the fingers on an object, and the local optimization problem is solved to satisfy physical constraints, including collision, robot reachability, force closure, and friction cone (9m49s).
• The pipeline achieves nearly 90% success rate over 20 objects and 120 trials, with objects ranging from those seen during training to those with similar shapes or never seen before (11m8s).
• The pipeline allows for grasping in different configurations, even for the same objects, and enables the robot to use multimodal strategies to pick up objects (11m31s).
• Dexter grasping can be used to pick up objects that cannot be picked up with a parallel gripper, demonstrating the power of Dexter grasping (11m54s).
• The takeaway of the Dexter grasping task is that a grasp data set can be generated with physics, a grasp predictor can be learned, and the prediction can be refined with optimization that considers friction cone and force closure (12m2s).
• The pipeline can also be used to solve for the hand collision, a 22 degrees of freedom problem, using kinematics (12m21s).

Dexterous Pre-grasping

• In real-world scenarios, objects to be picked up may be ungraspable and require movement before a graspable position can be reached, and a pre-grasp is necessary to establish a grasp or rigid linkage to the object (12m44s).
• Identifying a good pre-grasp is challenging because it is defined by the quality of the grasp it can lead to, and computing a pre-grasp is computationally expensive due to factors like finger movement, object contact, and potential collisions (13m14s).
• A key insight is that only one environment contact is needed during a pre-grasp, and only two fingers are required to establish a grasp, allowing for a reduction in search space and the use of model-based methods (13m44s).
• A proposed pipeline involves offline training, constructing a contact state graph, and using model-based methods for optimization and synthesizing hand motion (13m57s).
• Offline training includes learning a grasp generator and a score function to evaluate pre-grasps, which offloads expensive computations to offline processing (14m12s).
• A contact state graph is built based on finger placement on the object surface, and edges represent transitions between contact states (14m49s).
• A scoring function is trained to evaluate contact configurations, and trajectory optimization is performed to find the best path on the graph (15m11s).
• The pipeline is used to plan a contact transition and synthesize full hand motion with kinematics, resulting in physics-realistic trajectories (15m41s).
• The approach is tested in various environments where direct grasping is not possible, and the pipeline achieves efficient pre-grasp planning (16m9s).
• The takeaway from the project is the use of a scoring function to rank contact states, a grasp predictor to complete grasps, and a contact mode to guide pre-grasp planning (16m29s).

Extrinsic Manipulation

• Extrinsic manipulation is a versatile mode of interaction that involves manipulating an object using environment context, and it's challenging due to various contact configurations and unknown factors like friction coefficients (17m3s).
• A divide and conquer approach can be taken by breaking down extrinsic manipulation into primitives based on contact configurations, allowing for the training of robust policies within the same contact configuration using reinforcement learning (17m35s).
• The challenge lies in switching between primitives with different contact requirements, and a framework using physical models can be built to enforce these constraints and stitch the primitives together (18m5s).
• A primitive library has been built with four primitives: pushing, pulling, pivoting, and grasping, which can be used to describe context and allow the robot to move freely between contact transitions (18m44s).
• The framework uses a mixture of classical control tools and learning-based tools, and it can retarget the contact configuration to different scenes and objects using a demonstration and remapping the task (18m33s).
• The robot can execute long-horizon tasks, including up to four different contact configurations, using the same primitives by mapping the contact configuration (20m22s).
• The framework has been demonstrated to achieve the same task on a variety of different objects and environments using a single demonstration and remapping the contact configuration (20m46s).
• A divide and conquer framework can be built using physical models and contact constraints, allowing for the learning of goal-conditioned motion primitives with standard learning tools, and achieving long-horizon tasks that are otherwise impossible (21m1s).

Summary and Discussion

• Three tasks were covered: Des grasping, dextrous grasping, and extrinsic manipulation, all of which used analytical methods to make computation more efficient by moving expensive computation offline and ensuring that learning methods satisfy physical constraints at all times (21m22s).
• The main issues with robotic manipulation are generalizability and robustness, with current policies being mostly fragile due to complex physical constraints (21m56s).
• Physical models can be used to solve these issues through synthetic data set generation, robust constraint satisfaction, composability, and multitask generalization, as shown in the three examples provided (22m13s).
• The Gring project only considered finger contact, which is the minimum needed, and the 14th finger was not used, but the same method could potentially work for more fingers (22m54s).
• The Preg grasp project required two fingers, but in the book case, only one finger was used, because two fingers plus the environment form a rigid caging, but a patch contact on the environment only requires one finger (23m22s).
• The Deen generation pipeline starts with an offline learned grasp generator, followed by a model-based optimization procedure to refine the grasp, and requires a shape or model of the object to run the optimization (24m2s).
• The method can be compared to a purely learning-free approach, which can also generate grasp points using a traditional grasp point generator, but the proposed method is more efficient due to the difficulty of global optimization (25m7s).
• Learning provides a good local initial guess for robotic manipulation, allowing for diverse grasping configurations even for the same observation, and can also see the initial guess differently to achieve this diversity (25m27s).
• The hard part of robotic manipulation is not choosing the initial contact state, but rather the motion planning required to get the object into a state where it can be grasped, which is an accessibility problem as much as a contact problem (26m3s).
• The motion planning part is the most important and computationally expensive aspect, requiring evaluation of the entire motion planning process to decide if a pre-grasp is good or not (26m26s).
• To address this, a score function is learned to provide a guess of how good a pre-grasp will be without having to solve the full motion planning problem, allowing for offline motion planning (26m47s).
• The current success rate of 87% is not enough for real-world applications, and failures are often due to engineering challenges such as the hand overheating or colliding with the table (27m15s).
• Improving the success rate from 87% to 95% or higher requires significant engineering work to address these challenges, often referred to as the "last mile problem" (27m58s).
• While there are ways to improve the success rate, they may be less principled and require more engineering effort (28m14s).