publications

Deep reinforcement learning (RL) enables agile autonomous navigation from onboard sensory data, but standard continuous and memoryless RL-based policies lack robustness in topologically obstructed spaces. Specifically, when bypassing obstacles in SE(3) or tracking attitudes in SO(3), topological obstructions create decision boundaries where perturbations can induce rapid switching (chattering) between passage routes or rotation directions, trapping the vehicle at unaligned attitudes or causing collisions. To resolve these vulnerabilities, we propose Latent Multi-Policy Hysteresis-Based RL (L-MultiHyRL) for robust, perception-driven obstacle avoidance in SE(3). L-MultiHyRL identifies the critical points of a baseline RL policy, where trajectories locally diverge around an obstacle, and partitions the state space into overlapping, contractible partitions. It then predicts whether the vehicle is inside a partition directly from onboard sensory data and trains a specialized local policy for each partition. Similarly, to resolve the obstruction in SO(3) attitude tracking, it defines overlapping tracking domains for the antipodal quaternion set-points. Finally, a discretized closed-loop hybrid system orchestrates the switching between these local policies and attitude set-points using hysteresis logic to achieve robustness against chattering. Simulations and physical quadrotor experiments demonstrate that L-MultiHyRL is robust against adversarial perturbations that cause a standard continuous RL-based policy to get trapped at unaligned attitudes and crash in single- and multi-obstacle environments.

We address the challenge of switching among multiple learned policies in reinforcement learning control systems, where conventional value function-based methods can lead to chattering in the presence of small measurement noise. Our goal is to design a switching logic that assures asymptotic stability and maintains a robustness margin so that rapid switching is prevented under bounded measurement noise. To this end, we propose a timer-based hybrid supervisor that integrates a resettable timer that enforces a minimum dwell time on the active policy. This dwell time is adaptively adjusted by predicting the evolution of the state of the system, ensuring that a switch occurs only when a significantly better alternative is predicted. We derive sufficient conditions under which the hybrid supervisor is guaranteed to exhibit non-Zeno behavior and render a compact set robustly globally asymptotically stable in the presence of bounded measurement noise. Simulation results on representative decision-making problems demonstrate that our hybrid supervisor is robust under noisy conditions where a conventional switching strategy fails.

Standard continuous reinforcement learning policies lack formal robustness guarantees against adversarial perturbations in the observation space during obstacle avoidance, where topological obstructions create vulnerable decision boundaries. MultiHyRL identifies critical points of a baseline policy and constructs overlapping relaxed Voronoi partitions with hysteresis switching, guaranteeing robust navigation across multi-obstacle environments.

Combines hybrid control theory with reinforcement learning by embedding hysteresis-based switching logic across overlapping state-space regions, eliminating chattering vulnerabilities in obstacle avoidance tasks.

Deep reinforcement learning (RL) has achieved impressive empirical success in complex control tasks, many of which are fundamentally set-stabilization problems. To safely deploy learned policies, existing methods often modify RL algorithms to enforce formal stability guarantees. However, these algorithmic modifications alter the core RL objective, which can restrict natural exploration and complicate training. This paper offers a complementary perspective: while such algorithmic modifications are valuable for training-time safety during exploration, modifying the core RL objective is often unnecessary to guarantee the stability of the deployed policy. We prove that the unmodified discounted RL objective intrinsically guarantees closed-loop practical asymptotic stability in deterministic continuous control settings. By adapting the discrete-time input-to-state stability (ISS) framework and drawing inspiration from discounted model predictive control (MPC), we model the suboptimality induced by the discount factor and the neural network approximation error as bounded disturbances. We establish that near-optimal parameterized feedback laws steer the closed-loop system into a bounded neighborhood of the target set. The size of this neighborhood vanishes in the limit as the approximation error approaches zero and, subsequently, the discount factor approaches one.

In robotic manipulation tasks involving contact, switching between force and position controllers can lead to undesired oscillations in contact forces and manipulator position. This paper presents an algorithm that enables a 6-link manipulator operating in three-dimensional space to robustly approach, grasp, and manipulate objects at predetermined locations under real-world noise and perturbations. The method incorporates hysteresis to mitigate chattering effects during contact with the object, thereby improving end effector stability and force regulation. Experimental results validate the effectiveness of the proposed algorithm through comparison with a discontinuous switching baseline, demonstrating improved robustness for robotic applications requiring stable manipulation.

Investigates thermo-viscous acoustic attenuation and stop-band optimization in micro-slit resonant metamaterials for low-frequency noise control.