Enhancing Deep Reinforcement Learning for Robotic Motion Planning via Task Decomposition

Practical Tips

Practical tips and how-to guidance will be added by our editorial team.

中文解读

中文解读待补充：本站将优先为睡眠改善、失眠治疗、助眠方法等高价值文章补充中文说明。

Article Summary

Training robotic manipulation policies in simulation is widely adopted due to safety and cost efficiency; however, tasks involving obstacles remain under-performing and often converge slowly under Deep Reinforcement Learning (DRL). To address this challenge, we propose a task decomposition framework for DRL that integrates Transfer Learning (TL) with Reward Redesign (RR) to improve learning efficiency and robustness in obstacle environments. The TL strategy selectively transfers only the first-layer critic parameters across structurally related tasks, since that layer captures general state--action representations while deeper layers encode reward- and obstacle-specific value geometry; RR decomposes tasks into staged sub-objectives to provide denser optimization signals. Experiments on the PyBullet Panda platform show consistent gains in obstacle settings. On Push with obstacle, our method achieves $0.83 \pm 0.05$, outperforming the strongest baseline (Truncated Quantile Critics (TQC), $0.77 \pm 0.06$) by approximately $7.8\%$. On Slide with obstacle, performance reaches $0.95 \pm 0.02$, exceeding Soft Actor-Critic (SAC) ($0.84 \pm 0.06$) by about $11\%$ success rates. In addition to higher final performance, forward transfer reduces convergence time by up to $70\%$ in obstacle tasks, and RR shortens sparse-reward Pick-and-Place training from roughly $0.75$M to $0.20$M steps ($\approx73\%$ reduction). The best policy is deployed on a real Franka Emika FR3 robot as a qualitative feasibility demonstration.

Sources & References

• Original source
• Full article (PDF)

Comments