Publications

[google scholar]


all task planning motion planning multi-robot multi-object manipulation hri perception applied

Collaborative automation is a broad theme of research that brings together robotic capabilities of long-horizon planning, multi-agent coordination, human-robot interactions, perception, learning, and control to push towards teams of robots and humans working together to solve shared tasks in real environments.

Task planning enables robots to reason over a sequence of valid discrete actions to change the state of the world towards a goal defined by the task objective. In the robotics problems of interest this reasoning is typically used within the broader framework of task and motion planning.

Motion planning pertains to computing continous valid motions of a robot that can take it from an initial configuration to a goal region. A robot configuration can be represented by all the degrees of freedom, for e.g. joint angles of a manipulator. In the robotics problems of interest this reasoning can be used to continuously interact with the world over task-level actions within task and motion planning.

Multi-robot and multi-agent reasoning allows teams of robots to work together, coordinate, and collaborate to move together and achieve shared motion and task objectives. Multi-robot reasoning retains all the complexity of underlying planning problems while introducing significant challenges arising from the need to reason over the different robots interacting with each other.

Multi-object reasoning introduces a class of problems where a robot can interact with multiple objects with actions and motions to change the state of the objects towards achieving a task objective. This encompasses problem categories including object rearrangement and task and motion planning.

Manipulation involves reasoning about a category of robots that are capable of manipulating objects in the environment, with common forms being robotic arms. Manipulation planning is commonly studied as a class of problems involving robots performing object interactions using manipulation actions like pick-and-place, grasping, and pushing.

Human-robot interaction studies the research problems that arise at the junction of robotic reasoning and human factors. Pushing towards collaborative teams involves both planning methods that can incorporate reasoning about human collaborators, as well as elements of cognitive science, modeling, and learning. Core considerations here include designing ethical AI systems that can avoid introducing biases within human interactions.

Perception describes learning and sensing problems that are used to create a representation of the world that can reasoned over. World representation is sometimes implicit within broader learning-based methods, or can be explicit to apply planning to enact changes to the model of the world.

Applied robotics challenges uses robotics and automation within real-world use cases. Robotics is often used as a tool of protoype, deploy, monitor, and test effective automation systems that enhance the efficiency and safety of existing application domains.



Robots as AI Double Agents: Privacy in Motion Planning
Rahul Shome and Kingston, Zachary and Kavraki, Lydia E.
2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
Robotics and automation are poised to change the landscape of home and work in the near future. Robots are adept at deliberately moving, sensing, and interacting with their environments. The pervasive use of this technology promises societal and economic payoffs due to its capabilities—conversely, the capabilities of robots to move within and sense the world around them is susceptible to abuse. Robots, unlike typical sensors, are inherently autonomous, active, and deliberate. Such automated agents can become AI double agents liable to violate the privacy of coworkers, privileged spaces, and other stakeholders. In this work we highlight the understudied and inevitable threats to privacy that can be posed by the autonomous, deliberate motions and sensing of robots. We frame the problem within broader sociotechnological questions alongside a comprehensive review. The privacy-aware motion planning problem is formulated in terms of cost functions that can be modified to induce privacy-aware behavior— preserving, agnostic, or violating. Simulated case studies in manipulation and navigation, with altered cost functions, are used to demonstrate how privacy-violating threats can be easily injected, sometimes with only small changes in performance (solution path lengths). Such functionality is already widely available. This preliminary work is meant to lay the foundations for near-future, holistic, interdisciplinary investigations that can address questions surrounding privacy in intelligent robotic behaviors determined by planning algorithms.
Optimal Grasps and Placements for Task and Motion Planning in Clutter
Quintero-Pena, Carlos and Kingston, Zachary and Pan, Tianyang and Rahul Shome and Kyrillidis, Anastasios and Kavraki, Lydia E.
2023 IEEE International Conference on Robotics and Automation (ICRA)
Many methods that solve robot planning problems, such as task and motion planners, employ discrete symbolic search to find sequences of valid symbolic actions that are grounded with motion planning. Much of the efficacy of these planners lies in this grounding—bad placement and grasp choices can lead to inefficient planning when a problem has many geometric constraints. Moreover, grounding methods such as naive sampling often fail to find appropriate values for these choices in the presence of clutter. Towards efficient task and motion planning, we present a novel optimization-based approach for grounding to solve cluttered problems that have many constraints that arise from geometry. Our approach finds an optimal grounding and can provide feedback to discrete search for more effective planning. We demonstrate our method against baseline methods in complex simulated environments.
Efficient Inference of Temporal Task Specifications from Human Demonstrations using Experiment Design
Sobti, Shlok and Rahul Shome and Kavraki, Lydia E.
2023 IEEE International Conference on Robotics and Automation (ICRA)
Robotic deployments in human environments have motivated the need for autonomous systems to be able to interact with humans and solve tasks effectively. Human demonstrations of tasks can be used to infer underlying task specifications, commonly modeled with temporal logic. State-of-the-art methods have developed Bayesian inference tools to estimate a temporal logic formula from a sequence of demonstrations. The current work proposes the use of experiment design to choose environments for humans to perform these demonstrations. This reduces the number of demonstrations needed to estimate the unknown ground truth formula with low error. A novel computationally efficient strategy is proposed to generate informative environments by using an optimal planner as the model for the demonstrator. Instead of evaluating all possible environments, the search space reduces to the placement of informative orderings of likely eventual goals along an optimal planner's solution. A human study with 600 demonstrations from 20 participants for 4 tasks on a {2D} interface validates the proposed hypothesis and empirical performance benefit in terms of convergence and error over baselines. The human study dataset is also publicly shared. .
Failure is an option: Task and Motion Planning with Failing Executions
Pan, Tianyang and Wells, Andrew M. and Rahul Shome and Kavraki, Lydia E.
2022 IEEE International Conference on Robotics and Automation (ICRA)
Future robotic deployments will require robots to be capable of solving a variety of tasks in a specific domain, and be able to repeatedly service such requests. Task and motion planning addresses complex robotic problems that combine discrete reasoning over states and actions and geometric interactions during action executions. Moving beyond deterministic settings, stochastic actions can be handled by modeling the problem as a Markov Decision Process. The underlying probabilities however are typically hard to model since failures might be caused by hardware imperfections, sensing noise, or physical interactions. We propose a framework to address a task and motion planning setting where actions can fail during execution. For a task goal to be achieved actions need to be computed and executed despite failures. The robot has to infer what actions are robust and for each new problem effectively choose a solution that reduces expected execution failures. The key idea is to continually recover and refine the underlying beliefs associated with actions across multiple different problems in the domain to find solutions that reduce the expected number of discrete, executed actions. Results in physics-based simulation indicate that our method outperforms baseline replanning strategies to deal with failing executions. .
Failure is an option: Task and Motion Planning with Failing Executions
Pan, Tianyang and Wells, Andrew M. and Rahul Shome and Kavraki, Lydia E.
2022 IEEE International Conference on Robotics and Automation (ICRA)
Future robotic deployments will require robots to be capable of solving a variety of tasks in a specific domain, and be able to repeatedly service such requests. Task and motion planning addresses complex robotic problems that combine discrete reasoning over states and actions and geometric interactions during action executions. Moving beyond deterministic settings, stochastic actions can be handled by modeling the problem as a Markov Decision Process. The underlying probabilities however are typically hard to model since failures might be caused by hardware imperfections, sensing noise, or physical interactions. We propose a framework to address a task and motion planning setting where actions can fail during execution. For a task goal to be achieved actions need to be computed and executed despite failures. The robot has to infer what actions are robust and for each new problem effectively choose a solution that reduces expected execution failures. The key idea is to continually recover and refine the underlying beliefs associated with actions across multiple different problems in the domain to find solutions that reduce the expected number of discrete, executed actions. Results in physics-based simulation indicate that our method outperforms baseline replanning strategies to deal with failing executions. .
A General Task and Motion Planning Framework For Multiple Manipulators
Pan, Tianyang and Wells, Andrew M. and Rahul Shome and Kavraki, Lydia E.
2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
Many manipulation tasks combine high-level discrete planning over actions with low-level motion planning over continuous robot motions. Task and motion planning (TMP) provides a powerful general framework to combine discrete and geometric reasoning, and solvers have been previously proposed for single-robot problems. Multi-robot TMP expands the range of TMP problems that can be solved but poses significant challenges when considering scalability and solution quality. We present a general TMP framework designed for multiple robotic manipulators. This is based on two contributions. First, we propose an optimal task planner designed to support simultaneous discrete actions. Second, we introduce an intermediate scheduler layer between task planner and motion planner to evaluate alternate robot assignments to these actions. This aggressively explores the search space and typically reduces the number of expensive task planning calls. Several benchmarks with a rich set of actions for two manipulators are evaluated. We show promising results in scalability and solution quality of our TMP framework with the scheduler for up to six objects. A demonstration indicates scalability to up to five robots. .
A Sampling-based Motion Planning Framework for Complex Motor Actions
Sobti, Shlok and Rahul Shome and Chaudhuri, Swarat and Kavraki, Lydia E.
2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
We present a framework for planning complex motor actions such as pouring or scooping from arbitrary start states in cluttered real-world scenes. Traditional approaches to such tasks use dynamic motion primitives (DMPs) learned from human demonstrations. We enhance a recently proposed state-of-the-art DMP technique capable of obstacle avoidance by including them within a novel hybrid framework. This complements DMPs with sampling-based motion planning algorithms, using the latter to explore the scene and reach promising regions from which a DMP can successfully complete the task. Experiments indicate that even obstacle-aware DMPs suffer in task success when used in scenarios which largely differ from the trained demonstration in terms of the start, goal, and obstacles. Our hybrid approach significantly outperforms obstacle-aware DMPs by successfully completing tasks in cluttered scenes for a pouring task in simulation. We further demonstrate our method on a real robot for pouring and scooping .
Asymptotically Optimal Kinodynamic Planning Using Bundles of Edges
Rahul Shome and Lydia Kavraki
2021 IEEE International Conference on Robotics and Automation (ICRA)
Using sampling to estimate the connectivity of high-dimensional configuration spaces has been the theoretical underpinning for effective sampling-based motion planners. Typical strategies either build a roadmap, or a tree as the underlying search structure that connects sampled configurations, with a focus on guaranteeing completeness and optimality as the number of samples tends to infinity. Roadmap-based planners allow preprocessing the space, and can solve multiple kinematic motion planning problems, but need a steering function to connect pairwise-states. Such steering functions are difficult to define for kinodynamic systems, and limit the applicability of roadmaps to motion planning problems with dynamical systems. Recent advances in the analysis of single-query tree-based planners has shown that forward search trees based on random propagations are asymptotically optimal. The current work leverages these recent results and proposes a multi-query framework for kinodynamic planning. Bundles of kinodynamic edges can be sampled to cover the state space before the query arrives. Then, given a motion planning query, the connectivity of the state space reachable from the start can be recovered from a forward search tree reasoning about a local neighborhood of the edge bundle from each tree node. The work demonstrates theoretically that considering any constant radial neighborhood during this process is sufficient to guarantee asymptotic optimality. Experimental validation in five and twelve dimensional simulated systems also highlights the ability of the proposed edge bundles to express high-quality kinodynamic solutions. Our approach consistently finds higher quality solutions compared to SST, and RRT, often with faster initial solution times. The strategy of sampling kinodynamic edges is demonstrated to be a promising new paradigm. .
Fast, High-Quality Two-Arm Rearrangement in Synchronous, Monotone Tabletop Setups
Rahul Shome and Solovey, K. and Yu, J. and Bekris, K. E. and Halperin, D.
2021 IEEE Transactions on Automation Science and Engineering
Rearranging objects on a planar surface arises in a variety of robotic applications, such as product packaging. Using two arms can improve efficiency but introduces new computational challenges. This paper studies the problem structure of \revisions{object rearrangement using two arms in} synchronous, monotone tabletop setups and develops an optimal mixed integer model. It then describes an efficient and scalable algorithm, which first minimizes the cost of object transfers and then of moves between objects. This is motivated by the fact that, asymptotically, object transfers dominate the cost of solutions. Moreover, a lazy strategy minimizes the number of motion planning calls and results in significant speedups. Theoretical arguments support the benefits of using two arms and indicate that synchronous execution, in which the two arms perform together either transfers or moves, introduces only a small overhead. Experiments support these claims and show that the scalable method can quickly compute solutions close to the optimal for the considered setup. .
Roadmaps for Robot Motion Planning With Groups of Robots
Rahul Shome
2021 Current Robotics Reports
Autonomous robotic systems require the core capability of planning motions and actions. Centralized motion planning exhibits significant challenges when applied to multi-robot problems. Reasoning about groups of robots typically cause an exponential increase in the size of the search space that an algorithm has to explore. Moreover each robot by itself might be an articulated mechanism with a large number of controllable joints, or degrees of freedom which can pose its own difficulties in planning. Roadmaps have been a popular graph-based method of representing the connectivity of valid motions in such large search spaces including specialized variants for multi-robot motion planning. This article primarily covers recent algorithmic advances that are based on roadmaps for motion planning, with specific optimizations necessary for the multi-robot domain. The structure of the multi-robot problem domain leads to efficient graphical decomposition of the problem on roadmaps. These algorithms provide some desired theoretical properties of being guaranteed to find a solution, as well as optimality of the discovered solution. Extensions to richer planning applications are also discussed. The design of efficient multi-robot planning algorithms like the roadmap-based ones discussed in this article provides the cornerstone for the deployment of large-scale multi-robot teams to solve real-world problems. .
Synchronized Multi-Arm Rearrangement Guided by Mode Graphs with Capacity Constraints
Rahul Shome and Bekris, Kostas E.
2021 Springer Proceedings in Advanced Robotics, Vol 17. 243-260
Solving task planning problems involving multiple objects and multiple robotic arms poses scalability challenges. Such problems involve not only coordinating multiple high-DoF arms, but also searching through possible sequences of actions including object placements, and handoffs. The current work identifies a useful connection between multi-arm rearrangement and recent results in multi-body path planning on graphs with vertex capacity constraints. Solving a synchronized multi-arm rearrangement at a high-level involves reasoning over a modal graph, where nodes correspond to stable object placements and object transfer states by the arms. Edges of this graph correspond to pick, placement and handoff operations. The objects can be viewed as pebbles moving over this graph, which has capacity constraints. For instance, each arm can carry a single object but placement locations can accumulate many objects. Efficient integer linear programming-based solvers have been proposed for the corresponding pebble problem. The current work proposes a heuristic to guide the task planning process for synchronized multi-arm rearrangement. Results indicate good scalability to multiple arms and objects, and an algorithm that can find high-quality solutions fast and exhibiting desirable anytime behavior.
Pushing the Boundaries of Asymptotic Optimality in Integrated Task and Motion Planning
Rahul Shome, Daniel Nakhimovich and Bekris, Kostas E.
2021 Springer Proceedings in Advanced Robotics, Vol 17. 467-484
Integrated task and motion planning problems describe a multi-modal state space, which is often abstracted as a set of smooth manifolds that are connected via sets of transitions states. One approach to solving such problems is to sample reachable states in each of the manifolds, while simultaneously sampling transition states. Prior work has shown that in order to achieve asymptotically optimal (AO) solutions for such piecewise-smooth task planning problems, it is sufficient to double the connection radius required for AO sampling-based motion planning. This was shown under the assumption that the transition sets themselves are smooth. The current work builds upon this result and demonstrates that it is sufficient to use the same connection radius as for standard AO motion planning. Furthermore, the current work studies the case that the transition sets are non-smooth boundary points of the valid state space, which is frequently the case in practice, such as when a gripper grasps an object. This paper generalizes the notion of clearance that is typically assumed in motion and task planning to include such individual, potentially non-smooth transition states. It is shown that asymptotic optimality is retained under this generalized regime.
Asymptotically Optimal Sampling-Based Planners
Bekris, Kostas E. and Rahul Shome
2021 Encyclopedia of Robotics
An asymptotically optimal sampling-based planner employs sampling to solve robot motion planning problems and returns paths with a cost that converges to the optimal solution cost, as the number of samples approaches infinity. This comprehensive article covers the theoretical characteristics of asymptotic optimality of motion planning algorithms, and traces its origins, analysis models, practical performance, extensions, and applications.
Task-driven Perception and Manipulation for Constrained Placement of Unknown Objects
Chaitanya Mitash, Rahul Shome, Bowen Wen, Abdeslam Boularias, and Kostas Bekris
2020 IEEE Robotics and Automation Letters
Recent progress in robotic manipulation has dealt with the case of previously unknown objects in the context of relatively simple tasks, such as bin-picking. Existing methods for more constrained problems, however, such as deliberate placement in a tight region, depend more critically on shape information to achieve safe execution. This work deals with pick-and-constrained placement of objects without access to geometric models. The objective is to pick an object and place it safely inside a desired goal region without any collisions, while minimizing the time and the sensing operations required to complete the task. An algorithmic framework is proposed for this purpose, which performs manipulation planning simultaneously over a conservative and an optimistic estimate of the object’s volume. The conservative estimate ensures that the manipulation is safe while the optimistic estimate guides the sensor-based manipulation process when no solution can be found for the conservative estimate. To maintain these estimates and dynamically update them during manipulation, objects are represented by a simple volumetric representation, which stores sets of occupied and unseen voxels. The effectiveness of the proposed approach is demonstrated by developing a robotic system that picks a previously unseen object from a table-top and places it in a constrained space. The system comprises of a dual-arm manipulator with heterogeneous end-effectors and leverages hand-offs as a re-grasping strategy. Real-world experiments show that straightforward pick-sense-and-place alternatives frequently fail to solve pick-and-constrained placement problems. The proposed pipeline, however, achieves more than 95% success rate and faster execution times as evaluated over multiple physical experiments. .
The Problem Of Many: Efficient Multi-arm, Multi-object Task And Motion Planning With Optimality Guarantees
Rahul Shome
2020 PhD Dissertation, Department of Computer Science, Rutgers University, New Brunswick, USA.
This thesis deals with task and motion planning challenges, specifically those involving manipulating multiple objects using multiple robot manipulators. The contributions range from a new foundational understanding of the problem and the conditions for achieving asymptotic optimality to devising application-oriented and efficient planning algorithms as well as experiments on real systems. A key focus corresponds to overcoming scalability challenges in motion planning and dealing with hybrid planning domains, i.e., those that combine continuous and discrete action spaces, to solve manipulation problems that involve multiple types of actions, such as picks, placements and handoffs. .
That and There: Judging the Intent of Pointing Actions with Robotic Arms
Alikhani, Malihe and Khalid, Baber and Rahul Shome and Mitash, Chaitanya and Bekris, Kostas E. and Stone, Matthew
2020 Thirty-Fourth AAAI Conference on Artificial Intelligence
Collaborative robotics requires effective communication between a robot and a human partner. This work proposes a set of interpretive principles for how a robotic arm can use pointing actions to communicate task information to people by extending existing models from the related literature. These principles are evaluated through studies where English-speaking human subjects view animations of simulated robots instructing pick-and-place tasks. The evaluation distinguishes two classes of pointing actions that arise in pick-and-place tasks: referential pointing (identifying objects) and spatial pointing (identifying locations). The study indicates that human subjects show greater flexibility in interpreting the intent of referential pointing compared to spatial pointing, which needs to be more deliberate. The results also demonstrate the effects of variation in the environment and task context on the interpretation of pointing. The corpus and the experiments described in this work can impact models of context and coordination as well as the effect of common sense reasoning in human-robot interactions.
Anytime Motion Planning for Prehensile Manipulation in Dense Clutter
Kimmel, Andrew and Rahul Shome and Bekris, Kostas E.
2020 Advanced Robotics
Many methods have been developed for planning the motion of robotic arms for picking and placing, ranging from local optimization to global search techniques, which are effective for sparsely placed objects. Dense clutter, however, still adversely affects the success rate, computation times, and quality of solutions in many real-world setups. The proposed method achieves high success ratio in clutter with anytime performance by returning solutions quickly and improving their quality over time. The method first explores the lower dimensional end effector's task space efficiently by ignoring the arm, and build a discrete approximation of a navigation function. This is performed online, without prior knowledge of the scene. Then, an informed sampling-based planner for the entire arm uses Jacobian-based steering to reach promising end effector poses given the task space guidance. This process is also comprehensive and allows the exploration of alternative paths over time if the task space guidance is misleading. This paper evaluates the proposed method against alternatives in picking or placing tasks among varying amounts of clutter for a variety of robotic manipulators with different end-effectors. The results suggest that the method reliably provides higher quality solution paths quicker, with a higher success rate relative to alternatives.
Towards Robust Product Packing with a Minimalistic End-Effector
Rahul Shome and Tang, W. N. and Song, C. and Mitash, C. and Kourtev, C. and Yu, J. and Boularias, A. and Bekris, K. E.
2019 IEEE International Conference on Robotics and Automation (ICRA)
Advances in sensor technologies, object detection algorithms, planning frameworks and hardware designs have motivated the deployment of robots in warehouse automation. A variety of such applications, like order fulfillment or packing tasks, require picking objects from unstructured piles and carefully arranging them in bins or containers. Desirable solutions need to be low-cost, easily deployable and controllable, making minimalistic hardware choices desirable. The challenge in designing an effective solution to this problem relates to appropriately integrating multiple components, so as to achieve a robust pipeline that minimizes failure conditions. The current work proposes a complete pipeline for solving such packing tasks, given access only to RGB-D data and a single robot arm with a minimalistic, vacuum-based end-effector. To achieve the desired level of robustness, three key manipulation primitives are identified, which take advantage of the environment and simple operations to successfully pack multiple cubic objects. The overall approach is demonstrated to be robust to execution and perception errors. The impact of each manipulation primitive is evaluated by considering different versions of the proposed pipeline that incrementally introduce reasoning about object poses and corrective manipulation actions.
Anytime Multi-arm Task and Motion Planning for Pick-and-Place of Individual Objects via Handoffs
Rahul Shome and Kostas E. Bekris
2019 2nd IEEE International Symposium on Multi-Robot and Multi-Agent Systems (MRS)
Automation applications are pushing the deployment of many high DoF manipulators in warehouse and manufacturing environments. This has motivated many efforts on optimizing manipulation tasks involving a single arm. Coordinating multiple arms for manipulation, however, introduces additional computational challenges arising from the increased DoFs, as well as the combinatorial increase in the available operations that many manipulators can perform, including handoffs between arms. The focus here is on the case of pick-and-place tasks, which require a sequence of handoffs to be executed, so as to achieve computational efficiency, asymptotic optimality and practical anytime performance. The paper leverages recent advances in multi-robot motion planning for high DoF systems to propose a novel multi-modal extension of the dRRT* algorithm. The key insight is that, instead of naively solving a sequence of motion planning problems, it is computationally advantageous to directly explore the composite space of the integrated multi-arm task and motion planning problem, given input sets of possible pick and handoff configurations. Asymptotic optimality guarantees are possible by sampling additional picks and handoffs over time. The evaluation shows that the approach finds initial solutions fast and improves their quality over time. It also succeeds in finding solutions to harder problem instances relative to alternatives and can scale effectively as the number of robots increases.
dRRT*: Scalable and informed asymptotically-optimal multi-robot motion planning
Rahul Shome and Solovey, Kiril and Dobson, Andrew and Halperin, Dan and Bekris, Kostas E.
2019 Autonomous Robots
Many exciting robotic applications require multiple robots with many degrees of freedom, such as manipulators, to coordinate their motion in a shared workspace. Discovering high-quality paths in such scenarios can be achieved, in principle, by exploring the composite space of all robots. Sampling-based planners do so by building a roadmap or a tree data structure in the corresponding configuration space and can achieve asymptotic optimality. The hardness of motion planning, however, renders the explicit construction of such structures in the composite space of multiple robots impractical. This work proposes a scalable solution for such coupled multi-robot problems, which provides desirable path-quality guarantees and is also computationally efficient. In particular, the proposed dRRT* is an informed, asymptotically-optimal extension of a prior sampling-based multi-robot motion planner, dRRT. The prior approach introduced the idea of building roadmaps for each robot and implicitly searching the tensor product of these structures in the composite space. This work identifies the conditions for convergence to optimal paths in multi-robot problems, which the prior method was not achieving. Building on this analysis, dRRT is first properly adapted so as to achieve the theoretical guarantees and then further extended so as to make use of effective heuristics when searching the composite space of all robots. The case where the various robots share some degrees of freedom is also studied. Evaluation in simulation indicates that the new algorithm, dRRT* converges to high-quality paths quickly and scales to a higher number of robots where various alternatives fail. This work also demonstrates the planner's capability to solve problems involving multiple real-world robotic arms.
Exploring the Utility of Robots in Exposure Studies
Elisabeth Feld-Cook and Rahul Shome and Rosemary Zaleski and Krishnan Mohan and Hristiyan Kourtev and Kostas E. Bekris and Clifford Weisel and Jennifer Shin
2019 Journal of exposure science & environmental epidemiology
Obtaining valid, reliable quantitative exposure data can be a significant challenge for industrial hygienists, exposure scientists, and other health science professionals. In this proof-of-concept study, a robotic platform was programmed to perform a simple task as a plausible alternative to human subjects in exposure studies for generating exposure data. The use of robots offers several advantages over the use of humans. Research can be completed more efficiently and there is no need to recruit, screen, or train volunteers. In addition, robots can perform tasks repeatedly without getting tired allowing for collection of an unlimited number of measurements using different chemicals to assess exposure impacts from formulation changes and new product development. The use of robots also eliminates concerns with intentional human exposures while removing health research ethics review requirements which are time consuming. In this study, a humanoid robot was programmed to paint drywall, while volatile organic compounds were measured in air for comparison to model estimates. The measured air concentrations generally agreed with more advanced exposure model estimates. These findings suggest that robots have potential as a methodology for generating exposure measurements relevant to human activities, but without using human subjects.
Fast, Anytime Motion Planning for Prehensile Manipulation in Clutter
Kimmel, A. and Rahul Shome and Littlefield, Z. and Bekris, K. E.
2018 2018 IEEE-RAS 18th International Conference on Humanoid Robots
Many methods have been developed for planning the motion of robotic arms for picking and placing, ranging from local optimization to global search techniques, which are effective for sparsely placed objects. Dense clutter, however, still adversely affects the success rate, computation times, and quality of solutions in many real-world setups. The current work integrates tools from existing methodologies and proposes a framework that achieves high success ratio in clutter with anytime performance by returning solutions quickly and improving their quality over time, measured in terms of end effector's displacement. The idea is to first explore the lower dimensional end effector's task space efficiently by ignoring the arm, and build a discrete approximation of a navigation function, which guides the end effector towards the set of available grasps or object placements. This is performed online, without prior knowledge of the scene. Then, an informed sampling-based planner for the entire arm uses Jacobian-based steering to reach promising end effector poses given the task space guidance. While informed, the method is also comprehensive and allows the exploration of alternative paths over time if the task space guidance does not lead to a solution. This paper evaluates the proposed method against alternatives in picking or placing tasks among varying amounts of clutter for a variety of robotic manipulators with different end-effectors. The results suggest that the method reliably provides higher quality solution paths quicker, with a higher success rate relative to alternatives.
Fast and High-Quality Dual-Arm Rearrangement in Synchronous, Monotone Tabletop Setups
Rahul Shome and Solovey, K. and Yu, J. and Bekris, K. E. and Halperin, D.
2018 Workshop on the Algorithmic Foundations of Robotics (WAFR)
Rearranging objects on a planar surface arises in a variety of applications, such as packaging. Using two arms can improve efficiency but introduces new combinatorial challenges. This paper studies the structure of dual-arm rearrangement for synchronous, monotone tabletop setups and develops an optimal MILP model. It then describes an efficient and scalable algorithm, which first minimizes the cost of object transfers and then of transitions between objects. This is motivated by the fact that asymptotically object transfers dominate the cost of solutions. Moreover, a lazy strategy minimizes the number of motion planning calls and results in significant speedups. Theoretical arguments support the benefits of using two arms and indicate that synchronous operation introduces only a small cost increase. Experiments support these points and show that the scalable method can quickly compute solutions close to optimal for the considered setup.
Acoustic-telemetry payload control of an autonomous underwater vehicle for mapping tagged fish
Dodson, Tom and Grothues, Thomas M and Eiler, John H and Dobarro, Joseph A and Rahul Shome
2018 Limnology and Oceanography: Methods
Autonomous underwater vehicles (AUVs) have demonstrated superior performance for tracking marine animals tagged with individually coded acoustic transmitters. However, AUVs engaged in mapping the distribution of multiple tagged fish have not previously been able to alter search paths to achieve precise position estimates. This problem is solved by the development of payload control software (Synthetic Aperture Override, SAOVR) that allows the AUV to maneuver with trajectories favorable for solving the tag's location from a synthetic aperture. Upon tag detection during a default mission search path, SAOVR (running on an embedded guest computer) seeks permission to take over navigation from the vehicle's native system after checking constraints of geography, timing, tag identification, signal strength, and current navigation state. Permitted maneuvers are then chosen from a template library and executed before returning the AUV to the point of first deviation for continued searching of other tags. Field evaluation on moored reference tags showed a high level of predictability in the AUV's behavior at SAOVR initiation and through maneuvers. Trials suggest that this logic system is highly beneficial to AUV use for fish telemetry in challenging environments such as narrow, deep fjords, or among reefs. Any mission programmed with the AUV's native software can be run with the SAOVR package to allow scientists to easily implement and manipulate synthetic aperture geometries without altering any of the software. Further modeling can help improve template design specific to expected movements of different fish species and relative to the designation of signal strength-defined execution thresholds.
Scalable Asymptotically-Optimal Multi-Robot Motion Planning
Dobson, A. and Solovey, K. and Rahul Shome and Halperin, D. and Bekris, K. E.
2017 1st IEEE International Symposium on Multi-Robot and Multi-Agent Systems (MRS)
Discovering high-quality paths for multi-robot problems can be achieved, in principle, through asymptotically-optimal data structures in the composite space of all robots, such as a sampling-based roadmap or a tree. The hardness of motion planning, however, which depends exponentially on the number of robots, renders the explicit construction of such structures impractical. This work proposes a scalable, sampling-based planner for coupled multi-robot problems that provides desirable path-quality guarantees. The proposed dRRT* is an informed, asymptotically-optimal extension of a prior method dRRT, which introduced the idea of building roadmaps for each robot and implicitly searching the tensor product of these structures in the composite space. The paper describes the conditions for convergence to optimal paths in multi-robot problems. Moreover, simulated experiments indicate dRRT* converges to high-quality paths and scales to higher numbers of robots where various alternatives fail. It can also be used on high-dimensional challenges, such as planning for robot manipulators.
Improving the Scalability of Asymptotically Optimal Motion Planning for Humanoid Dual-arm Manipulators
Rahul Shome and Bekris, K. E.
2017 IEEE International Conference on Humanoid Robots
Due to high-dimensionality, many motion planners for dual-arm systems follow a decoupled approach but do not provide guarantees. Asymptotically optimal sampling-based planners provide guarantees, but in practice face computational scalability challenges. This work improves the computational scalability of the latter methods in this domain. It builds on top of recent advances in multi-robot motion planning, which provide guarantees without having to explicitly construct a roadmap in the composite space of all robots. The proposed framework builds roadmaps for components of a humanoid robot's kinematic chain. Then, the tensor product of these component roadmaps is searched implicitly online in a way that asymptotic optimality is provided. Appropriate heuristics from the component roadmaps are utilized for discovering the solution in the composite space effectively. Evaluation on various dual-arm problems show that the method returns paths of increasing quality, has significantly reduced space requirements and improved convergence rate relative to the standard asymptotically optimal approaches.
Evaluating End-Effector Modalities for Warehouse Picking: A Vacuum Gripper vs a 3-finger Underactuated Hand
Littlefield, Z. and Zhu, S. and Kourtev, C. and Psarakis, Z. and Rahul Shome and Kimmel, A. and Dobson, A. and Ferreira De Souza, A. and Bekris, K. E.
2016 12th IEEE International Conference on Automation Science and Engineering (IEEE CASE)
This paper evaluates two end-effector modalities in the context of warehouse picking tasks, where a robot has to grasp objects inside shelves. The two end-effectors correspond to (i) a recently developed, underactuated three-finger hand and (ii) a custom built, vacuum-based gripper. The two systems significantly differ on how they need to be placed relative to an object so that a successful grasp occurs. The first tool provides increased flexibility, while the vacuum alternative is simpler and has smaller form. The objective is to highlight how the end-effector choice can significantly influence the success rate of robotic picking as well as the speed of the overall solution. For the evaluation, the same grasping planning process is followed with both end-effectors given knowledge of an objects' pose. Multiple objects with different geometries and characteristics are placed in various poses for testing purposes. The resulting trajectories are executed on a real system to evaluate the effectiveness of the corresponding end-effector modalities in practice. The results indicate that, under different conditions, different types of end-effectors can be beneficial, which motivates the development of hybrid solutions.
A Dataset for Improved RGBD-based Object Detection and Pose Estimation for Warehouse Pick-and-Place
Rennie, C. and Rahul Shome and Bekris, K. E. and Ferreira De Souza, A.
2016 IEEE Robotics and Automation Letters (RA-L) [Also accepted to appear at the 2016 IEEE International Conference on Robotics and Automation (ICRA)]
An important logistics application of robotics involves manipulators that pick-and-place objects placed in warehouse shelves. A critical aspect of this task corresponds to detecting the pose of a known object in the shelf using visual data. Solving this problem can be assisted by the use of an RGB-D sensor, which also provides depth information beyond visual data. Nevertheless, it remains a challenging problem since multiple issues need to be addressed, such as low illumination inside shelves, clutter, texture-less and reflective objects as well as the limitations of depth sensors. This paper provides a new rich data set for advancing the state-of-the-art in RGBD-based 3D object pose estimation, which is focused on the challenges that arise when solving warehouse pick-and-place tasks. The publicly available data set includes thousands of images and corresponding ground truth data for the objects used during the first Amazon Picking Challenge at different poses and clutter conditions. Each image is accompanied with ground truth information to assist in the evaluation of algorithms for object detection. To show the utility of the data set, a recent algorithm for RGBD-based pose estimation is evaluated in this paper. Based on the measured performance of the algorithm on the data set, various modifications and improvements are applied to increase the accuracy of detection. These steps can be easily applied to a variety of different methodologies for object pose detection and improve performance in the domain of warehouse pick-and-place.
Cloud automation: Precomputing roadmaps for flexible manipulation
Bekris, Kostas and Rahul Shome and Krontiris, Athanasios and Dobson, Andrew
2015 IEEE Robotics & Automation Magazine
This work aims to highlight the benefits of Cloud Automation, the opportunities that arise for industrial adopters and some of the research challenges that must be addressed in this process. The focus is on the use of cloud computing for efficiently planning the motion of new robot manipulators designed for flexible manufacturing floors. In particular, different ways that a robot can interact with a computing cloud are considered, where an architecture that splits computation between the remote cloud and the robot appears advantageous. Given this synergistic robot-cloud architecture, this work describes how solutions from the recent motion planning literature can be employed on the cloud during a periodically updated preprocessing phase to efficiently answer manipulation queries on the robot given changes in the workspace. In this setup, interesting tradeoffs arise between path quality and computational efficiency, which are evaluated in simulation. These tradeoffs motivate further research on how motion planning should be executed given access to a computing cloud.
Rearranging Similar Objects with a Manipulator using Pebble Graphs
Krontiris, A. and Rahul Shome and Dobson, A. and Kimmel, A. and Bekris, K. E.
2014 IEEE-RAS International Conference on Humanoid Robots (HUMANOIDS)
This work proposes a method for effectively computing manipulation paths to rearrange similar objects in a cluttered space. Rearrangement is a challenging problem as it involves combinatorially large, continuous configuration spaces due to the presence of multiple bodies and kinematically complex manipulators. This work leverages ideas from algorithmic theory, multi-robot motion planning and manipulation planning to propose appropriate graphical representations for this challenge. These representations allow to quickly reason whether manipulation paths allow the transition between entire sets of object arrangements without having to explicitly store these arrangements. The proposed method also allows to take advantage of precomputation given a manipulation roadmap for transferring a single object in the same cluttered space. The resulting approach is probabilistically complete for a wide set of problem instances. It is evaluated in simulation for a realistic model of a Baxter robot and executed on the real system, showing that the approach solves complex instances and is promising in terms of scalability and success ratio.
An Extensible Software Architecture for Composing Motion and Task Planners
Littlefield, Z. and Krontiris, A. and Kimmel, A. and Dobson, A. and Rahul Shome and Bekris, K. E.
2014 International Conference on Simulation, Modeling, and Programming for Autonomous Robots (SIMPAR)
This paper describes a software infrastructure for developing and composing task and motion planners. The functionality of motion planners is well defined and they provide a basic primitive on top of which it is possible to develop planners for addressing higher level tasks. It is more challenging, however, to identify a common interface for task planners, given the variety of challenges that they can be used for. The proposed software platform follows a hierarchical, object-oriented structure and identifies key abstractions that help in integrating new task planners with popular sampling-based motion planners. Examples of use cases that can be implemented within this common software framework include robotics applications such as planning among dynamic obstacles, object manipulation and rearrangement, as well as decentralized motion coordination. The described platform has been used to plan for a Baxter robot rearranging similar objects in an environment in an efficient way.
An Experimental Study for Identifying Features of Legible Manipulator Paths
Zhao, M. and Rahul Shome and Yochelson, I. and Bekris, K. E. and Kowler, E.
2014 International Symposium on Experimental Robotics (ISER)
The increasing availability of low-cost, compliant and human-friendly manipulators, such as Rethink Robotics' Baxter, allows robots to share a common workspace and cooperate with human workers. It is important that the human is able to easily understand the robot's intentions by observing its actions. Legible motion plans are an important part of making the robot understandable by human co-workers intuitively. The legible robot motion has been investigated in previous studies, which focus on generating and discriminating the legibility of motion with two reachable targets. The current study is to expand upon the existing experimental studies in two primary directions. Firstly, this work considers a workspace with many potential targets for the robot to interact with, as well as to avoid. Secondly, experiments are performed under the guidance of psychophysicists where human subjects are placed in close proximity to but out of reach of the manipulator robot. These experiments confirm aspects of previous work, such as the contradictory nature of shortest and legible paths, and they also reveal important features of legible paths in cluttered scenes. For instance, the direction and path of the end effector is shown to significantly influence a human observer's capability to realize the robot's intended target.
A potential field based method for autonomous lunar rover navigation in 3D terrain
Nanadikar, Parth and Rahul Shome and Dutta Ashish
2011 26th International Conference on CAD/CAM and Factories of the Future
The development of planetary rovers is essential for the success of planetary missions. This paper discusses an algorithm based on the potential field method for navigation of a rover in an unknown 3D terrain containing obstacles. A 3D map of the terrain is generated using a structured light system, and the terrain is then divided into square grids having gradients. Assigning positive and negative gradients to the goal, obstacles, rover and grids we generate a potential field function. Using this function the rover finds the best path to reach a goal point. Unlike potential field functions in 2D this method works in 3D and also considers the rover kinematics.