0
Research Papers

Force-Controlled Exploration for Updating Virtual Fixture Geometry in Model-Mediated Telemanipulation

[+] Author and Article Information
Long Wang

Advanced Robotics and
Mechanism Applications,
Department of Mechanical Engineering,
Vanderbilt University,
Nashville, TN 37235
e-mail: long.wang@Vanderbilt.edu

Zihan Chen

Laboratory for Computational
Sensing and Robotics,
Department of Computer Science,
Johns Hopkins University,
Baltimore, MD 21218
e-mail: zihan.chen@jhu.edu

Preetham Chalasani

Laboratory for Computational
Sensing and Robotics,
Department of Computer Science,
Johns Hopkins University,
Baltimore, MD 21218
e-mail: pchalas1@jhu.edu

Rashid M. Yasin

Advanced Robotics and
Mechanism Applications,
Department of Mechanical Engineering,
Vanderbilt University,
Nashville, TN 37235
e-mail: rashid.m.yasin@Vanderbilt.edu

Peter Kazanzides

Laboratory for Computational
Sensing and Robotics,
Department of Computer Science,
Johns Hopkins University,
Baltimore, MD 21218
e-mail: pkaz@jhu.edu

Russell H. Taylor

Laboratory for Computational
Sensing and Robotics,
Department of Computer Science,
Johns Hopkins University,
Baltimore, MD 21218
e-mail: rht@jhu.edu

Nabil Simaan

Advanced Robotics and
Mechanism Applications,
Department of Mechanical Engineering,
Vanderbilt University,
Nashville, TN 37235
e-mail: nabil.simaan@Vanderbilt.edu

Manuscript received October 13, 2016; final manuscript received December 16, 2016; published online March 9, 2017. Assoc. Editor: Hai-Jun Su.

J. Mechanisms Robotics 9(2), 021010 (Mar 09, 2017) (11 pages) Paper No: JMR-16-1304; doi: 10.1115/1.4035684 History: Received October 13, 2016; Revised December 16, 2016

This paper proposes an approach for using force-controlled exploration data to update and register an a priori virtual fixture geometry to a corresponding deformed and displaced physical environment. An approach for safe exploration implementing hybrid motion/force control is presented on the slave robot side. During exploration, the shape and the local surface normals of the environment are estimated and saved in an exploration data set. The geometric data collected during this exploration scan are used to deform and register the a priori environment model to the exploration data set. The environment registration is achieved using a deformable registration based on the coherent point drift method. The task-description of the high-level assistive telemanipulation law, called a virtual fixture (VF), is then deformed and registered in the new environment. The new model is updated and used within a model-mediated telemanipulation framework. The approach is experimentally validated using a da-Vinci research kit (dVRK) master interface, a dVRK patient side manipulator, and a Cartesian stage robot. Experiments demonstrate that the updated VF and the updated model allow the users to improve their path following performance and to shorten their completion time when the updated path following VF is applied. The approach presented has direct bearing on a multitude of surgical applications including force-controlled ablation.

Copyright © 2017 by ASME
Your Session has timed out. Please sign back in to continue.

References

Sotiras, A. , Davatzikos, C. , and Paragios, N. , 2013, “ Deformable Medical Image Registration: A Survey,” IEEE Trans. Med. Imaging, 32(7), pp. 1153–1190. [CrossRef] [PubMed]
Mirota, D. J. , Ishii, M. , and Hager, G. D. , 2011, “ Vision-Based Navigation in Image-Guided Interventions,” Annu. Rev. Biomed. Eng., 13(1), pp. 297–319. [CrossRef] [PubMed]
Cash, D. M. , Sinha, T. K. , Chapman, W. C. , Terawaki, H. , Dawant, B. M. , Galloway, R. L. , and Miga, M. I. , 2003, “ Incorporation of a Laser Range Scanner Into Image-Guided Liver Surgery: Surface Acquisition, Registration, and Tracking,” Med. Phys., 30(7), pp. 1671–1682. [CrossRef] [PubMed]
Hayashibe, M. , 2006, “ Laser-Scan Endoscope System for Intraoperative Geometry Acquisition and Surgical Robot Safety Management,” Med. Image Anal., 10(4), pp. 509–519. [CrossRef] [PubMed]
Lathrop, R. A. , Rucker, D. C. , and Webster, R. J., III , 2010, “ Guidance of a Steerable Cannula Robot in Soft Tissue Using Preoperative Imaging and Conoscopic Surface Contour Sensing,” International Conference on Robotics and Automation (ICRA), Anchorage, AK, May 3–7, pp. 5601–5606.
Xia, T. , Leonard, S. , Kandaswamy, I. , Blank, A. , Whitcomb, L. L. , and Kazanzides, P. , 2013, “ Model-Based Telerobotic Control With Virtual Fixtures for Satellite Servicing Tasks,” IEEE International Conference on Robotics and Automation (ICRA), Karlsruhe, Germany, May 6–10, pp. 1479–1484.
Mitra, P. , and Niemeyer, G. , 2008, “ Model-Mediated Telemanipulation,” Int. J. Rob. Res., 27(2), pp. 253–262. [CrossRef]
Kazanzides, P. , Chen, Z. , Deguet, A. , Fischer, G. S. , Taylor, R. H. , and DiMaio, S. , 2014, “ An Open-Source Research Kit for the da Vinci[Textregistered] Surgical Robot,” IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, China, May 31–June 7, pp. 6434–6439.
Galea, A. , and Howe, R. , 2002, “ Tissue Stiffness From Tactile Imaging,” Second Joint 24th Annual Conference and the Annual Fall Meeting of the Biomedical Engineering Society Conference in Engineering in Medicine and Biology (EMBS/BMES), Houston, TX, Oct. 23–26, Vol. 2, pp. 935–936.
Kesner, S. B. , and Howe, R. D. , 2011, “ Discriminating Tissue Stiffness With a Haptic Catheter: Feeling the Inside of the Beating Heart,” World Haptics Conference (WHC), Istanbul, Turkey, June 21–24, pp. 13–18.
Egorov, V. , Van Raalte, H. , and Sarvazyan, A. P. , 2010, “ Vaginal Tactile Imaging,” IEEE Trans. Biomed. Eng., 57(7), pp. 1736–1744. [CrossRef] [PubMed]
Liu, H. , Noonan, D. P. , Challacombe, B. J. , Dasgupta, P. , Seneviratne, L. D. , and Althoefer, K. , 2010, “ Rolling Mechanical Imaging for Tissue Abnormality Localization During Minimally Invasive Surgery,” IEEE Trans. Biomed. Eng., 57(2), pp. 404–414. [CrossRef] [PubMed]
Xu, K. , and Simaan, N. , 2010, “ Intrinsic Wrench Estimation and Its Performance Index for Multisegment Continuum Robots,” IEEE Trans. Rob., 26(3), pp. 555–561. [CrossRef]
Nichols, K. , and Okamura, A. M. , 2013, “ Autonomous Robotic Palpation: Machine Learning Techniques to Identify Hard Inclusions in Soft Tissues,” IEEE International Conference on Robotics and Automation (ICRA), Karlsruhe, Germany, May 6–10, pp. 4384–4389.
Bajo, A. , and Simaan, N. , 2016, “ Hybrid Motion/Force Control of Multi-Backbone Continuum Robots,” Int. J. Rob. Res., 35(4), pp. 422–434. [CrossRef]
Goldman, R. E. , Bajo, A. , and Simaan, N. , 2013, “ Algorithms for Autonomous Exploration and Estimation in Compliant Environments,” Robotica, 31(1), pp. 71–88. [CrossRef]
Sanan, S. , Tully, S. , Bajo, A. , Simaan, N. , and Choset, H. , 2014, “ Simultaneous Compliance and Registration Estimation for Robotic Surgery,” Robotics: Science and Systems, Berkeley, CA.
Srivatsan, R. A. , Ayvali, E. , Wang, L. , Roy, R. , Simaan, N. , and Choset, H. , 2016, “ Complementary Model Update: A Method for Simultaneous Registration and Stiffness Mapping in Flexible Environments,” IEEE International Conference on Robotics and Automation (ICRA), Stockholm, Sweden, May 16–21, pp. 924–930.
Chalasani, P. , Wang, L. , Roy, R. , Simaan, N. , Taylor, R. H. , and Kobilarov, M. , 2016, “ Concurrent Nonparametric Estimation of Organ Geometry and Tissue Stiffness Using Continuous Adaptive Palpation,” IEEE International Conference on Robotics and Automation (ICRA), Stockholm, Sweden, May 16–21, pp. 4164–4171.
Ayvali, E. , Srivatsan, R. A. , Wang, L. , Roy, R. , Simaan, N. , and Choset, H. , 2016, “ Using Bayesian Optimization to Guide Probing of a Flexible Environment for Simultaneous Registration and Stiffness Mapping,” IEEE International Conference on Robotics and Automation (ICRA), Stockholm, Sweden, May 16–21, pp. 931–936.
Myronenko, A. , and Song, X. , 2010, “ Point Set Registration: Coherent Point Drift,” IEEE Trans. Pattern Anal. Mach. Intell., 32(12), pp. 2262–2275. [CrossRef] [PubMed]
Billings, S. D. , Boctor, E. M. , and Taylor, R. H. , 2015, “ Iterative Most-Likely Point Registration (Imlp): A Robust Algorithm for Computing Optimal Shape Alignment,” PloS One, 10(3), p. e0117688.
Wang, L. , Chen, Z. , Chalasani, P. , Pile, J. , Kazanzides, P. , Taylor, R. H. , and Simaan, N. , 2016, “ Updating Virtual Fixture From Exploration Data in Force-Controlled Model-Based Telemanipulation,” ASME Paper No. DETC2016-59305.
Jung, M. Y. , Deguet, A. , and Kazanzides, P. , 2010, “ A Component-Based Architecture for Flexible Integration of Robotic Systems,” IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Taipei, Taiwan, Oct. 18–22, pp. 6107–6112.
Chen, Z. , Deguet, A. , Taylor, R. , DiMaio, S. , Fischer, G. , and Kazanzides, P. , 2013, “ An Open-Source Hardware and Software Platform for Telesurgical Robot Research,” Department of Computer Science, Johns Hopkins University, Baltimore, MD, pp. 22–26.
Khatib, O. , 1987, “ A Unified Approach for Motion and Force Control of Robot Manipulators: The Operational Space Formulation,” IEEE J. Rob. Autom., RA-3(1), pp. 43–53. [CrossRef]
Featherstone, R. , Thiebaut, S. , and Khatib, O. , 1999, “ A General Contact Model for Dynamically-Decoupled Force/Motion Control,” IEEE International Conference on Robotics and Automation (ICRA), Detroit, MI, May 10–15, pp. 3281–3286.
Whitney, D. E. , 1969, “ Resolved Motion Rate Control of Manipulators and Human Prostheses,” IEEE Trans. Man-Mach. Syst., 10(2), pp. 47–53. [CrossRef]
Kapoor, A. , Li, M. , and Taylor, R. , 2006, “ Constrained Control for Surgical Assistant Robots,” IEEE International Conference on Robotics and Automation (ICRA), Orlando, FL, May 15–19, pp. 231–236.
Yoshikawa, T. , and Sudou, A. , 1993, “ Dynamic Hybrid Position/Force Control of Robot Manipulators—On-Line Estimation of Unknown Constraint,” IEEE Trans. Rob. Autom., 9(2), pp. 220–226. [CrossRef]
Namvar, M. , and Aghili, F. , 2005, “ Adaptive Force-Motion Control of Coordinated Robots Interacting With Geometrically Unknown Environments,” IEEE Trans. Rob., 21(4), pp. 678–694. [CrossRef]
Yin, Y. , Hu, H. , and Xia, Y. , 2004, “ Active Tracking of Unknown Surface Using Force Sensing and Control Technique for Robot,” Sens. Actuators A, 112(2), pp. 313–319. [CrossRef]
Karayiannidis, Y. , and Doulgeri, Z. , 2009, “ Adaptive Control of Robot Contact Tasks With On-Line Learning of Planar Surfaces,” Automatica, 45(10), pp. 2374–2382. [CrossRef]
Kwartowitz, D. M. , Herrell, S. D. , and Galloway, R. L. , 2006, “ Toward Image-Guided Robotic Surgery: Determining Intrinsic Accuracy of the da Vinci Robot,” Int. J. Comput. Assisted Radiol. Surg., 1(3), pp. 157–165. [CrossRef]
Linte, C. A. , Moore, J. , and Peters, T. M. , 2010, “ How Accurate is Accurate Enough? A Brief Overview on Accuracy Considerations in Image-Guided Cardiac Interventions,” Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Buenos Aires, Argentina, Aug. 3–Sept. 4, pp. 2313–2316.
Krücker, J. , Xu, S. , Glossop, N. , Viswanathan, A. , Borgert, J. , Schulz, H. , and Wood, B. J. , 2007, “ Electromagnetic Tracking for Thermal Ablation and Biopsy Guidance: Clinical Evaluation of Spatial Accuracy,” J. Vasc. Interventional Radiol., 18(9), pp. 1141–1150. [CrossRef]

Figures

Grahic Jump Location
Fig. 1

System architecture

Grahic Jump Location
Fig. 2

A custom Cartesian slave robot system: (a) experiment setup, (b) ball probe finger ATI force torque sensor, and (c) a phantom model used in experiment

Grahic Jump Location
Fig. 3

PSM experiment setup: (a) experiment setup, (b) ball probe finger adapter integrated with EM tracker, and (c) a phantom model mounted on a force plate

Grahic Jump Location
Fig. 4

PSM force-controlled slave MLC-LLC controller

Grahic Jump Location
Fig. 5

Vanderbilt slave MLC-LLC controller

Grahic Jump Location
Fig. 6

Master MLC impedance type controller: q—joint position; q˙—joint velocity; x—Cartesian position; x˙—Cartesian velocity; T—total joint torque applied to robot; TVF—joint torque from virtual fixture controller Tgc—joint torque from gravity compensation

Grahic Jump Location
Fig. 7

Surface following frame with master robot tip and force feedback

Grahic Jump Location
Fig. 8

Curve following frame with master robot tip and force feedback

Grahic Jump Location
Fig. 9

Force-controlled exploration strategy

Grahic Jump Location
Fig. 10

Contact location and surface norm estimation

Grahic Jump Location
Fig. 11

Orientation optimization for force-controlled exploration using a robot with wrist

Grahic Jump Location
Fig. 12

Force-controlled exploration using custom Cartesian robot: (a) is the planned scan pattern and (b) is the actual scanned point cloud

Grahic Jump Location
Fig. 13

Force-controlled exploration using dVRK PSM: (a) is the silicone phantom organ and (b) is the actual collected point cloud from exploration

Grahic Jump Location
Fig. 14

Creating an apriori model of the silicone phantom: (a) apriori STL model (PA), (b) digitizing the target curve (Cdig) using Faro Arm, and (c) laser scan (Pls) and the digitized curve (Cdig) in red

Grahic Jump Location
Fig. 15

The process of updating the virtual fixture (VF) geometry: (a) the apriori model (pre-operative model) with a VF curve, (b) the deformed environment obtained from exploration data, and (c) using correspondence list to find the VF points in the exploration data set that match the curve from apriori data set

Grahic Jump Location
Fig. 16

Deformable registration using exploration data from dVRK PSM robot and ground truth: (a)–(g) show iterations of deformable registration using PSM robot data where iteration numbers are {1, 3, 5, 10, 20, 50, 100}, (h) is the deformable registration result using laser scan data

Grahic Jump Location
Fig. 17

Errors between the updated VF curve and the digitized ground truth fitted curve: (a) PSM robot case, (b) PSM laser comparison, (c) Cartesian robot case, and (d) Cartesian laser comparison

Grahic Jump Location
Fig. 18

Ground truth digitization data points and the smooth curve fit

Grahic Jump Location
Fig. 19

Comparisons of the users' performance with and without virtual fixture on target curve tracing RMS errors and completion time

Tables

Errata

Discussions

Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In