Difference between revisions of "Project Week 25/Human-Computer Interaction under sterile conditions"
From NAMIC Wiki
(7 intermediate revisions by 2 users not shown) | |||
Line 31: | Line 31: | ||
## With OSC communication protocol, David will make quick, flexible sound methods | ## With OSC communication protocol, David will make quick, flexible sound methods | ||
## foot/eye interaction (?) | ## foot/eye interaction (?) | ||
− | |||
− | |||
| | | | ||
<!-- Progress and Next steps (fill out at the end of project week), please start each sentence in a new line. --> | <!-- Progress and Next steps (fill out at the end of project week), please start each sentence in a new line. --> | ||
− | # Connected Slicer to OSC: added SlicerSoundControl extension (https://github.com/SlicerIGT/SlicerSoundControl) | + | # Connected Slicer to OSC: added SlicerSoundControl extension (https://github.com/SlicerIGT/SlicerSoundControl), see below for necessary OSC message names |
# Tested avoidance warning with experimental integration into breast lumpectomy navigation module (LumpNav): https://youtu.be/gSz8IHmogMo | # Tested avoidance warning with experimental integration into breast lumpectomy navigation module (LumpNav): https://youtu.be/gSz8IHmogMo | ||
# tried using gestures to zoom windows in and out | # tried using gestures to zoom windows in and out | ||
|} | |} | ||
+ | |||
+ | ==Previous Work== | ||
+ | Demo Videos | ||
+ | * ablation needle or general tool placement, with U. Magdeburg and U. Hannover, [https://www.dropbox.com/s/35xzp51tk0rui6x/Singing.m4v?dl=0 “singing" method] and [https://www.dropbox.com/s/05et9abov10yvmk/AblationDemoVideo1.m4v?dl=0 synthetic sounds] | ||
+ | * [https://www.dropbox.com/s/ullaaab4gp9pucq/BWH_Dual-frequency-feedback.mp4?dl=0 ureteroscopy], with Harvard Surgical Planning Lab | ||
+ | * [https://www.dropbox.com/s/cdfc77ugjgg8o96/LIM%20Short.mp4?dl=0 sacral neuromodulation], with University Carlos III Madrid | ||
+ | * [https://www.dropbox.com/s/bn0u0g22yu14uvt/AD%20Resection%20short.m4v?dl=0 resection guidance], with Robert Bosch Hospital Stuttgart ([https://www.youtube.com/watch?v=gCg5nJSI2pY longer version with explanation]) | ||
+ | |||
+ | In addition, please see the following articles: | ||
+ | * [https://link.springer.com/article/10.1007/s11548-017-1537-1 Auditory feedback to support image-guided medical needle placement] | ||
+ | * [https://link.springer.com/article/10.1007/s11548-017-1547-z A Survey of auditory display in image-guided interventions] | ||
+ | * [https://www.researchgate.net/publication/233791033_Auditory_support_for_resection_guidance_in_navigated_liver_surgery Auditory support for resection guidance in navigated liver surgery.] | ||
+ | |||
+ | ==Illustrations== | ||
+ | |||
+ | {| class="wikitable" | ||
+ | ! style="text-align: left; width:50%" | [[image:OpenSoundControl.png|400px]] OpenSoundControl module for configuring and testing OpenSoundControl server communication | ||
+ | ! style="text-align: left; width:50%" | [[image:SoundNavigation.png|400px]] SoundNavigation module for automatically generating OpenSoundControl messages for tool navigation, from tool and reference transform nodes | ||
+ | |- style="vertical-align:top;" | ||
+ | |} | ||
+ | |||
+ | <embedvideo service="youtube">https://youtu.be/gSz8IHmogMo</embedvideo> | ||
==SlicerSoundControl== | ==SlicerSoundControl== | ||
Line 53: | Line 73: | ||
Translation: each fro 0 to 100 mm | Translation: each fro 0 to 100 mm | ||
− | /TranslationX | + | * /TranslationX |
− | /TranslationY | + | * /TranslationY |
− | /TranslationZ | + | * /TranslationZ |
− | /Distance (absolute distance) | + | * /Distance (absolute distance) |
Translation: each from 0 to 180 degrees | Translation: each from 0 to 180 degrees | ||
− | /OrientationX | + | * /OrientationX |
− | /OrientationY | + | * /OrientationY |
− | /OrientationZ | + | * /OrientationZ |
− | /Orientation (overall orientation) | + | * /Orientation (overall orientation) |
+ | |||
+ | |||
+ | GUI for test: | ||
+ | [[File:OSCTest GUI.png|500px|frameless|OSCTest GUI]] | ||
=== Environment for Sterile Gestures (planned, not yet implemented) === | === Environment for Sterile Gestures (planned, not yet implemented) === |
Latest revision as of 13:23, 30 June 2017
Home < Project Week 25 < Human-Computer Interaction under sterile conditionsContents
Back to Projects List
Key Investigators
- Christian Hansen (University of Magdeburg, Germany)
- Julian Hettig (University of Magdeburg, Germany)
- Andras Lasso (PerkLab, Queen's)
- Benjamin Hatscher (University of Magdeburg, Germany)
- David Black (University of Bremen; Fraunhofer Institute for Medical Image Computing MEVIS, Bremen, Germany)
- Marco Nolden (German Cancer Research Center (DKFZ), Germany)
- Juan Ruiz Alzola (University of Las Palmas de Gran Canaria, Spain)
- Salvatore Scaramuzzino ("Magna Graecia" University - ASL Vercelli, Italy)
Project Description
Objective | Approach and Plan | Progress and Next Steps |
---|---|---|
Human-Computer Interaction under Sterile Conditions.
|
|
|
Previous Work
Demo Videos
- ablation needle or general tool placement, with U. Magdeburg and U. Hannover, “singing" method and synthetic sounds
- ureteroscopy, with Harvard Surgical Planning Lab
- sacral neuromodulation, with University Carlos III Madrid
- resection guidance, with Robert Bosch Hospital Stuttgart (longer version with explanation)
In addition, please see the following articles:
- Auditory feedback to support image-guided medical needle placement
- A Survey of auditory display in image-guided interventions
- Auditory support for resection guidance in navigated liver surgery.
Illustrations
SlicerSoundControl
see https://github.com/SlicerIGT/SlicerSoundControl
Environment for Testing Slicer Sound Control
OSC Message names:
Base name: /SoundNav/Instrument/
Translation: each fro 0 to 100 mm
- /TranslationX
- /TranslationY
- /TranslationZ
- /Distance (absolute distance)
Translation: each from 0 to 180 degrees
- /OrientationX
- /OrientationY
- /OrientationZ
- /Orientation (overall orientation)
Environment for Sterile Gestures (planned, not yet implemented)
- In/ out
- --> "/InOut 0" out
- --> "/InOut 1" in
- Edge distance (0 to 3 cm?)
- --> "/EdgeDist x" where x is float between 0 and 3
Ambient
- Overall acceleration
- --> "/OverallAccel x" where x is float between 0 and 1
- Background noise
- Absolute position (x/y, not depth) to show current selected window
- --> "/AbsPolX x" where x is float between 0 and 1
- --> "/AbsPolY x" where x is float between 0 and 1
Gestures
- select image
- --> "/SelectGesture x" where x is gesture number
- zoom to left window
- zoom to right window
- put back into small viewer
Background
Example of previous related work using standard surgical gloves and OR compatible plastic draping confirmed to be compatible with sterile requirements at BWH AMIGO:
]