Stanford Seminar - Integrating interactive devices with the user's body

02 Mar 2024 (10 months ago)

Integration of Devices with the Human Body

Pedro, an associate professor in computer science at the University of Chicago, discusses the concept of "integration" between computers and the human body beyond wearable devices.
He suggests that the next step in the evolution of computing is for devices to integrate more intimately with the human body.
He explores the idea of designing for the overlap between the user's body and the device, where the device intentionally borrows parts of the user's body to perform its function.

Pedro discusses the issue of virtual reality (VR) lacking physical force feedback, which limits its use for training physical skills.
He presents exoskeleton devices as a solution for providing physical force feedback in VR but notes their incompatibility with current untethered and mobile VR headsets.
Pedro proposes that integrating devices with the body could enable the creation of more realistic and immersive VR experiences.
Electrical muscle stimulation (EMS) can be used as a force feedback interface in VR.
EMS works by applying a tiny electrical impulse over a pair of electrodes, causing the muscles to contract and create a sense of force.
EMS has advantages over traditional force feedback devices as it requires much smaller hardware and can be easily miniaturized.

Researchers are exploring other sensory systems, such as temperature, and using illusions to create realistic experiences in VR without bulky hardware.
The device projects a minuscule amount (0.1 microliters) of capsaicin in a droplet form factor, creating the illusion of heat when inhaled.
Eucalyptol, found in breath mints, is used to create a cooling sensation by activating a different receptor in the trigeminal nerve.
The sensations are localized to the face, but some participants reported feeling them on their whole body.
The device can also simulate the feeling of chemicals passing through the skin, creating a heightened sense of realism for training purposes.

Muscle stimulation can be used to assist in skill acquisition, such as learning to play the piano, by providing precise movements and feedback on force and speed.
Controlling muscle stimulation precisely is challenging due to the difficulty in stopping and maintaining a specific position.
A combination of muscle stimulation and mechanical devices can improve precision, as demonstrated in a prototype that allows a person to play guitar chords and sign language.
A researcher is working on a glove that uses muscle stimulation to help people learn sign language.
The glove locks certain fingers in place while allowing others to move freely, which helps to create more precise finger poses.
The researcher is also interested in how this technology can be used to enhance human abilities, such as reaction time.
They conducted an experiment to see how people's sense of agency is affected when their reaction time is sped up using muscle stimulation.
They found that people can have their reaction time sped up by a small amount without noticing, and that they can still feel a sense of agency over their actions even when the stimulation is delayed by 80 milliseconds.
Training with delayed muscle stimulation can actually lead to a long-term improvement in reaction time, even after the electrodes are removed.
Muscle stimulation can improve reaction time by 8 milliseconds, which is significant in sports and other activities.

Optimizing the sense of agency during movement is important, but it becomes more complicated when the intention is not aligned with the machine's actions.
Control is also crucial, not just during movement but also before and after. Different methods have been developed to provide users with control over muscle stimulation systems.
Ethical implications of extreme forms of control should be considered, as demonstrated through an artwork that traps people's arms and uses their power to generate electricity.

Post-wearables that integrate with the body can solve problems such as the force haptics problem in VR and provide tactile sensations when touching virtual objects.
Haptic gloves that cover the finger pads limit dexterity and the ability to interact with the real world.
Electrical tactile stimulation in strategic locations on the back of the hand can deliver sensations to the fingertips while keeping the finger pads free for real-world interactions.

Integration of technology into the body offers new possibilities for enhancing human abilities and experiences.
Muscle stimulation can be used to assist or correct physical movements, potentially improving learning and skill development.
Integration can take various forms, including living organisms incorporated into devices, which can foster a sense of responsibility and environmental awareness.
Integration has the potential to change people's relationships with their bodies by allowing them to explore new movements and physical possibilities.