Kids goof around with all kinds of things, not knowing what will occur. It’s how they learn how the world works, and it’s why they can learn novel electronic user interfaces so much more easily than adults (which we mistake as intuition). Well, in one such bout of goofing about on our apple orchard, I was monkeying with balloons over the opening of a bottle of apple cider.
You see, my dad had this thing where he’d poke a few needle holes in a balloon and then place it over the bottle opening. Fermentation would start, inflating the balloon, which opened the little holes and let the CO2 escape. When the process completed, the balloon deflated and the holes closed, blocking oxygen from reaching the inside – and preventing a continuation of the process that would otherwise lead to vinegar.*
The thing I remember most about this has nothing to do with cider or alcohol. Who knows what inspired this (if it can even be called inspiration), but I pushed my finger into the balloon, stretching it into the bottle until my finger was dipping into the cider. And I was sure that I was stretching the balloon enough to open those holes and let the liquid through, because my finger felt wet when in the liquid.
Except that, when I pulled my finger out, it was 100% perfectly dry.
This caused no end of cognitive dissonance: my senses were completely convinced that my finger was wet, when in fact it wasn’t at all. I came to the conclusion that the sensation of “wet” was actually a combination of temperature and proximity to and pressure on the skin. The balloon was such a thin barrier that my skin thought this cool substance was truly touching and wetting it. You could fake out my brain with the right sensory stimulation.
This is a real thing going on in the haptics world now. We’ve talked about haptics before; it’s all about feedback through touch. It’s that satisfying click when you push a button that tells you that you have successfully actuated the button. No click? Then you didn’t push hard enough.
At the Touch Gesture Motion conference late last fall, Tactus gave an interesting presentation that delved into the biology of haptics. And there were some surprising results.
Tactus’s main technology today has to do with the ability to create bubbles or other shapes in a thin film in real time. In other words, taking a flat surface and suddenly having buttons rise up out of it. In this case, the tactile feedback is coming from real (if temporary) buttons having real mechanical characteristics that have a particular feel.
That’s the way most haptics work today: there is a specific engine or mechanical device that creates a vibration or a “thunk” or some other touch sensation appropriate to the information being conveyed. But it turns out that the biology of touch consists of various discrete pieces, and if you can stimulate them without real mechanical effects, then you can simulate sensation.
First, let’s look through what’s under or in our skin. It turns out that there are different kinds of cells that respond to different types of stimulus. Part of what differentiates them is the rate at which the cells “adapt” to a situation.
Our body tends to normalize sensations. Conjuring up yet another memory from teenagehood, I recall drifting off to sleep near a speaker – I’d turn the music lower and lower until I could barely hear it. The thing was, after turning it down, it started to sound not so quiet, so I’d turn it down again – several times. What was happening was that, a while after the change, with no other loud sounds available for calibration, my brain said, “OK, this is now the normal level. It’s not loud or quiet; it’s normal.” So I had to turn it down yet further – and then that new level was normalized after a while.
In the morning, with all the clash of the day getting underway, the music was completely inaudible: the new normal was loud and active, and what had been normal in the quiet of the night was now far below soft.
This “adapting” process apparently works in our skin cells as well. Some cells adjust rapidly; some adjust slowly. Rapidly adjusting cells have higher bandwidth – after one stimulus, they’re ready for yet another. We have four fundamental cell types – two rapidly adjusting, two slowly adjusting. They have “human” names, descriptive names, and shortcut names.
- Meissner (or tactile) corpuscles; RA1 (rapidly-adapting 1): these cells detect low-frequency vibrations – on the order of 50 Hz. They contribute to our sense of texture and how well or poorly our skin slips along a surface.
- Pacinian (or lamellar) corpuscles; RA2: these are sensitive to high-frequency vibrations – on the order of 250 Hz – across the fingers and hand.
- Ruffini (or bulbous) corpuscles; SA1 (slowly-adapting 1): these respond to tension deep within the skin, the stretching of skin, and sustained pressure; they also give us information about our finger and hand position.
- Merkel disks; SA2): these have high spatial resolution, detecting pressure, helping to sense our position, and – critically – helping to discriminate shapes and edges. When this presentation was given, by the way, someone in the audience commented that these cells were only newly identified – their acceptance was something like two weeks old, so it should considered provisional. That said, they’re well documented in Wikipedia and other places (with no such caveats).
- The previous cells are specific structures that grow on nerve endings. There are also so-called “free nerve endings” –the ends of nerves with no extra hardware on them. They have intermediate adaptation rates, and they are used in detecting pain, touch, pressure, and stretching.
It’s more complex than this, of course. Neighboring Meissner corpuscles may respond differently. And muscle memory is involved, as is the concept of proprioception: sensing where our body is, and there is a whole different set of cells located in ligaments and muscles that help out with that.
This information can let us start to target specific sensations not by recreating the sensation in the real world, but by fooling these cells into reporting a sensation that’s not real. For instance, you can create the illusion of friction without actually creating a rough surface.
A few years ago, there was a project called Teslatouch. They played with different ways of creating common tactile sensations simply by using electric fields and currents. There are three ways to do this:
- Electrocutaneous: the current enters the skin and directly stimulates the cells.
- Electrostatic: this requires an intermediate object. A current or field makes the object vibrate, for instance, and that vibration is then transferred to the skin.
- Electrovibration: this is mechanical vibration caused by an electric field and current, but the current remains within the substrate being touched – no current reaches or enters the skin (much safer!).
The Teslatouch project involved electrovibration, and the team assembled a panel of lucky guinea-pig humans who would touch a surface under various potentials and frequencies and then rate how they perceived that surface. They took all four combinations of two frequencies (80 Hz, 400 Hz) and amplitude (80 and 115 VPP) and then had the contestants rate four axes on a scale of 1-5:
- Whether the surface felt sticky or smooth and waxy
- Whether the surface felt smooth or bumpy
- Whether the finger sensation was pleasant or unpleasant
- Whether the sensation was of friction or of vibration
I’ve replotted their results on “star” graphs, showing how each of the combinations was perceived. Each shows a range (I’ve omitted the error bars). To reduce clutter, I’ve shown the axes first, for reference when viewing the actuals.
Figure 1. Unpopulated axes.
The following is how people perceived the 400 Hz/115 and 80 VPP combinations:
Figure 2. 400 Hz, 115 VPP; 400 Hz, 80 VPP
Reducing the frequency to 80 Hz gives the following:
Figure 3. 80 Hz, 115 VPP; 80 Hz, 80 VPP
The width of the colored region illustrates the variation in how the sensation was perceived; wider (or more color) means that there was more of a difference in perception amongst participants. Narrow means that more people agreed on the sensation.
Most of the combinations were towards the middle of the pleasantness range, but the 80/115 combination (Figure 3), for some reason, was the most unpleasant. High frequency felt smoother than low frequency. And so forth.
Through experiments like these, you can start to assemble a combination of signals that mimic real sensation without actually causing it. As with many things, though, set and setting matter. The brain evaluates these sensations in some context. So, for instance, my cider experiment yielded a particular interpretation because there was liquid present. Had I detected the same sensation when, say, I was holding my mouse at the desktop, it’s not clear to me whether I would have interpreted my finger as being wet. (And I have no good way to test that…)
Are these just tantalizing tidbits luring us into a dead end? I certainly don’t know. Then again, I’m one of those guys who believe that more knowledge rarely, if ever, hurts, so a dead end where you learned a lot isn’t necessarily a loss. (That’s one of the differences between research and development.)
I tried to follow up with the Teslatouch project leader to see what the current state is; as of publication, I haven’t heard back.
It’s also not clear exactly where Tactus plans to take this; it was presented more as a general survey of efforts to take advantage of our growing understanding of human biology. And yet another way to augment our reality – partly by faking out our brains.
But it’s certainly spurring some folks to go out and generate the next generation of tactile feedback – further blurring the distinctions between what’s real and what isn’t.
*Lest any alcohol regulators notice this, what we ended up with was a lightly effervescent beverage with a low alcohol content – nothing like the ciders we can purchase today. Frankly, letting it ferment to completion resulted in a pretty nasty-tasting substance. And we minors took only the slightest tastes. Nothing more to see here. Besides, aren’t there statutes of limitation?
More info:
TeslaTouch: Electrovibration for Touch Surfaces
[Edited to clarify tha the comment about Merkel disks being two weeks old was not from the presenter, but from the audience.]
What other ways do you know of to fake out our sense of touch for haptic use?