1. RNP
  2. Sensorimotor Loop and Motor Control
  3. When the senses disagree: sensory weighting and calibration
  4. Multisensory integration: definition, principles and what merging the senses changes for athletes

Sensorimotor Loop and Motor ControlSeries 2/14

SeriesWhen the senses disagree: sensory weighting and calibration2 of 14

Multisensory integration: definition, principles and what merging the senses changes for athletes

Multisensory integration: definition, three principles, optimal cue combination, sports examples and how it differs from sensory integration.

LabO RNP·19 min read·reviewed on

Tonight, while watching a film, listen to where the actor's voice comes from: a speaker sitting next to the screen, and yet you hear it coming out of their mouth. Tomorrow, from the stand on the far side, watch a clearance: the foot strikes, and the sound of the ball reaches you a moment later. Both times, your brain decides whether that image and that sound describe the same event.


No time? Remember this.
Multisensory integration is the process by which the nervous system combines information from several senses (vision, hearing, touch, proprioception, the vestibular system) about the same event to produce a single percept. That percept is more reliable than any one sense on its own, because the brain gives the most weight to whichever sense is most precise in that situation. Three principles govern it, first described in neurons of the superior colliculus: temporal coincidence, spatial coincidence and inverse effectiveness. In adults, the combination often follows a statistically optimal calculation (Ernst and Banks, 2002). Neuro-Postural Reprogramming (RNP) draws a field rule from this: read each input before judging how they merge.

Hello to you, movement professional,

The cinema effect has a very old name: ventriloquism. Alais and Burr measured it. When the image pinpoints the source clearly, vision pulls the sound toward it. When they blurred the image heavily, sound pulled vision instead. And the location estimated with both senses was generally more precise than with either one alone (Alais and Burr, 2004). The brain computes an average, weighted by how much it trusts each source.

On the field, this calculation runs nonstop. A tennis player returning serve sees the opponent's shoulder open, hears the impact and feels their feet slide on clay, and all of it fits into a single decision. The pillar article on information pickup in sport covers the whole chain. Here, we zoom in on one link of the sensorimotor loop: the moment several senses become a single piece of information.

A serious objection comes up quickly: these principles come from cat neurons and laboratory tasks, far removed from a winger sprinting at full speed. That objection is fair. The mechanisms belong to neurophysiology, from Stein and Meredith's recordings to Ernst and Banks's psychophysics. Translating them into a field reading is the work of RNP, the framework LabO RNP built for coaches, strength and conditioning coaches and physical therapists. That translation still has to be checked by measurement, and this article keeps the two apart at every step.


Multisensory integration: definition and the merging of the senses

Multisensory integration refers to the way the nervous system combines signals from different senses into a single representation of an event, an object or the body itself. It is also called sensory fusion or multimodal integration.

The phenomenon was described neuron by neuron in the superior colliculus, a paired structure in the roof of the midbrain. It overlays visual, auditory and somatosensory maps of space and drives orienting movements of the eyes and head. Meredith and Stein compared how a single cell responded to a visual, auditory or tactile stimulus presented alone, then to combinations of them. The combination markedly enhanced or depressed activity, and these interactions were most often multiplicative rather than additive (Meredith and Stein, 1986a). Most of these cells project to brainstem and spinal centers that orient the eyes, ears and head: merging the senses serves action first and foremost. A goalkeeper who turns their head toward the sound of a strike outside their visual field is a good field illustration.

Other levels of the nervous system do this work too. The RNP Level 01 training manual cites the cerebellum, which combines vestibular, proprioceptive and visual signals, and the posterior parietal cortex, which merges them into the body schema. In at least one circuit studied in animals, this ability only appears long after birth and may require specific experience (Stein and Stanford, 2008).


In plain terms: your senses agree on a single version of what is happening, and that version beats any one of them on its own.

The three principles of multisensory integration

The principles of multisensory integration are three rules, established by Barry Stein and Alex Meredith in neurons of the cat superior colliculus, that predict when two signals reinforce each other: they arrive at roughly the same time, come from roughly the same place, and the gain grows when each signal on its own is weak. The reference book remains The Merging of the Senses (Stein and Meredith, 1993), summarized in our entry Stein and Meredith: multisensory integration.

Temporal coincidence. Enhancement peaks when the discharges triggered by each sense overlap, declines as they are pulled apart, and eventually the same pair of stimuli ends up depressing the response (Meredith et al., 1987). The brain works with a time window and accounts for each pathway's own delays: in the lab, artificial vestibular stimulation has to come about 160 ms before the other stimuli to be perceived as simultaneous (Barnett-Cowan and Harris, 2009).

Spatial coincidence. Whether the interaction is enhancement or depression, and how large it is, depends on where each stimulus falls within the neuron's receptive field. Hence the value of sensory maps that are aligned in the brain (Meredith and Stein, 1986b).

Inverse effectiveness. The percentage of enhancement grows when the response to each stimulus on its own is weak (Meredith and Stein, 1986a). Be cautious with this last one: statistical biases, including floor and ceiling effects, can create it or erase it depending on the analysis (Holmes, 2009).

The RNP training space, free for life

Get access to RNP training, no strings attached.

Welcome! Introduce yourself

SecureNo cardNo password

By continuing, you agree to our Terms of Use and our Privacy Policy.

Temporal coincidence, spatial coincidence, inverse effectiveness: the principles on the field

Carried over to sport, these three principles become questions for reading the athlete, not measurements. Here is what each one suggests on the field.

Principle

What it predicts

Field transposition (a reading, not measured)

Temporal coincidence

Two signals reinforce each other if they fall within the same time window

The sound of impact helps read the ball if it lines up with the image of contact

Spatial coincidence

Two signals reinforce each other if they come from the same place

A teammate's voice coming from where they actually are

Inverse effectiveness

The gain from fusion grows when each signal is weak

A trail runner at night, under a headlamp, with a poor view of the ground


In plain terms: two senses help each other when they talk about the same moment and the same place, and it is when each one only whispers that their union pays off most.

And when two senses don't tell the same story, which one wins?


Which sense does the brain believe when they disagree? Optimal cue combination

Optimal cue combination is the rule by which the brain gives each sense a weight proportional to its precision, like a statistical estimator known as maximum likelihood. Ernst and Banks demonstrated it with the height of an object judged by sight and by touch: vision dominates when it is most reliable and gives way to touch when it is degraded (Ernst and Banks, 2002).

So the brain believes whichever sense is most reliable at that moment. A disagreement can even produce a percept that neither sense contains: a syllable heard and dubbed onto lips articulating another one often makes people perceive a third syllable, "da" for a heard "ba" paired with an articulated "ga" (McGurk and MacDonald, 1976).

No sense gets a fixed share: a percentage assigned to vision, the inner ear or the feet only holds for the condition in which it was measured. For balance, healthy people rely more on vestibular input as movements of the support surface or the visual surroundings get larger (Peterka, 2002). This adjustment over time, sensory reweighting, has its own article.


In plain terms: when the senses disagree, the brain listens to whichever one is least wrong at that moment. Your athlete has shifting weights far more than a dominant sense.

Multisensory integration and sport: the ball, the opponent, the ground, the sound of the strike

In sport, multisensory integration is the fusion, under time pressure, of the visual, auditory, tactile, proprioceptive and vestibular cues that tell athletes about the ball, the opponent, the ground and their own body. It is the central link in information pickup in sport.

In tennis, the sound that accompanies the strike has been studied closely. On videos, a brief noise added at the moment of contact made responses about the ball's direction slower and less accurate (Sinnett and Kingstone, 2010). A more recent study showed players rallies cut off before the end, with the hitter's grunt amplified, reduced or removed: the louder the grunt, the more they predicted a long trajectory, without their prediction error increasing. The authors interpret this as sound being integrated into the prediction rather than acting as a distraction (Müller et al., 2019).

In soccer, the eyes are busy with the ball, teammates and opponents. A review reports that in soccer players as in surfers, the share of vision in balance control decreases as the level of competition rises, with proprioception and the vestibular system taking over, which frees the eyes for the game. Dance, performed in a stable environment, is the exception (Paillard, 2019). The feet then have to do double duty, as our pillar article on foot proprioception explains. For soccer players, head turns before receiving the ball are covered in our article on information pickup in soccer.


In plain terms: athletes look, but they also listen to the strike and feel the ground beneath them, and their brain melts it all into a single decision.

Sensory integration vs multisensory integration: what's the difference?

Sensory integration refers to an occupational therapy approach founded by American therapist A. Jean Ayres (Ayres Sensory Integration), which looks at how a child organizes sensations from their body and environment in order to act, play and learn. Multisensory integration refers to a mechanism studied by neuroscience: the combination of several senses into a single percept.

Sensory integration (occupational therapy)

Multisensory integration (neuroscience)

Origin

A. Jean Ayres, 1960s and 1970s (Ayres, 1972; Ayres, 1979)

Neurophysiology, Stein and Meredith from the 1980s onward

Question asked

How does a child organize sensations in order to act and learn?

How do signals from different senses combine?

Scale of study

The person in everyday activities

The neuron, the network, perceptual behavior

Methods

Observation, questionnaires, real-life scenarios run by trained professionals

Neural recordings, psychophysics, statistical models

What matters to the athlete

A history of sensory and motor development

The rules that decide how ball, sound, ground and body combine

Both fields share one intuition: the senses work together, and action depends on it. Ayres saw the vestibular system as a unifier, a reference from which the other sensations are interpreted, and the RNP manual takes up that idea. For athletes, the right term remains “multisensory integration”; our entry on Ayres Sensory Integration presents the other field in its own vocabulary.


Bottom line: same root, two professions. The first supports children in their daily lives; the second describes how the sight and sound of a strike merge in the brain.

The RNP reading of multisensory integration: read each input before judging the fusion

For RNP, multisensory integration is the tier of the loop where the three main GPS systems, vision, vestibular and somatosensory, are weighed and combined before any motor decision. Somatosensation covers proprioception and touch, including the sole of the foot; hearing, outside the three GPS, remains an input to the loop.

The onion model places this integration layer just above the core (spinal cord, brainstem, cerebellum) and below motor coordination, emotion and cognition. The manual draws a pivotal sentence from it: “As long as pure sensory input is unreliable, nothing stable can emerge above it.” The three GPS and the onion are images shaped at the LabO so that a coach can name in one second what journals write in equations. If a coach talks to you about the three GPS of their winger, they are speaking the language of our training rooms.

Hence the reading rule: read each input before judging how they combine, and optimal combination explains why. A noisy input is given less weight, silently, and the output can stay clean while one of the three sensory inputs has almost stopped saying anything, because the other two make up the difference. The manual describes the cost: an unstable sensory layer triggers motor or cognitive compensation, and a brain busy stabilizing itself processes information less well, as detailed in our article on the cognitive tax of a costly posture. The player who reads the game less well late in the match may be paying this kind of bill, a field interpretation that still needs to be measured.


Bottom line: a clean output can hide a silent input that the other two are covering for. Read each sense on its own, then how they agree.

Are three senses better than two? When adding an input adds nothing

Studies that combine three inputs answer case by case. Vision and vestibular stimulation each increase tactile sensitivity, but they do not add up when combined, and even show a sub-additive trend (Ferrè et al., 2015). A third modality only improves the perception of a hand movement if the one added to sight and touch is muscle proprioception (Blanchard et al., 2013). To recover head-on-trunk orientation, in ten subjects, neck proprioception does most of the work: when it is present alongside vision or vestibular input, adding the third sense brings nothing more (Mooti and Park, 2021). A drill that stacks lights, an unstable platform and a dual task therefore does not necessarily add up the benefits, and may mask the input that is causing the problem, a field interpretation that still needs checking. RNP follows the manual's order, sensory, sensorimotor, then motor: make each input reliable, learn to link them, then perform.

When a situation stops being predictable, the manual describes a brain that prefers to play it safe and “pulls the handbrake.” In the RNP reading, inputs that contradict each other over time make the situation less predictable and cap force output, a field hypothesis detailed in our article when the sensors contradict each other, the brain caps force; the entry on multimodal integration of the three sensory systems gives the glossary version. The RNP model reads this brake on the threat scale: two senses that contradict each other load the protection side. Our framework draws a formula specific to this topic: getting the senses to agree already means making the system safe. It applies our basic rule to contradicting senses: read the outputs, train the inputs.


Bottom line: three senses are not always better than two. Stacking stimulations does not necessarily add up their effects; make each input reliable before linking them.

What the movement professional can observe, without a homemade protocol

The reading follows the RNP manual's analysis funnel: a history that looks for weak signals, static and dynamic postural analysis, targeted tests that confirm or rule out hypotheses, then an individualized program.

Take a reproducible sporting output (single-leg stance, change of direction, landing, a striking action) and watch it when the senses agree, then when one of them is removed, scrambled or shifted. The gap between conditions, compared with the athlete's baseline, points you in a direction. A movement that falls apart as soon as the head turns does not point to the same input as one that gives way on soft ground. Removing a sense to see which one was carrying the output: strength and conditioning coaches, physical therapists and adapted physical activity instructors trained at the LabO repeat this move with their athletes, then explain it to their teams. From practitioner to team, the reading has spread out from the room where it was first laid down.

The Test–Stimulation–Retest cycle closes the reading. Regarding the vestibular test, the manual sets a demanding rule: when a sensorimotor loop is well calibrated, the retest after stimulation stays neutral, and an improvement just like a deterioration signals a sensory imbalance, and therefore an input worth a closer look. The retest tells you about the input; it does not validate the session.

Dizziness, unexplained loss of balance, or a drop in hearing or vision fall outside this scope and require a medical opinion.


Can multisensory integration be trained?

Multisensory integration builds with age and experience, and appears to remain modifiable in adults, though we do not yet know how to target it in isolation in athletes.

Before age 8, children do not combine sight and touch optimally: one of the two senses dominates entirely, even when it is the less precise one. Around 8 to 10 years, the combination becomes optimal, as in adults (Gori et al., 2008). Our pillar on sensory and motor development follows this thread from child to athlete.

In adults, a review argues that multisensory protocols, closer to natural situations, serve perceptual learning better than single-sense protocols (Shams and Seitz, 2008). On the sports side, postural adaptations remain highly specific to the practice context, with little or no transfer to unpracticed tasks (Paillard, 2017), and a review recommends that perceptual training resemble the game as closely as possible to promote transfer (Broadbent et al., 2015), as developed in our article on the transfer of neurocognitive training.

The resulting framework: train the combinations of senses the sport demands, in tasks that resemble it, with measurement as the judge. Dosage and progression belong to the reading of each athlete.


Sensory fusion: what we cannot promise you yet

Claim

Level of evidence

The brain weights each sense according to its reliability

Established (laboratory)

Temporal and spatial coincidence govern fusion

Established in animals, replicated in humans

Inverse effectiveness

Debated, sensitive to statistical analysis

Optimal combination acquired around 8 to 10 years

One landmark study, awaiting replication

The hitter's grunt changes anticipation in tennis

Shown on video, not yet on court

Training sensory fusion improves performance

Not yet measured in athletes

A weak input silently compensated for costs late in the match

RNP field interpretation

No one can guarantee you today that multisensory work will make your athlete faster at reading the game. The gap comes from a lack of studies, and an effect not yet measured is not the same as a zero effect.

The missing study would compare, in tennis players, three anticipation training conditions (real impact sound, no sound, sound shifted outside the temporal window), then measure on-court ball reading and its retention. If sensory fusion explains the gain, the synchronous-sound group improves more and the shifted-sound group does no better than the no-sound group. If all three improve equally, the sound had nothing to do with it.


Your questions about multisensory integration

What is multisensory integration?

Multisensory integration is the process by which the nervous system combines information from several senses about the same event into a single percept that is more reliable than any one sense on its own, giving the most weight to the most precise sense. In athletes, it fuses the sight of the ball, the sound of the strike and the feel of foot contact with the ground.

What is the difference between sensory integration and multisensory integration?

Sensory integration is an occupational therapy approach founded by A. Jean Ayres. It focuses on how a child organizes sensations in order to act and learn. Multisensory integration is a mechanism studied in neuroscience: the combination of signals from different senses into one percept. Both rest on the idea that the senses work together.

What are the principles of multisensory integration?

Stein and Meredith described three principles, based on neurons in the superior colliculus. Two signals reinforce each other if they arrive within the same time window (temporal coincidence) and come from the same location (spatial coincidence). The gain is larger when each signal is weak on its own (inverse effectiveness), a principle that is still debated.

Which sense does the brain trust when the senses disagree?

The brain trusts whichever sense is most reliable in that situation and weights the others according to their precision. Ernst and Banks showed this in 2002: vision dominates as long as it is sharp, then gives way to touch when it is degraded. No sense keeps a fixed share. Its weight changes with the context.

Can multisensory integration be trained?

It develops with age and experience. Optimal combination of vision and touch appears around 8 to 10 years of age. In adults, multisensory protocols appear to support perceptual learning better. In athletes, the effect on performance has yet to be measured, and transfer seems to depend on how closely the training resembles the game.

Why does multisensory integration matter for athletes?

Athletes make fast decisions on incomplete information. Fusing what they see, the sound of the strike and what they feel through their feet gives a more reliable estimate than any single sense. In tennis, making the hitter's grunt louder leads opponents to predict a longer ball trajectory. A weak input can also be silently compensated by the other senses, and this is what the movement professional learns to read.

What is the superior colliculus?

The superior colliculus is a paired structure on the roof of the midbrain. It overlays visual, auditory and somatosensory maps of space and drives orienting movements of the eyes and head. The principles of multisensory integration were first described in its neurons. It links the fusion of the senses to an action: turning toward what matters.

Is multisensory integration a lab concept or a coaching tool?

It started as a lab concept: Stein and Meredith laid down its principles in the superior colliculus in the 1980s, and Ernst and Banks set out the weighting rule in 2002. It becomes a coaching tool within the framework LabO RNP has taught movement professionals since 2020. It comes down to a question you ask about the athlete: which sense is holding steady, and which one is giving false information?


Sources

Alais D, Burr D. The ventriloquist effect results from near-optimal bimodal integration. Curr Biol. 2004;14(3):257-262. PMID 14761661. DOI 10.1016/j.cub.2004.01.029.

Ayres AJ. Sensory integration and learning disorders. Western Psychological Services; 1972.

Ayres AJ. Sensory integration and the child. Western Psychological Services; 1979.

Barnett-Cowan M, Harris LR. Perceived timing of vestibular stimulation relative to touch, light and sound. Exp Brain Res. 2009;198(2-3):221-231. PMID 19352639. DOI 10.1007/s00221-009-1779-4.

Blanchard C, Roll R, Roll JP, Kavounoudias A. Differential contributions of vision, touch and muscle proprioception to the coding of hand movements. PLoS One. 2013;8(4):e62475. PMID 23626826. DOI 10.1371/journal.pone.0062475.

Broadbent DP, Causer J, Williams AM, Ford PR. Perceptual-cognitive skill training and its transfer to expert performance in the field: future research directions. Eur J Sport Sci. 2015;15(4):322-331. PMID 25252156. DOI 10.1080/17461391.2014.957727.

Ernst MO, Banks MS. Humans integrate visual and haptic information in a statistically optimal fashion. Nature. 2002;415(6870):429-433. PMID 11807554. DOI 10.1038/415429a.

Ferrè ER, Walther LE, Haggard P. Multisensory interactions between vestibular, visual and somatosensory signals. PLoS One. 2015;10(4):e0124573. PMID 25875819. DOI 10.1371/journal.pone.0124573.

Gori M, Del Viva M, Sandini G, Burr DC. Young children do not integrate visual and haptic form information. Curr Biol. 2008;18(9):694-698. PMID 18450446. DOI 10.1016/j.cub.2008.04.036.

Holmes NP. The principle of inverse effectiveness in multisensory integration: some statistical considerations. Brain Topogr. 2009;21(3-4):168-176. PMID 19404728. DOI 10.1007/s10548-009-0097-2.

McGurk H, MacDonald J. Hearing lips and seeing voices. Nature. 1976;264(5588):746-748. PMID 1012311. DOI 10.1038/264746a0.

Meredith MA, Stein BE. Visual, auditory, and somatosensory convergence on cells in superior colliculus results in multisensory integration. J Neurophysiol. 1986;56(3):640-662. PMID 3537225. DOI 10.1152/jn.1986.56.3.640. (cited as 1986a)

Meredith MA, Stein BE. Spatial factors determine the activity of multisensory neurons in cat superior colliculus. Brain Res. 1986;365(2):350-354. PMID 3947999. DOI 10.1016/0006-8993(86)91648-3. (cited as 1986b)

Meredith MA, Nemitz JW, Stein BE. Determinants of multisensory integration in superior colliculus neurons. I. Temporal factors. J Neurosci. 1987;7(10):3215-3229. PMID 3668625. DOI 10.1523/JNEUROSCI.07-10-03215.1987.

Mooti R, Park H. Contribution of cervical proprioception, vision, and vestibular feedback on reducing dynamic head-trunk orientation error in the yaw direction. Front Neurosci. 2021;15:774448. PMID 35140583. DOI 10.3389/fnins.2021.774448.

Müller F, Jauernig L, Cañal-Bruland R. The sound of speed: how grunting affects opponents' anticipation in tennis. PLoS One. 2019;14(4):e0214819. PMID 30986241. DOI 10.1371/journal.pone.0214819.

Paillard T. Plasticity of the postural function to sport and/or motor experience. Neurosci Biobehav Rev. 2017;72:129-152. PMID 27894829. DOI 10.1016/j.neubiorev.2016.11.015.

Paillard T. Relationship between sport expertise and postural skills. Front Psychol. 2019;10:1428. PMID 31293483. DOI 10.3389/fpsyg.2019.01428.

Peterka RJ. Sensorimotor integration in human postural control. J Neurophysiol. 2002;88(3):1097-1118. PMID 12205132. DOI 10.1152/jn.2002.88.3.1097.

Shams L, Seitz AR. Benefits of multisensory learning. Trends Cogn Sci. 2008;12(11):411-417. PMID 18805039. DOI 10.1016/j.tics.2008.07.006.

Sinnett S, Kingstone A. A preliminary investigation regarding the effect of tennis grunting: does white noise during a tennis shot have a negative impact on shot perception? PLoS One. 2010;5(10):e13148. PMID 20957210. DOI 10.1371/journal.pone.0013148.

Stein BE, Meredith MA. The merging of the senses. MIT Press; 1993.

Stein BE, Stanford TR. Multisensory integration: current issues from the perspective of the single neuron. Nat Rev Neurosci. 2008;9(4):255-266. PMID 18354398. DOI 10.1038/nrn2331.


The laws governing how the senses merge are published. Checking that each of your athlete's senses has something reliable to say is hands-on work, and that work has a home: the LabO.

👉 I want to be one of the professionals who read the loop upstream.

To lay the foundations of this approach, the Introduction to RNP is free.

By the LabO RNP team

LabO RNP is neither medical nor paramedical. It complements a course of care and never replaces it.

€0Your free RNP PathsStart