Channel-Specific Stimulus Sensitivity, Reliability, and Complementarity of Multimodal Wearable Sensing During Short-Form Influencer Video Exposure
Journal
Sensors
ISSN
14248220
Date Issued
2026-10-08
Author(s)
Del-Valle-Soto, Carolina
Corona, Violeta
Gómez Padilla, Regina
Walls Fernández, Fátima Sofía
López-Pimentel, Juan-Carlos
Mex-Perera, Carlos
Visconti, Paolo
Type
Article
Abstract
Multimodal wearable sensing is increasingly used to quantify audience engagement with short-form video, yet it remains unclear which sensing channels actually carry stimulus-driven information at the exposure durations typical of this format. We recorded electrodermal activity, photoplethysmographic pulse, triaxial acceleration, video-based eye tracking, and automated facial expression analysis from 50 young adults while they viewed 20 short-form influencer clips (5.6–19.3 s) presented in individually randomized order, yielding 1000 participant × clip trials and 12 primary indices spanning four modalities. Using crossed random-effects variance partitioning, split-half reliability with Spearman–Brown projection, permutation-based multivariate testing, participant-wise decoding, and inter-subject correlation, we show that stimulus-driven variance is strongly channel-specific: it reaches 13.7% for fixation rate and 10.2% for facial engagement but only 0.11% for pulse rate and 0.23% for RMSSD. Clip-level estimates were highly reliable for facial and oculomotor indices ((Formula presented.) – (Formula presented.)) and poor for autonomic indices ((Formula presented.) – (Formula presented.)), which would require 134–370 participants to reach (Formula presented.). Between-modality dependence was almost six times weaker than within-modality dependence, and five components were retained by parallel analysis. Oculomotor and facial features jointly identified individual clips at 16.8% balanced accuracy against a 5% chance level, whereas electrodermal and pulse features, taken separately, did not exceed chance. Electrodermal responses were nonetheless stimulus-locked in 18 of 20 clips. A per-second re-analysis of nine channels showed that the ordering partly reverses for within-clip dynamics: tonic electrodermal and pupil time courses were reliable across viewers ((Formula presented.) and (Formula presented.)), facial time courses were not ((Formula presented.)), and pulse rate was unreliable under both treatments. We derive explicit design guidance for sensor selection and sample size in short-exposure multimodal engagement studies. © 2026 by the authors.