DelPreto, JosephHughes, JosieD'Aria, Matteode Fazio, MarcoRus, Daniela2022-08-092022-08-092022-08-09202210.1109/LRA.2022.3191232https://infoscience.epfl.ch/handle/20.500.14299/189879Advances in soft sensors coupled with machine learning are enabling increasingly capable wearable systems. Since hand motion in particular can convey useful information for developing intuitive interfaces, glove-based systems can have a significant impact on many application areas. A key remaining challenge for wearables is to capture, process, and analyze data from the high-degree-of-freedom hand in real time. We propose using a commercially available conductive knit to create an unobtrusive network of resistive sensors that spans all hand joints, coupling this with an accelerometer, and deploying machine learning on a low-profile microcontroller to process and classify data. This yields a self-contained wearable device with rich sensing capabilities for hand pose and orientation, low fabrication time, and embedded activity prediction. To demonstrate its capabilities, we use it to detect static poses and dynamic gestures from American Sign Language (ASL). By pre-training a long short-term memory (LSTM) neural network and using tools to deploy it in an embedded context, the glove and an ST microcontroller can classify 12 ASL letters and 12 ASL words in real time. Using a leave-one-experiment-out cross validation methodology, networks successfully classify 96.3% of segmented examples and generate correct rolling predictions during 92.8% of real-time streaming trials. AuthorCapacitive sensorsClassification (of information)ControllersDegrees of freedom (mechanics)Embedded systemsInteractive computer systemsLearning systemsLong short-term memoryMicrocontrollersReal time systemsWearable sensorsAmerican sign languageEmbedded-systemFacial ExpressionsGesture, posture, and facial expressionReal - Time systemRobot sensing systemSensors and actuatorsSoft actuatorsSoft sensorsWearable roboticsAccelerometerstext::journal::journal article::research article