Multimodal Signal Processing and Machine Learning for Edge-Based Health Monitoring

Master’s level project (“Projektarbeit"), may be offered either as a single- or a two-semester project

Prerequisites: Knowledge of digital signal processing and basic knowledge of statistical signal processing and machine learning. Programming skills in Python.

Open BCI EEG Headset (image credit: openbci.com) and Raspberry Pi (source: raspberrypi.com)

First kick-off meeting: Tuesday, October 13 at 1pm in P1.7.13.4

Contact: Dr. Tanuj Hasija

Background

Wearable devices make it possible to record information about both brain activity and the body during everyday activities. Electroencephalography (EEG) devices measure electrical activity in the brain, while wearable sensors—such as those found in smartwatches—can capture signals including pulse, skin conductance, skin temperature and movement. Together, these different data sources can provide a richer picture of a person’s physiological state than either source alone. Turning raw sensor signals into useful information presents several challenges. Signals can be noisy, devices may produce data at different rates, and a sensor may sometimes provide unreliable or incomplete measurements. Signal processing and machine learning can help address these challenges and identify patterns associated with states such as emotion, stress, and sleep, as well as seizure-related activity. Ideally, these models should also run on a small edge device rather than relying entirely on a remote server, enabling low-latency processing, lower energy consumption, and greater privacy.

Your task

In this project, you will develop a prototype system for multimodal health monitoring using EEG and wearable data. You will begin by working with publicly available datasets containing EEG and peripheral physiological signals to develop and evaluate models. You will also collect your own synchronized recordings using an EEG device (e.g., OpenBCI) and a wearable sensor (e.g., EmotiBit) to test the data-collection and analysis pipeline. You will process the signals to extract useful information—for example, frequency-band features from EEG and pulse or skin-conductance features from wearable data. You will compare different signal-processing and machine-learning approaches and investigate the fusion of EEG and wearable signals. An important part of the project will be making the system robust to real-world conditions, such as noisy signals, unreliable sensors, or a temporarily missing data stream. The final aim is to run a trained model on a Raspberry Pi 5 with an AI accelerator, with the option of developing a simple Android application to display signal quality and model outputs.

Notentranskriptor: MP3 zu MIDI

Bachelor Projekt-/Proseminar

Voraussetzungen: Signaltheorie. Programmiererfahrung in Python von Vorteil, aber nicht unbedingt notwendig.

Wird momentan nicht angeboten

Hintergrund

In diesem Bachelor Projekt-/Proseminar sollen MP3-Dateien zu MIDI konvertiert werden. Jede Note entspricht einer Frequenz, z.B. 440Hz für den Kammerton A. Mithilfe der Short-Term Fourier-Transformation können die gespielten Frequenzen zu verschiedenen Zeitpunkten erkannt werden und so die gespielten Noten über die Zeit aufgetragen werden.

Was wirst du lernen/anwenden?

Signaltheorie: FFT/STFT
Signalverarbeitung: Peak detection
Programmieren in Python
Verschiedene Dateiformate (MP3, MIDI)