Poster Type: Research Posters
Author: Julia Gersey (University of Michigan, Argonne National Laboratory), Nicola Ferrier (Argonne National Laboratory, Northwestern University)
Abstract: Multimodal environmental sensing networks routinely capture temporally co-occurring conditions across incompatible representations: an RGB frame, a thermal frame, an audio clip, and environmental readings share no common feature space. We present MATCH, a contrastive learning framework that combines frozen, domain-matched pretrained backbones with lightweight trainable projection heads to align real-world node data across all four modalities in a shared 512-dimensional embedding space without task-specific labels. A multi-view InfoNCE loss aligns all six modality pairs. Evaluated on a 195-observation held-out test set from a single deployed node, retrieval improves over random across all six pairs, with RGB↔Thermal strongest — notably, 73.3% of its incorrect top-5 retrievals fall within ±2 hours of the correct time of day, versus a 20.4% chance baseline, indicating the learned space captures coarse diurnal structure.
At the poster we'll present our architecture, cross-modal retrieval examples, quantitative alignment results, and discuss next steps toward node-specific adaptation.
Best Poster Finalist (BP): no
Poster: PDF
Poster Summary: PDF