When Temporal Perturbations Act Like Sensor Biases: Label-Free Auditing of Wearable Activity Recognizers

Summary

This paper introduces SpectrumAudit, a novel method for auditing wearable activity recognition models by simulating sensor biases through temporal perturbations, demonstrating significant accuracy losses and distinguishing between DC and AC effects.

Key findings

• SpectrumAudit causes 2.87-40.83-point robust accuracy losses across 27 victims.
• DC perturbations are more damaging than AC on 24/27 victims and recover at least 90% of the full drop on 22/27.
• In a held-out UTD-MHAD check, the selected waveform causes 13.49-pp accuracy and 11.68-pp macro-F1 losses.
• Matched random changes result in minimal accuracy loss (-0.66 pp).

Methods

• SpectrumAudit: a label-sealed audit fitting a phase-randomized full-window stimulus on calibration windows.
• Replaying the exact DC projection and budget-constrained zero-mean residual on frozen victims without refitting.
• Testing across 27 victims from three datasets and three backbones.
• Held-out UTD-MHAD check to evaluate accuracy and macro-F1 losses.

Limitations

• The study focuses on specific types of perturbations (DC and AC) and may not cover all potential sensor biases.
• Performance of DC perturbations varies across datasets, with failures noted on WISDM.
• The code is to be released upon acceptance, limiting immediate replication.
• The study does not explore the impact of these perturbations on real-world deployment scenarios beyond the tested datasets.

Applications

• Auditing the robustness of wearable activity recognition systems.
• Identifying vulnerabilities in HAR models to sensor biases.
• Improving the reliability and security of wearable devices.
• Developing more resilient HAR algorithms.