Learning under label noise through few-shot human-in-the-loop refinement
Abstract Wearable technologies enable continuous monitoring of various health metrics, such as physical activity, heart rate, sleep, and stress levels. A key challenge with wearable data is obtaining quality labels. Unlike modalities like video where the videos themselves can be effectively used to...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2025-02-01
|
| Series: | Scientific Reports |
| Subjects: | |
| Online Access: | https://doi.org/10.1038/s41598-025-87046-z |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1823862516104560640 |
|---|---|
| author | Aaqib Saeed Dimitris Spathis Jungwoo Oh Edward Choi Ali Etemad |
| author_facet | Aaqib Saeed Dimitris Spathis Jungwoo Oh Edward Choi Ali Etemad |
| author_sort | Aaqib Saeed |
| collection | DOAJ |
| description | Abstract Wearable technologies enable continuous monitoring of various health metrics, such as physical activity, heart rate, sleep, and stress levels. A key challenge with wearable data is obtaining quality labels. Unlike modalities like video where the videos themselves can be effectively used to label objects or events, wearable data do not contain obvious cues about the physical manifestation of the users and usually require rich metadata. As a result, label noise can become an increasingly thorny issue when labeling wearable data. In this paper, we propose a novel solution to address noisy label learning, entitled Few-Shot Human-in-the-Loop Refinement (FHLR). Our method initially learns a seed model using weak labels. Next, it fine-tunes the seed model using a handful of expert corrections. Finally, it achieves better generalizability and robustness by merging the seed and fine-tuned models via weighted parameter averaging. We evaluate our approach on four challenging tasks and datasets, and compare it against eight competitive baselines designed to deal with noisy labels. We show that FHLR achieves significantly better performance when learning from noisy labels and achieves state-of-the-art by a large margin, with up to $$19\%$$ accuracy improvement under symmetric and asymmetric noise. Notably, we find that FHLR is particularly robust to increased label noise, unlike prior works that suffer from severe performance degradation. Our work not only achieves better generalization in high-stakes health sensing benchmarks but also sheds light on how noise affects commonly-used models. |
| format | Article |
| id | doaj-art-d01bbd75a785476ea5ce9517948f8d82 |
| institution | Kabale University |
| issn | 2045-2322 |
| language | English |
| publishDate | 2025-02-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj-art-d01bbd75a785476ea5ce9517948f8d822025-02-09T12:29:35ZengNature PortfolioScientific Reports2045-23222025-02-0115111010.1038/s41598-025-87046-zLearning under label noise through few-shot human-in-the-loop refinementAaqib Saeed0Dimitris Spathis1Jungwoo Oh2Edward Choi3Ali Etemad4Eindhoven University of TechnologyNokia Bell LabsKAISTKAISTQueen’s UniversityAbstract Wearable technologies enable continuous monitoring of various health metrics, such as physical activity, heart rate, sleep, and stress levels. A key challenge with wearable data is obtaining quality labels. Unlike modalities like video where the videos themselves can be effectively used to label objects or events, wearable data do not contain obvious cues about the physical manifestation of the users and usually require rich metadata. As a result, label noise can become an increasingly thorny issue when labeling wearable data. In this paper, we propose a novel solution to address noisy label learning, entitled Few-Shot Human-in-the-Loop Refinement (FHLR). Our method initially learns a seed model using weak labels. Next, it fine-tunes the seed model using a handful of expert corrections. Finally, it achieves better generalizability and robustness by merging the seed and fine-tuned models via weighted parameter averaging. We evaluate our approach on four challenging tasks and datasets, and compare it against eight competitive baselines designed to deal with noisy labels. We show that FHLR achieves significantly better performance when learning from noisy labels and achieves state-of-the-art by a large margin, with up to $$19\%$$ accuracy improvement under symmetric and asymmetric noise. Notably, we find that FHLR is particularly robust to increased label noise, unlike prior works that suffer from severe performance degradation. Our work not only achieves better generalization in high-stakes health sensing benchmarks but also sheds light on how noise affects commonly-used models.https://doi.org/10.1038/s41598-025-87046-zData-centric machine learningRobustnessHuman-in-the-loopModel mergingLabel noise |
| spellingShingle | Aaqib Saeed Dimitris Spathis Jungwoo Oh Edward Choi Ali Etemad Learning under label noise through few-shot human-in-the-loop refinement Scientific Reports Data-centric machine learning Robustness Human-in-the-loop Model merging Label noise |
| title | Learning under label noise through few-shot human-in-the-loop refinement |
| title_full | Learning under label noise through few-shot human-in-the-loop refinement |
| title_fullStr | Learning under label noise through few-shot human-in-the-loop refinement |
| title_full_unstemmed | Learning under label noise through few-shot human-in-the-loop refinement |
| title_short | Learning under label noise through few-shot human-in-the-loop refinement |
| title_sort | learning under label noise through few shot human in the loop refinement |
| topic | Data-centric machine learning Robustness Human-in-the-loop Model merging Label noise |
| url | https://doi.org/10.1038/s41598-025-87046-z |
| work_keys_str_mv | AT aaqibsaeed learningunderlabelnoisethroughfewshothumaninthelooprefinement AT dimitrisspathis learningunderlabelnoisethroughfewshothumaninthelooprefinement AT jungwoooh learningunderlabelnoisethroughfewshothumaninthelooprefinement AT edwardchoi learningunderlabelnoisethroughfewshothumaninthelooprefinement AT alietemad learningunderlabelnoisethroughfewshothumaninthelooprefinement |