Weakly-Supervised Deep Shape-From-Template
We propose WS-DeepSfT, a novel deep learning-based approach to the Shape-from-Template (SfT) problem, which aims at reconstructing the 3D shape of a deformable object from a single RGB image and a template. WS-DeepSfT addresses the limitations of existing SfT techniques by combining a weakly-supervi...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10854467/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1825207022748958720 |
|---|---|
| author | Sara Luengo-Sanchez David Fuentes-Jimenez Cristina Losada-Gutierrez Daniel Pizarro Adrien Bartoli |
| author_facet | Sara Luengo-Sanchez David Fuentes-Jimenez Cristina Losada-Gutierrez Daniel Pizarro Adrien Bartoli |
| author_sort | Sara Luengo-Sanchez |
| collection | DOAJ |
| description | We propose WS-DeepSfT, a novel deep learning-based approach to the Shape-from-Template (SfT) problem, which aims at reconstructing the 3D shape of a deformable object from a single RGB image and a template. WS-DeepSfT addresses the limitations of existing SfT techniques by combining a weakly-supervised deep neural network (DNN) for registration and a classical As-Rigid-As-Possible (ARAP) algorithm for 3D reconstruction. Unlike previous deep learning-based SfT methods, which require extensive synthetic data and depth sensors for training, WS-DeepSfT only requires regular RGB video of the deforming object and a segmentation mask to discriminate the object from the background. The registration model is trained without synthetic data, using videos where the object undergoes deformations, while ARAP does not require training and infers the 3D shape in real-time with minimal overhead. We show that WS-DeepSfT outperforms the state-of-the-art, in both accuracy and robustness, without requiring depth sensors or synthetic data generation. WS-DeepSfT thus offers a robust, efficient, and scalable approach to SfT, bringing it closer to applications such as augmented reality. |
| format | Article |
| id | doaj-art-64e517b6bf4f44cebad874cd8a0f0155 |
| institution | Kabale University |
| issn | 2169-3536 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-64e517b6bf4f44cebad874cd8a0f01552025-02-07T00:01:29ZengIEEEIEEE Access2169-35362025-01-0113228682289210.1109/ACCESS.2025.353427110854467Weakly-Supervised Deep Shape-From-TemplateSara Luengo-Sanchez0https://orcid.org/0000-0003-3942-3804David Fuentes-Jimenez1https://orcid.org/0000-0001-6424-4782Cristina Losada-Gutierrez2https://orcid.org/0000-0001-9545-327XDaniel Pizarro3https://orcid.org/0000-0003-0622-4884Adrien Bartoli4https://orcid.org/0000-0003-3545-7329Department of Electronics, Universidad de Alcalá (UAH), Alcala de Henares, SpainDepartment of Electronics, Universidad de Alcalá (UAH), Alcala de Henares, SpainDepartment of Electronics, Universidad de Alcalá (UAH), Alcala de Henares, SpainDepartment of Electronics, Universidad de Alcalá (UAH), Alcala de Henares, SpainEnCoV, Clermont-Ferrand, FranceWe propose WS-DeepSfT, a novel deep learning-based approach to the Shape-from-Template (SfT) problem, which aims at reconstructing the 3D shape of a deformable object from a single RGB image and a template. WS-DeepSfT addresses the limitations of existing SfT techniques by combining a weakly-supervised deep neural network (DNN) for registration and a classical As-Rigid-As-Possible (ARAP) algorithm for 3D reconstruction. Unlike previous deep learning-based SfT methods, which require extensive synthetic data and depth sensors for training, WS-DeepSfT only requires regular RGB video of the deforming object and a segmentation mask to discriminate the object from the background. The registration model is trained without synthetic data, using videos where the object undergoes deformations, while ARAP does not require training and infers the 3D shape in real-time with minimal overhead. We show that WS-DeepSfT outperforms the state-of-the-art, in both accuracy and robustness, without requiring depth sensors or synthetic data generation. WS-DeepSfT thus offers a robust, efficient, and scalable approach to SfT, bringing it closer to applications such as augmented reality.https://ieeexplore.ieee.org/document/10854467/Non-rigidShape-from-Templateweak-supervisionregistrationwide-baselinetemplate-based |
| spellingShingle | Sara Luengo-Sanchez David Fuentes-Jimenez Cristina Losada-Gutierrez Daniel Pizarro Adrien Bartoli Weakly-Supervised Deep Shape-From-Template IEEE Access Non-rigid Shape-from-Template weak-supervision registration wide-baseline template-based |
| title | Weakly-Supervised Deep Shape-From-Template |
| title_full | Weakly-Supervised Deep Shape-From-Template |
| title_fullStr | Weakly-Supervised Deep Shape-From-Template |
| title_full_unstemmed | Weakly-Supervised Deep Shape-From-Template |
| title_short | Weakly-Supervised Deep Shape-From-Template |
| title_sort | weakly supervised deep shape from template |
| topic | Non-rigid Shape-from-Template weak-supervision registration wide-baseline template-based |
| url | https://ieeexplore.ieee.org/document/10854467/ |
| work_keys_str_mv | AT saraluengosanchez weaklysuperviseddeepshapefromtemplate AT davidfuentesjimenez weaklysuperviseddeepshapefromtemplate AT cristinalosadagutierrez weaklysuperviseddeepshapefromtemplate AT danielpizarro weaklysuperviseddeepshapefromtemplate AT adrienbartoli weaklysuperviseddeepshapefromtemplate |