Scalable Motion Style Transfer with Constrained Diffusion Generation
Publikation: Bidrag til tidsskrift › Konferenceartikel › Forskning › fagfællebedømt
Standard
Scalable Motion Style Transfer with Constrained Diffusion Generation. / Yin, Wenjie; Yu, Yi; Yin, Hang; Kragic, Danica; Björkman, Mårten.
I: Proceedings of the AAAI Conference on Artificial Intelligence, Nr. 9, 2024, s. 10234-10242.Publikation: Bidrag til tidsskrift › Konferenceartikel › Forskning › fagfællebedømt
Harvard
APA
Vancouver
Author
Bibtex
}
RIS
TY - GEN
T1 - Scalable Motion Style Transfer with Constrained Diffusion Generation
AU - Yin, Wenjie
AU - Yu, Yi
AU - Yin, Hang
AU - Kragic, Danica
AU - Björkman, Mårten
N1 - Publisher Copyright: Copyright © 2024, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved.
PY - 2024
Y1 - 2024
N2 - Current training of motion style transfer systems relies on consistency losses across style domains to preserve contents, hindering its scalable application to a large number of domains and private data. Recent image transfer works show the potential of independent training on each domain by leveraging implicit bridging between diffusion models, with the content preservation, however, limited to simple data patterns. We address this by imposing biased sampling in backward diffusion while maintaining the domain independence in the training stage. We construct the bias from the source domain keyframes and apply them as the gradient of content constraints, yielding a framework with keyframe manifold constraint gradients (KMCGs). Our validation demonstrates the success of training separate models to transfer between as many as ten dance motion styles. Comprehensive experiments find a significant improvement in preserving motion contents in comparison to baseline and ablative diffusion-based style transfer models. In addition, we perform a human study for a subjective assessment of the quality of generated dance motions. The results validate the competitiveness of KMCGs.
AB - Current training of motion style transfer systems relies on consistency losses across style domains to preserve contents, hindering its scalable application to a large number of domains and private data. Recent image transfer works show the potential of independent training on each domain by leveraging implicit bridging between diffusion models, with the content preservation, however, limited to simple data patterns. We address this by imposing biased sampling in backward diffusion while maintaining the domain independence in the training stage. We construct the bias from the source domain keyframes and apply them as the gradient of content constraints, yielding a framework with keyframe manifold constraint gradients (KMCGs). Our validation demonstrates the success of training separate models to transfer between as many as ten dance motion styles. Comprehensive experiments find a significant improvement in preserving motion contents in comparison to baseline and ablative diffusion-based style transfer models. In addition, we perform a human study for a subjective assessment of the quality of generated dance motions. The results validate the competitiveness of KMCGs.
U2 - 10.1609/aaai.v38i9.28889
DO - 10.1609/aaai.v38i9.28889
M3 - Conference article
AN - SCOPUS:85189340183
SP - 10234
EP - 10242
JO - AAAI Conference on Artificial Intelligence
JF - AAAI Conference on Artificial Intelligence
SN - 2159-5399
IS - 9
T2 - 38th AAAI Conference on Artificial Intelligence, AAAI 2024
Y2 - 20 February 2024 through 27 February 2024
ER -
ID: 390997809