Abstract

Digital dance and theater archives are difficult to search and study because few tools can accurately extract 3D motion from the monocular, often degraded footage that dominates heritage collections. We present two cloud-ready pipelines that transform these videos into temporally dense SMPL-X reconstructions, per-frame segmentation masks, depth maps, and estimated camera trajectories. The PREMIERE pipeline targets archival material, while the MultiPerson pipeline is tuned for high-frame-rate smartphone or action camera recordings with up to five performers. Both pipelines couple Segment Anything v2.1, Neural Localizer Fields pose recovery, MoGe-based depth cues, WiLoR hand refinement and VGGT for camera parameters estimation. This is followed by scale-aware optimization and RBF smoothing.

Extensive testing on the AIST++ dance data set confirms that the resulting 3D assets surpass earlier monocular baselines in pose fidelity and temporal stability, while remaining robust to motion blur, extreme lighting and human occlusion on stage film. Outputs conform to a unified geometry schema that feeds two open-source WebGL viewers, enabling browser-based playback, VR/MR immersion and high-resolution render export with no additional recording hardware. All code, evaluation scripts and viewers will be released under an open licence, offering a reproducible, extensible foundation for large-scale motion mining and the preservation of intangible cultural heritage.