BABEL PJ/AUJ Reproducibility: Empty `test_processed.txt` and Missing Evaluation Protocol

[bring-nirachornkul]

Title: BABEL PJ/AUJ Reproducibility: Empty test_processed.txt and Missing Evaluation Protocol

Hello, thank you for releasing the FloodDiffusion code and checkpoints.

We are trying to reproduce the BABEL PJ/AUJ results reported in the paper:

• PJ: 0.713
• AUJ: 14.05

However, we found that test_processed.txt in the released BABEL dataset is empty. We were also unable to locate the exact evaluation artifacts needed to reproduce the reported results.

Could you please clarify or provide the following?

1. The exact BABEL test manifest used for the reported PJ/AUJ results.
2. The script used to construct or compose the fixed-length evaluation sequences and transition windows.
3. The exact BABEL-263 GT-jerk scalar used for AUJ, along with the script or procedure used to calculate it.
4. The transition-window length and frame rate used during evaluation.
5. The command and configuration used to generate and evaluate the reported BABEL results.
6. The valid-length, padding, cropping, and sequence-aggregation conventions.
7. If available, the generated-motion IDs or output files used for the Real, PRIMAL, MotionStreamer, and FloodDiffusion rows.

We verified the FlowMDM/Seamless calculate_jerk and evaluate_jerk implementations, but the released variable-length BABEL validation data does not reproduce the reported Real-motion PJ/AUJ values. This suggests that an additional composition or evaluation procedure may have been used.

Any clarification or missing artifacts would greatly help us reproduce the BABEL results fairly and accurately.

Thank you.

[caiyy17]

Thank you for pointing this out. The BABEL values reported in the paper were computed by applying the jerk calculation directly to the raw 263-D motion features, rather than to recovered 3D joint trajectories. Since the 263-D representation includes rotation and other non-position channels, these numbers are a raw-feature smoothness measure and should not be interpreted as standard joint-space PJ/AUJ. We apologize for the unclear labeling. BABEL test set has no frame level labels, so the metrics are calculated using the validation set.