[2025-09-23] 📚 Updates in #DiffM

(1) Uni3C: Unifying Precisely 3D-Enhanced Camera and Human Motion Controls for Video Generation
(2) Optimal Transport for Rectified Flow Image Editing: Unifying Inversion-Based and Direct Methods

🔍 More at researchtrend.ai/communities/DiffM

[2025-09-23] 📚 Updates in #VGen

(1) Uni3C: Unifying Precisely 3D-Enhanced Camera and Human Motion Controls for Video Generation
(2) The Sound of Simulation: Learning Multimodal Sim-to-Real Robot Policies with Generative Audio

🔍 More at researchtrend.ai/communities/VGen

Uni3C adds a plug‑and‑play PCDController that turns monocular depth maps into 3D point clouds for steering; it outperforms competitors in controllability and motion quality. Read more: https://getnews.me/uni3c-unified-3d-camera-and-human-motion-control-for-video-generation/ #uni3c #pcdcontroller

Chenjie Cao, Jingkai Zhou, Shikai Li, Jingyun Liang, Chaohui Yu, Fan Wang, Xiangyang Xue, Yanwei Fu
Uni3C: Unifying Precisely 3D-Enhanced Camera and Human Motion Controls for Video Generation
https://arxiv.org/abs/2504.14899

Chenjie Cao, Jingkai Zhou, Shikai Li, Jingyun Liang, Chaohui Yu, Fan Wang, Xiangyang Xue, Yanwei Fu: Uni3C: Unifying Precisely 3D-Enhanced Camera and Human Motion Controls for Video Generation https://arxiv.org/abs/2504.14899 https://arxiv.org/pdf/2504.14899 https://arxiv.org/html/2504.14899

to unify the control signals for camera and human motion, respectively. Extensive experiments confirm that PCDController enjoys strong robustness in driving camera motion for fine-tuned backbones of video generation. Uni3C substantially outperforms [6/7 of https://arxiv.org/abs/2504.14899v1]

Testing my #renderformer wrapper and #uni3c #controlnet for camera control in #WAN 2.1 video generation 📽️

In this example i am using a rendered 3d animation as a driver for the img2video generation without leaving @ComfyUI.

#ComfyUI #WANVideo #Ai #AIinArchitecture #AIGeneratedVisuals

github.com/alibaba-damo...
Uni3C
Allows to control camera and human motions in AI-generated videos.

Given a single-view image across various domains (real-world, text-to-image, animation), the proposed Uni3C can generate videos under specific camera trajectories, human motion characters or both of these conditions.
ewrfcas.github.io/Uni3C/

foundational models, PCDController shows impressive generalization, performing well regardless of whether the inference backbone is frozen or fine-tuned. This flexibility enables different modules of Uni3C to be trained in specific domains, i.e., [4/7 of https://arxiv.org/abs/2504.14899v1]

for both aspects. To overcome this, we present Uni3C, a unified 3D-enhanced framework for precise control of both camera and human motion in video generation. Uni3C includes two key contributions. First, we propose a plug-and-play control module [2/7 of https://arxiv.org/abs/2504.14899v1]