The End Of Knowledge - Vault 5/70 - CVPR - 2021 - Modeling Physical Interaction

graph LR classDef biomechanics fill:#f9d4d4, font-weight:bold, font-size:14px classDef subspaces fill:#d4f9d4, font-weight:bold, font-size:14px classDef learning fill:#d4d4f9, font-weight:bold, font-size:14px classDef deformation fill:#f9f9d4, font-weight:bold, font-size:14px classDef clothing fill:#f9d4f9, font-weight:bold, font-size:14px classDef skin fill:#d4f9f9, font-weight:bold, font-size:14px A[Modeling Physical Interaction
- How to
Build Expressive Subspaces] --> B[Simulating biomechanics, balancing
accuracy, generality, efficiency. 1] A --> C[Machine learning improves
subspace representations. 2] C --> D[Connecting learning methods
with physics-based subspaces. 3] C --> E[Compact non-linear, evolving
linear subspaces. 4] A --> F[Designing expressive deformation
subspaces. 5] F --> G[Including contact interactions
in subspace. 6] F --> H[Deriving skin model: pose,
shape, dynamics. 7] H --> I[Limited shape samples,
inter-subject variability challenges. 8] H --> J[Separating global, local
pose, shape deformations. 9] H --> K[Plausible dynamics for
arbitrary body shapes. 10] F --> L[Learning geometric deformations
leveraging subspace. 11] F --> M[Accounting for contact
when learning deformations. 12] A --> N[Learning for efficient
virtual clothing try-on. 13] N --> O[Challenges: loose garments,
collisions, post-processing. 14] N --> P[Canonical garment space,
constant body. 15] P --> Q[Un-posing, de-shaping garment
using diffused model. 16] P --> R[Variable skinning weights
avoid cloth-body collisions. 17] P --> S[Optimizing variational autoencoder
for collision-free garments. 18] N --> T[Runtime: canonical regressor, full-space
lift, blendshapes, skinning. 19] A --> U[Revisiting skin simulation
with physics. 20] U --> V[Retaining pose accuracy,
dynamics with data-driven model. 21] U --> W[Components: parameterized skeleton,
physics skin, subspace handles. 22] W --> X[Creating internal skeleton,
variable-thickness skin layer. 23] W --> Y[Modifying gradient, insensitive
to pose model. 24] W --> Z[Handle-based subspace respecting
skeleton, skin accuracy. 25] Z --> AA[Comparing handle-based to
PCA subspace. 26] U --> AB[Applications: hand simulation,
tracking. 27] U --> AC[Embedding contact information
in physics-based subspace. 28] AC --> AD[Aggregate model: linear handles
plus non-linear correction. 29] AC --> AE[Separate learning of internal,
external corrections. 30] class A,B biomechanics class C,D,E,AA subspaces class F,G,H,I,J,K,L,M deformation class N,O,P,Q,R,S,T clothing class U,V,W,X,Y,Z,AB,AC,AD,AE skin class C,D,L,M,S,V,AD,AE learning

Resume:

1.- Simulating biomechanics and interactions with objects, balancing accuracy, generality, and efficiency.

2.- Exploring if machine learning can improve expressive subspace representations without performance penalty.

3.- Connecting machine learning methods with physics-based simulation subspaces.

4.- Finding compact non-linear subspaces (Fulton) and evolving linear subspaces due to contact and dynamics (Holden).

5.- Designing expressive deformation subspaces - geometric (fast) and physics-based (general).

6.- Including contact interactions in subspace formulation.

7.- Deriving skin deformation model accounting for body pose, shape, and dynamics.

8.- Challenges in learning from limited shape samples due to inter-subject pose variability.

9.- Separating global and local pose and shape deformations in a network architecture.

10.- Achieving plausible dynamics for arbitrary body shapes, demonstrating subspace generalization.

11.- Learning geometric deformations leveraging subspace representation.

12.- Accounting for contact when learning deformations.

13.- Machine learning for efficient virtual try-on of tight clothing.

14.- Challenges in loose garments - collisions and post-processing.

15.- Canonical space of garment to work with constant body.

16.- Un-posing and de-shaping garment deformations using diffused human model.

17.- Variable skinning weights from diffuse model to avoid cloth-body collisions.

18.- Optimizing variational autoencoder for collision-free generative subspace of garments.

19.- Runtime pipeline: canonical subspace regressor, full-space lift, pose/shape blendshapes, skinning.

20.- Revisiting skin simulation with physics to handle external interactions.

21.- Retaining pose accuracy and dynamic deformation by extending data-driven model.

22.- Components: parameterized skeleton, physics-based skin, subspace handles.

23.- Creating internal skeleton and variable-thickness skin layer from body surface.

24.- Modifying deformation gradient to make physics model insensitive to pose model.

25.- Handle-based subspace respecting skeletal structure and skin surface accuracy.

26.- Comparing handle-based to PCA subspace - local support, better contact generalization.

27.- Applications in hand simulation and tracking.

28.- Embedding contact information in physics-based simulation subspace.

29.- Aggregate non-linear model: linear handle subspace plus learned non-linear correction.

30.- Separate learning of internal and external corrections to simplify and improve accuracy.

Knowledge Vault built byDavid Vivancos 2024