Knowledge Vault 5 /73 - CVPR 2022
Understanding Visual Appearance from Micron to Global Scale
Kavita Bala
< Resume Image >

Concept Graph & Resume using Claude 3 Opus | Chat GPT4o | Llama 3:

graph LR classDef physbased fill:#f9d4d4, font-weight:bold, font-size:14px classDef discovery fill:#d4f9d4, font-weight:bold, font-size:14px classDef micronres fill:#d4d4f9, font-weight:bold, font-size:14px classDef worldscale fill:#f9f9d4, font-weight:bold, font-size:14px classDef satellites fill:#f9d4f9, font-weight:bold, font-size:14px classDef cornellbox fill:#d4f9f9, font-weight:bold, font-size:14px classDef materials fill:#f9d4d4, font-weight:bold, font-size:14px classDef fabric fill:#d4f9d4, font-weight:bold, font-size:14px classDef challenges fill:#f9f9d4, font-weight:bold, font-size:14px classDef invgraphics fill:#f9d4f9, font-weight:bold, font-size:14px classDef scattering fill:#d4f9f9, font-weight:bold, font-size:14px classDef llms fill:#f9d4d4, font-weight:bold, font-size:14px classDef clothing fill:#d4f9d4, font-weight:bold, font-size:14px classDef responsibility fill:#d4d4f9, font-weight:bold, font-size:14px classDef futuredirs fill:#f9f9d4, font-weight:bold, font-size:14px A[Understanding Visual Appearance
from Micron to
Global Scale] --> B[Physics models capture reality,
enable mixed reality transitions 1] A --> C[Visual discovery: trends, events,
changes at planet scale 2] A --> D[Micron images show
critical hidden details 3] A --> E[World-scale images record
human lives unprecedentedly 4] A --> F[Satellites capture global
ecological changes 5] A --> G[Cornell box: visual
Turing test, physics graphics 6] G --> H[Real materials complex vs
simple Cornell box 7] A --> I[Fabric look from
micron yarn/weave structure 8] I --> J[CT scans provide
3D material structure 9] I --> K[Photo mean/std dev
approximate albedo, gloss 10] I --> L[Velvet fuzz, silk
shine from weave 11] L --> M[Best models: CT
for structure, photos for optics 12] I --> N[Model control: change
yarns/weaves to alter look 13] I --> O[Goal: end-to-end virtual
design matching real 14] I --> P[Challenges remain making
cloth models intuitive 15] A --> Q[Inverse graphics bridges
real/virtual with differentiable rendering 16] A --> R[Scattering key for
realism, needs volumetric models 17] A --> S[LLMs plausible but
physics enables control 18] A --> T[Photos reveal local,
global clothing styles 19] T --> U[Clothing influenced by
weather, culture, events 20] T --> V[Event spikes remain
after removing weather 21] T --> W[Visual tools aid
anthropology, find phenomena 22] A --> X[100 TB/day from
1500 satellites, huge opportunity 23] X --> Y[Satellites monitor environment,
ecology, crops, fires, drought 24] X --> Z[More CV needed
for unsupervised event detection 25] A --> AA[Experts help find
meaningful problems, impacts 26] A --> AB[With great power
comes great responsibility 27] A --> AC[Wave/quantum optics for
small-scale appearance 28] A --> AD[Beyond visible light
for crop monitoring 29] A --> AE[Steady research progress
in realistic appearance 30] class B physbased class C discovery class D micronres class E worldscale class F,X,Y,Z,AD satellites class G,H cornellbox class I,J,K,L,M,N,O,P fabric class Q,R,S,AC invgraphics class T,U,V,W clothing class AA,AB responsibility class AE futuredirs


1.- Physics-based models capture reality and enable seamless transitions between virtual and real worlds for mixed reality applications.

2.- Visual discovery at planet scale uses visual recognition to discover trends, cultural events, crop changes, and ecological events around the globe.

3.- Micron-resolution images reveal details critical to object and material appearance that are not visible to the naked eye.

4.- World-scale images allow understanding culture and society by recording human lives at an unprecedented rate.

5.- Satellite images at planet scale capture ecological changes like algal blooms and can help understand and aid the changing planet.

6.- The Cornell box in 1988 introduced a visual Turing test and the idea of physics-based graphics simulating light and appearance.

7.- Real-world materials like cloth, skin and food are complex and challenging to make look realistic compared to simple Cornell box.

8.- Fabric appearance comes from yarn and weave structure at the micron level, not just surface properties.

9.- CT scans provide 3D volumes of material structure at micron resolution to build realistic appearance models.

10.- Matching a photo's mean and standard deviation approximates material albedo and gloss, as structure does most of the work.

11.- Cloth appearance depends on structure - velvet's fuzziness from protruding yarns/fibers, silk's shine from interleaved warp and weft.

12.- Best velvet and silk appearance models to date use CT scans for structure and photos for simple optical parameters.

13.- Controllability of models is important - changing yarn parameters and weave patterns alters cloth appearance for design prototyping.

14.- Goal is end-to-end virtual prototyping pipeline matching real production pipeline to enable efficient, predictable design without physical trial-and-error.

15.- Despite progress, challenges remain in making realistic cloth models more intuitive for designers/artists to use.

16.- Inverse graphics bridges real and virtual worlds by recovering shape, materials and lighting from images using differentiable rendering.

17.- Scattering is critical for realism and requires going beyond simple surface models to volumetric models and participating media.

18.- Large language models produce plausible images but graphics models with physics are complementary and enable intuitive artistic control.

19.- Analyzing millions of worldwide photos reveals distinctive local styles and ubiquitous international styles of clothing.

20.- Clothing is influenced by weather, culture, occasion, fashion - spatio-temporal analysis shows seasonal and event-based style changes.

21.- Removing weather patterns reveals spikes corresponding to cultural events like holidays, protests, sporting events.

22.- Visual discovery tools could aid anthropologists in understanding culture and finding previously unknown global and local phenomena in photos.

23.- Satellites collect 100 TB/day from 1500 satellites compared to just 30 satellites 10-15 years ago - huge opportunities to leverage.

24.- Satellite data can monitor crop health, forest fires, drought, construction to understand environmental and ecological change.

25.- More computer vision research is needed on planet-scale unsupervised event detection leveraging the scale and temporal resolution of satellite data.

26.- Working with domain experts is important to learn what problems are meaningful to solve and to understand societal impact.

27.- With great power comes great responsibility - consciously consider societal impact as computer vision grows in influence.

28.- Going beyond classical optics to wave/quantum optics may be needed for material appearance at small scales.

29.- Expanding beyond visible light to other wavelengths can provide information for applications like crop health monitoring.

30.- Progress in realistic appearance modeling has been steady in the research community with models improving in quality and performance over time.

Knowledge Vault built byDavid Vivancos 2024