Moderators: Adriana Romero-Soriano · Yale Song ICLR 2021 - Outstanding Paper Session 1

**Concept Graph & Resume using Claude 3 Opus | Chat GPT4 | Gemini Adv | Llama 3:**

graph LR
classDef alex fill:#f9d4d4, font-weight:bold, font-size:14px;
classDef eigengame fill:#d4f9d4, font-weight:bold, font-size:14px;
classDef sde fill:#d4d4f9, font-weight:bold, font-size:14px;
classDef meshgraphnets fill:#f9f9d4, font-weight:bold, font-size:14px;
A[ICLR 2021

Outstanding Paper Session 1] --> B[Alex: neurosynthesis of binaural

speech for VR. 1] B --> C[Captured data around mannequin

to train model. 2] C --> D[Model warps mono signal

to synthesize binaural. 3] D --> E[Key components: warping, WaveNet,

modified L2 loss. 4] B --> F[Outperforms baselines, enables realistic

VR audio. 5] A --> G[Ian: Eigengame reformulates PCA

as multi-player game. 6] G --> H[Top-k PCA is Nash

equilibrium of game. 7] G --> I[Distributed updates via matrix-vector

products, scalable. 8] I --> J[Computes top components quickly

on huge dataset. 9] G --> K[Novel applications, outperforms baselines. 10] A --> L[Yang: generative model reverses

SDE perturbing data. 11] L --> M[Neural net estimates score

function at noise levels. 12] L --> N[Samples generated by solving

reverse-time SDE. 13] L --> O[Exact likelihoods computed by

transforming SDE to ODE. 14] L --> P[State-of-the-art sample quality, enables

conditional generation. 15] A --> Q[Tobias: MeshGraphNets for mesh-based

simulations using GNNs. 16] Q --> R[Encodes physical system as

graph of mesh elements. 17] R --> S[GNN predicts mesh evolution,

generalizes to complex scenarios. 18] Q --> T[Models cloth, fluids, elasticity

by changing mesh quantities. 19] Q --> U[Adaptive remeshing allocates resolution,

generalizes to larger meshes. 20] class A,B,C,D,E,F alex; class G,H,I,J,K eigengame; class L,M,N,O,P sde; class Q,R,S,T,U meshgraphnets;

Outstanding Paper Session 1] --> B[Alex: neurosynthesis of binaural

speech for VR. 1] B --> C[Captured data around mannequin

to train model. 2] C --> D[Model warps mono signal

to synthesize binaural. 3] D --> E[Key components: warping, WaveNet,

modified L2 loss. 4] B --> F[Outperforms baselines, enables realistic

VR audio. 5] A --> G[Ian: Eigengame reformulates PCA

as multi-player game. 6] G --> H[Top-k PCA is Nash

equilibrium of game. 7] G --> I[Distributed updates via matrix-vector

products, scalable. 8] I --> J[Computes top components quickly

on huge dataset. 9] G --> K[Novel applications, outperforms baselines. 10] A --> L[Yang: generative model reverses

SDE perturbing data. 11] L --> M[Neural net estimates score

function at noise levels. 12] L --> N[Samples generated by solving

reverse-time SDE. 13] L --> O[Exact likelihoods computed by

transforming SDE to ODE. 14] L --> P[State-of-the-art sample quality, enables

conditional generation. 15] A --> Q[Tobias: MeshGraphNets for mesh-based

simulations using GNNs. 16] Q --> R[Encodes physical system as

graph of mesh elements. 17] R --> S[GNN predicts mesh evolution,

generalizes to complex scenarios. 18] Q --> T[Models cloth, fluids, elasticity

by changing mesh quantities. 19] Q --> U[Adaptive remeshing allocates resolution,

generalizes to larger meshes. 20] class A,B,C,D,E,F alex; class G,H,I,J,K eigengame; class L,M,N,O,P sde; class Q,R,S,T,U meshgraphnets;

**Resume: **

**1.-**Alex from Facebook Reality Labs presents research on neurosynthesis of binaural speech from mono audio for realistic sound in virtual reality.

**2.-**They captured data of speech around a mannequin to train an end-to-end model with a neural time warping component.

**3.-**The model warps the mono signal over time based on transmitter/receiver positions to synthesize binaural audio matching the ground truth.

**4.-**Key components are neural time warping, a WaveNet-style network, and a modified L2 loss balancing optimization of amplitude and phase.

**5.-**The approach outperforms digital signal processing baselines and enables realistic, immersive audio rendering for VR applications.

**6.-**Ian Gemp presents Eigengame, reformulating PCA as the solution to a multi-player game to make it more naturally distributed and decentralized.

**7.-**They prove the top-k PCA solution is a Nash equilibrium of the game they construct with an interpretable utility function for each player.

**8.-**Eigengame's distributed updates rely only on matrix-vector products, allowing it to scale to massive datasets on TPU hardware.

**9.-**On a huge dataset from a pre-trained ResNet-200, Eigengame computes the top 32 principal components in 9 hours using 32 TPUs.

**10.-**Eigengame enables novel applications like interpretable analysis of neural networks and outperforms baselines like frequent directions.

**11.-**Yang Song presents a generative model based on learning to reverse a stochastic differential equation (SDE) that perturbs data into noise.

**12.-**A neural network is trained to estimate the score function (gradient of the log probability) of the perturbed data distribution at each noise level.

**13.-**Samples are generated by drawing random noise and numerically solving the learned reverse-time SDE using the score network.

**14.-**Exact likelihoods can be computed for the model by transforming the reverse-time SDE into an ODE and using the instantaneous change of variables formula.

**15.-**The model achieves state-of-the-art sample quality and likelihood scores on image datasets like CIFAR-10 and enables various conditional generation tasks.

**16.-**Tobias Pfaff presents MeshGraphNets, a method for learning mesh-based simulations using graph neural networks.

**17.-**Mesh-based finite element methods are crucial for efficient and adaptive physical simulations but rarely used in machine learning.

**18.-**Their model encodes a physical system as a graph with nodes/edges representing mesh elements and their connectivity in physical and world space.

**19.-**A learned graph neural network is applied to the input graph to predict the time evolution of the mesh in a way that generalizes to complex unseen scenarios.

**20.-**The same architecture can model cloth, fluids, and elasticity by simply changing the input and output mesh quantities.

**21.-**Learned adaptive remeshing allows the model to allocate more resolution to important regions and generalize to much larger meshes than seen during training.

Knowledge Vault built byDavid Vivancos 2024