Knowledge Vault 2/14 - ICLR 2014-2023
Percy Liang ICLR 2015 - Keynote - Learning Latent Programs for Question Answering
<Resume Image >

Concept Graph & Resume using Claude 3 Opus | Chat GPT4 | Gemini Adv | Llama 3:

graph LR classDef task fill:#f9d4d4, font-weight:bold, font-size:14px; classDef solution fill:#d4f9d4, font-weight:bold, font-size:14px; classDef learning fill:#d4d4f9, font-weight:bold, font-size:14px; classDef data fill:#f9f9d4, font-weight:bold, font-size:14px; classDef future fill:#f9d4f9, font-weight:bold, font-size:14px; A[Percy Liang
ICLR 2015] --> B[New challenge: reasoning
over web tables 1] A --> C[Solution: inducing programs
for reasoning 2] C --> D[Programs: compact,
powerful representation 3] A --> E[QA overview: statistical
to semantic parsing 4] B --> F[Unseen tables,
generalizable models 5] B --> G[Operations beyond retrieval:
count, compare, arithmetic 6] C --> H[Tables converted to graph,
queried by programs 7] H --> I[Logical primitives composed:
entities, sets, argmax 8] C --> J[Maps questions to
logical forms, answers 9] J --> K[Needle-in-a-haystack search,
only observes answer 10] J --> L[Features over question,
logical form tokens 11] L --> M[Extensive pruning,
marginalization required 12] A --> N[Results: 37% accuracy,
beats baselines 13] N --> O[Challenges: comparisons, temporal,
external knowledge, schema 14] C --> P[Utterances mapped to programs,
executed for answer 15] P --> Q[Applied to map NL
to robotic actions 16] C --> R[Deep learning: high-level abstractions
via nonlinear transforms 17] A --> S[Programs, vectors complementary strengths:
fuzziness vs logic 18] S --> T[Sentences map state to
boolean, like functions 19] A --> U[Language understanding vs
world knowledge factorization 20] A --> V[Data bottleneck for
semantic parsing 21] V --> W[Invert collection: generate from
KB, paraphrase 22] W --> X[Rapidly build overnight
given target database 23] B --> Y[Stresses multi-step reasoning,
needle-in-a-haystack search 24] A --> Z[Combine discrete programs,
continuous representations 25] Z --> AA[Memory/attention architectures for
complex reasoning 26] A --> AB[Programs: build complexity
from simple primitives 27] AB --> AC[Understanding distinct from knowledge,
controller combines 28] B --> AD[Scaling challenges: representation,
learning, inference 29] C --> AE[Implicit iteration via sets,
robot uses planner 30] class A,B,F,G,Y,AD task; class C,D,H,I,J,K,L,M,O,P,Q,R,Z,AA,AB,AC,AE solution; class E,N,S,T,U learning; class V,W,X data; class Z,AA future;


1.-Talk introduces a new challenge task for representation learning - answering questions requiring reasoning over web tables.

2.-Current solution is based on inducing hidden programs to represent the reasoning steps needed to answer the question.

3.-Programs can be a compact and powerful representation for capturing the meaning and reasoning behind questions.

4.-Gives overview of question answering, from early statistical/retrieval methods to more recent semantic parsing approaches.

5.-New task requires answering questions over unseen Wikipedia tables at test time by learning generalizable models.

6.-Questions require operations like lookup, counting, superlatives, comparisons, arithmetic - more than just simple retrieval.

7.-Tables are converted to a graph format and logical forms/programs are used to query the graph to find the answer.

8.-Logical language includes primitives for entities, sets, count, argmax/min, intersection, etc. that can be composed.

9.-Learning maps questions to logical forms to answers, but only observes the final answer, making it a needle-in-a-haystack search problem.

10.-Features are defined over question and logical form tokens, and a linear model is trained to maximize getting the right answer.

11.-Extensive pruning and marginalization over logical forms is required. Takes 10 hours to train on 23K examples.

12.-Results show 37% accuracy, outperforming baselines. Getting right answer for wrong reasons is a challenge.

13.-Failures include language phenomena like comparisons, temporal relations, external knowledge, and mapping to table schema.

14.-Paradigm maps utterances to programs that are executed to produce the answer. Learning is done from input-output only.

15.-Framework also applied to mapping high-level natural language instructions to low-level robotic actions via post-conditions.

16.-Claims this fits the definition of "deep learning" by learning high-level abstractions via complex nonlinear transformations.

17.-Programs and vectors/matrices have complementary strengths and weaknesses in representing things like fuzziness vs crisp logical operations.

18.-Sentences map a world state to a boolean and behave like functions. Representations should be able to capture this.

19.-There is a factorization between understanding language and knowing facts about the world. Programs make this explicit.

20.-Data has been a bottleneck for semantic parsing. New datasets were collected but are still small compared to vision.

21.-Data collection can be inverted - start from the KB and generate canonical examples that are then paraphrased by humans.

22.-This allows rapidly building semantic parsers for new domains "overnight" if you have the target database.

23.-Task stresses reasoning with multiple computational steps during prediction and needle-in-a-haystack search during training.

24.-Potential to combine discrete programs with continuous representations for better generalization while maintaining compositionality.

25.-Recent memory/attention architectures may provide new ways to approach these complex reasoning tasks.

26.-Programs provide a way to build up complexity compositionally from simple primitives - a powerful representation.

27.-Understanding is distinct from knowledge/memory and a controller learns to combine them to perform sequential reasoning.

28.-Scaling to large KBs with millions of entities poses challenges in representation, learning and efficient inference.

29.-Programs here use implicit iteration via set-based operations rather than explicit loops. Robot application uses a planner.

30.-Framework provides a way to probe the limits and capabilities of representations for language understanding and reasoning.

Knowledge Vault built byDavid Vivancos 2024