Concept Graph & Resume using Claude 3 Opus | Chat GPT4 | Gemini Adv | Llama 3:

Resume:

1.-Talk introduces a new challenge task for representation learning - answering questions requiring reasoning over web tables.

2.-Current solution is based on inducing hidden programs to represent the reasoning steps needed to answer the question.

3.-Programs can be a compact and powerful representation for capturing the meaning and reasoning behind questions.

4.-Gives overview of question answering, from early statistical/retrieval methods to more recent semantic parsing approaches.

5.-New task requires answering questions over unseen Wikipedia tables at test time by learning generalizable models.

6.-Questions require operations like lookup, counting, superlatives, comparisons, arithmetic - more than just simple retrieval.

7.-Tables are converted to a graph format and logical forms/programs are used to query the graph to find the answer.

8.-Logical language includes primitives for entities, sets, count, argmax/min, intersection, etc. that can be composed.

9.-Learning maps questions to logical forms to answers, but only observes the final answer, making it a needle-in-a-haystack search problem.

10.-Features are defined over question and logical form tokens, and a linear model is trained to maximize getting the right answer.

11.-Extensive pruning and marginalization over logical forms is required. Takes 10 hours to train on 23K examples.

12.-Results show 37% accuracy, outperforming baselines. Getting right answer for wrong reasons is a challenge.

13.-Failures include language phenomena like comparisons, temporal relations, external knowledge, and mapping to table schema.

14.-Paradigm maps utterances to programs that are executed to produce the answer. Learning is done from input-output only.

15.-Framework also applied to mapping high-level natural language instructions to low-level robotic actions via post-conditions.

16.-Claims this fits the definition of "deep learning" by learning high-level abstractions via complex nonlinear transformations.

17.-Programs and vectors/matrices have complementary strengths and weaknesses in representing things like fuzziness vs crisp logical operations.

18.-Sentences map a world state to a boolean and behave like functions. Representations should be able to capture this.

19.-There is a factorization between understanding language and knowing facts about the world. Programs make this explicit.

20.-Data has been a bottleneck for semantic parsing. New datasets were collected but are still small compared to vision.

21.-Data collection can be inverted - start from the KB and generate canonical examples that are then paraphrased by humans.

22.-This allows rapidly building semantic parsers for new domains "overnight" if you have the target database.

23.-Task stresses reasoning with multiple computational steps during prediction and needle-in-a-haystack search during training.

24.-Potential to combine discrete programs with continuous representations for better generalization while maintaining compositionality.

25.-Recent memory/attention architectures may provide new ways to approach these complex reasoning tasks.

26.-Programs provide a way to build up complexity compositionally from simple primitives - a powerful representation.

27.-Understanding is distinct from knowledge/memory and a controller learns to combine them to perform sequential reasoning.

28.-Scaling to large KBs with millions of entities poses challenges in representation, learning and efficient inference.

29.-Programs here use implicit iteration via set-based operations rather than explicit loops. Robot application uses a planner.

30.-Framework provides a way to probe the limits and capabilities of representations for language understanding and reasoning.

Knowledge Vault built byDavid Vivancos 2024