Knowledge Vault 2/88 - ICLR 2014-2023
H. Sebastian Seung ICLR 2022 - Invited Talk - Petascale connectomics and beyond
<Resume Image >

Concept Graph & Resume using Claude 3 Opus | Chat GPT4 | Gemini Adv | Llama 3:

graph LR classDef main fill:#f9d4d4, font-weight:bold, font-size:14px; classDef brain fill:#d4f9d4, font-weight:bold, font-size:14px; classDef connectome fill:#d4d4f9, font-weight:bold, font-size:14px; classDef methods fill:#f9f9d4, font-weight:bold, font-size:14px; classDef future fill:#f9d4f9, font-weight:bold, font-size:14px; A[H. Sebastian Seung
ICLR 2022] --> B[Brain observed at
cellular resolution 1] A --> C[Observing connectivity
computationally intensive 2] A --> D[C. elegans
connectome mapped 3] A --> E[Serial section
electron microscopy
technique 4] E --> F[Fruit fly brain
imaged, 100TB data 5] F --> G[Flywire: AI, human
analyze images 6] G --> H[Convolutional nets
used in Flywire 7] G --> I[Experts correct
AI in Flywire 8] G --> J[CT1 neurons
revealed in Flywire 9] G --> K[Synapses detected
in fly brain 10] K --> L[Visual pathway mapped
in connectome 11] F --> M[Two projects map
fly connectome 12] M --> N[Fly connectome
enables discoveries 13] A --> O[Graph analysis
of connectomes 14] O --> P[Spectral clustering
reveals brain regions 15] A --> Q[Nature vs nurture
in connectomes 16] Q --> R[Innate vs learned
connectivity 17] R --> S[Flies have
stereotyped connectome 18] A --> T[Mouse cortex studied
for intelligence 19] T --> U[Mouse visual
cortex imaged 20] U --> V[Neural activity recorded
before imaging 21] U --> W[Mouse data
publicly available 22] U --> X[Electron microscopy
reveals details 23] X --> Y[Cajal limited by
lack of synapses 24] U --> Z[Connection strengths
from synapse size 25] Z --> AA[Multiple synapses
as multi-graph 26] AA --> AB[Dual connections suggest
binary synapses 27] AB --> AC[Binary synapses challenge
for learning 28] A --> AD[Fly connectome semi-automated,
0.1PB 29] A --> AE[Mouse connectome proposed,
1EB, automated 30] class A main; class B,C,D,E,F,T,U,V,W,X,Y brain; class G,H,I,J,K,L,M,N,AD,AE connectome; class O,P,Q,R,S,Z,AA,AB,AC methods;


1.-Neuroscientists have made significant progress in observing the brain at cellular resolution, aiming to see molecules, neural activity, and connectivity.

2.-Observing connectivity is particularly challenging and computationally intensive, even after obtaining primary data through electron microscopy.

3.-The connectome of C. elegans, a tiny worm, was first mapped in 1986 and updated in 2006 through manual analysis.

4.-Serial section electron microscopy involves cutting brain tissue into thin slices, imaging them, aligning the images, and stacking them into a 3D volume.

5.-Davi Bock's team applied this technique to image an entire fruit fly brain, generating 100 terabytes of data from a tiny volume.

6.-Flywire, an online community using AI and human expertise, was created to analyze the fruit fly brain images and extract the connectome.

7.-Flywire uses convolutional nets for image alignment, artifact detection, segmentation, and synapse detection, along with post-processing algorithms for global computation.

8.-Human experts can correct AI mistakes in Flywire, reducing manual labor by orders of magnitude compared to purely manual analysis.

9.-Proofread neurons in Flywire reveal highly stereotyped, identified neurons in the fly brain, such as the CT1 neurons involved in visual motion processing.

10.-Jan Funke's lab detected about two million synapses between roughly 100,000 neurons in the fly brain, incorporating them into Flywire.

11.-With reconstructed neurons and synapses, pathways can be mapped inside the Drosophila connectome, such as a visual pathway from photoreceptors to deep brain regions.

12.-Flywire and FlyEM (Janelia and Google collaboration) are two projects working on mapping the Drosophila connectome using different electron microscopy approaches.

13.-Parts of the fly connectome have already led to discoveries in visual motion detection, olfaction, learning and memory, and navigation.

14.-Connectomes provide opportunities for computer scientists to apply graph analysis methods, such as spectral clustering and stochastic block models.

15.-Spectral clustering of the fly brain revealed sensory and motor clusters, including the ellipsoid body involved in navigation and heading direction representation.

16.-In artificial neural networks, the connection matrix is determined by the starting architecture (nature) and modified by learning algorithms based on examples (nurture).

17.-In real brains, the innate connectome is determined by genetically predetermined cell types, while synaptic plasticity (learning) shapes the connectome through experience.

18.-Flies have highly stereotyped, identified cells, suggesting that their connectome is largely shaped by innate structure with some learning-induced modifications.

19.-The mammalian cortex, the largest brain structure, is important for intelligence and learning and can be studied in mice.

20.-A collaboration between Baylor College of Medicine, Allen Institute, and Princeton studied a piece of mouse visual cortex using electron microscopy.

21.-Before electron microscopy, visually evoked neural activity was recorded in the mouse cortex using calcium imaging while presenting visual stimuli.

22.-The reconstructed mouse cortex data, containing 75,000 neurons and half a billion synapses, is publicly available at the Microns Explorer website.

23.-Electron microscopy of the mouse cortex reveals detailed structures, such as capillaries, mitochondria, and the complex morphology of pyramidal neurons.

24.-Santiago Ramon y Cajal's pioneering work on cortical neurons was limited by the inability to see synapses and complete connectional structure.

25.-Connection strengths can be estimated from the size and number of synapses involved in the connection, similar to estimating muscle strength from size.

26.-Connections between neurons can be mediated by multiple synapses, represented as a multi-graph with multi-edges in the connectome.

27.-Analysis of dual connections in the cortex reveals correlated synapse sizes and a bimodal distribution, suggesting synapses might be binary switches.

28.-Binary synapses have implications for biological learning, as low-precision connections pose challenges for learning in artificial neural networks.

29.-The Drosophila connectome, expected in 2023, involves a tenth of a petabyte of image data and semi-automated image analysis.

30.-The National Institutes of Health is considering a 10-year project to map the entire mouse brain connectome, requiring one exabyte of data and full automation.

Knowledge Vault built byDavid Vivancos 2024