Chapters: Ch 1 · Ch 2 · Ch 3 · Ch 4 · Ch 5
Hallucination, Alignment, Verification
Vishal Misra: LLMs Cannot Recursively Self-Improve Columbia professor Vishal Misra argues that current LLM architectures have no mechanism for recursive self-improvement because they have no way to verify whether their outputs are correct — they generate probabilistically without a ground-truth oracle. This is a fundamental architectural point distinguishing LLMs from symbolic AI systems that can check their own reasoning against formal constraints. https://x.com/vishalmisra/status/1801982071713276275
What’s Next for AI Agentic Workflows — Andrew Ng (Sequoia Capital AI Ascent) Andrew Ng’s talk on agentic AI workflows — the shift from single-shot LLM calls to iterative agent loops where models plan, act, observe results, and revise. Ng argues that agentic workflows dramatically expand what AI can accomplish on complex multi-step tasks, and previews the architectural patterns (plan-act-observe, multi-agent collaboration, tool use) that have since become standard. https://youtu.be/sal78ACtGTc
NLP
Large Linguistic Models — Analyzing Theoretical Linguistic Abilities of LLMs (arXiv 2305.00948) A paper systematically evaluating LLMs against formal linguistic theories — phonology, morphology, syntax, semantics, pragmatics — asking whether models that achieve high benchmark scores have genuinely acquired the underlying linguistic knowledge or are sophisticated pattern matchers. The analysis is grounded in formal linguistics rather than downstream NLP task performance. https://arxiv.org/abs/2305.00948
Do We Need Language to Think? (NYT) A science piece reporting on neuroscientists arguing that inner speech is primarily a communication tool rather than the substrate of reasoning — that thought, planning, and problem-solving proceed in neural representations that are not inherently linguistic. The finding has implications for understanding whether LLMs that manipulate linguistic tokens are actually “reasoning” in any meaningful sense. https://www.nytimes.com/2024/06/19/science/brain-language-thought.html
Indian Languages and Text Normalization Part 1 (kavyamanohar.com) A blog post by Malayalam linguist Kavya Manohar covering text normalization challenges specific to Indian language scripts — handling zero-width joiners, nukta variants, Unicode normalization forms, and the gap between typographic representations and phonological reality. An essential starting point for anyone building NLP pipelines for Devanagari, Malayalam, or Kannada text. https://kavyamanohar.com/
VictorTaelin Repositories (GitHub) Victor Taelin’s GitHub, home of projects like HVM (Higher-order Virtual Machine) — a parallel functional runtime — and Bend (a functional language that compiles to GPU-native parallel code). Taelin’s work explores whether functional programming’s mathematical structure can be harnessed to make massively parallel computation tractable, with interesting implications for ML training infrastructure. https://github.com/VictorTaelin
Topos Institute
Topos Institute: Understanding UMAP The Topos Institute’s explainer on UMAP (Uniform Manifold Approximation and Projection) — a dimensionality reduction algorithm grounded in topological data analysis and Riemannian geometry. The Topos treatment is notably more mathematically honest than typical UMAP tutorials, explaining the actual assumptions (local metric approximation, fuzzy simplicial sets) rather than hand-waving toward “it preserves structure.” https://topos.site/
Misc AI Notes
“The Hardest Things in Computer Science Is Caching and…” The classic Phil Karlton quote: “There are only two hard things in computer science: cache invalidation and naming things.” Saved here as a reminder that fundamental engineering problems (state synchronization, semantic precision) recur at every level of abstraction — and that the same problems surface in AI system design (context caching, prompt naming, concept labeling).
NotebookLM’s Summary of Notes (Sep 17, 2024) A personal archive of a NotebookLM-generated podcast summary of accumulated notes, organized into: (I) Health and Fitness — Uncommon Mobility in Your 40s; (II) Technology and Science — KAN networks, OSSU, X rearchitecting; (III) Anthropology — Neanderthal extinction; (IV) Biology — “Third State” of existence; (V) Miscellaneous — UCSF Parnassus Campus. A snapshot of NotebookLM’s early capabilities applied to personal knowledge management.