Skip to main content


Showing posts from 2019

Brain Microarchitecture : Feedback from Higher-order areas to Lower-order areas

Some questions that arise in Machine Learning involve the prospect of using feedback from Higher-order areas (downstream) to Lower-order areas (upstream), and using Global Knowledge for Local Processing.  A desire to gain insight into those issues from Neuroscience ("how does the brain do it?") led me to some fascinating investigations into the Microcircuits of the Cerebral Cortex.  This blog entry is a broad review of the field, in the context of the original motivating questions from Machine Learning.   Starting out with a quote from the “bible of Neuroscience”: From Principles of Neural Science, 5th edn  (Online book location 1435.3 / 5867).  Emphasis and note added by me: Sensory pathways are not exclusively serial; in each functional pathway higher-order areas project back to the lower-order areas from which they receive input. In this way neurons in higher-order areas, sensitive to the global pattern of sensory input, can modulate the activity of neurons in lowe

Associative Memory and Attention Function in Machine Learning - Key, Value & Query

The fascinating but terse 2017 paper on Machine Translation “ Attention Is All You Need ” , by Vaswani et al, has generated a lot of interest – and plenty of head-scratching to digest it!  As far as I can tell, the Attention Function , with its Key , Value and Query inputs, is one of the obstacles in wrapping one's head around the Deep-Learning " Transformer " architecture presented in that paper. In brief: whereas an RNN (Recursive Neural Network) processes inputs sequentially, and uses each one to update an internal state, the Transformer architecture takes in all inputs at once, and makes extensive use of “Key/Value/Query memory spaces” (called “Multi-Head Attention” in the paper.)  Advantages include faster speed, and overcoming the problem of “remembering early inputs” that affects RNN’s. That Attention Function , and related concepts, is what I will address in this blog entry.   I will NOT discuss anything else about the Transformer architecture... but at the

PET/CT Combined Scanners - a 2018 Breakthrough of the Year... and a Personal Story

Image source Recently, a co-worker in her 20's was diagnosed with a brain tumor!  At times like these, the importance of medical imaging jumps to the fore! Most people have heard of CT ("CAT") scanners – at least enough to know that they don't actually involve cats – but less well-known are PET scanners (which likewise don't involve pets!), and the synergistic combination of the two. A Marriage Made in Heaven What do those scanners do?  And why are they being combined in single devices? Voted 2018 Breakthrough of the Year by a science magazine , the improved PET/CT combined scanner has been a game changer. The EXPLORER PET/CT scanner – the world’s first medical imaging system that can capture a 3D image of the entire human body simultaneously – has produced its first human images. Developed by UC Davis scientists and a multi-institutional consortium, EXPLORER can scan up to 40 times faster, or use up to 40 times less radiation dose, than

Interactomics + Super (or Quantum) Computers + Machine Learning : the Future of Medicine?

[Updated Mar. 2021] Interactomics today bears a certain resemblance to genomics in the  1990s...  Big gaps in knowledge, but an explosively-growing field of great promise. If you're unfamiliar with the terms, genomics is about deciphering the gene sequence of an organism, while interactomics is about describing all the relevant bio-molecules and their web of interactions. A Detective Story Think of a good police-detective story; typically there is a multitude of characters, and an impossible-to-remember number of relationships: A hates B, who loves C, who had a crush on D, who always steers clear of E, who was best friends with A until D arrived... Yes, just like those detective stories, things get very complex with our biological story!  Examples of webs of interactions, familiar to many who took intro biology, are the Krebs cycle for metabolism or the Calvin cycle to fix carbon into sugars in plant photosynthesis. Now, imagine vastly expanding those cycles of rea

Multimedia Knowledge Representation and Management : "Brain Annex"

 (Updated Feb. 2022) Wouldn't it be fantastic to have a "butler" to help us as we constantly face drowning in information? That need was crushingly pressing for me , as a polymath with a thirst for knowledge in several fields, not to mention numerous very technical jobs over the years, several complex research projects, old notes from college and grad school, an endless stream of online courses I take , a tech startup I founded and used to run, the many conferences I attend, life in general, and even hobbies that tend to generate abundant information (such as flying airplanes and studying multiple foreign languages!)   I was immensely eager for some sort of powerful assistance, something so helpful that I could poetically describe as an " annex " to my brain.. In this blog entry, I'll describe how deep frustration with existing software tools led to the start of the open-source project, a web-based knowledge representation and manageme

Online Course Review: "An Introduction to Formal Logic" (prof. Gimbel, from "The Great Courses")

Informative, fascinating class and a VERY entertaining professor, with a delightful understated humor and clear explanations, motivations and examples. I was expecting a class entirely about Mathematical Logic... so I was surprised (a good surprise, it turned out) when I discovered that this class roams much more broadly into the Philosophy of Logic.  I was satisfied that enough chapters were about Mathematical Logic - and actually I found the broader context interesting.  For example, I had long heard about Aristotle's syllogisms , but had never looked into them. While there are no Math prerequisites for this class, you'll probably find your head spin by the middle of the course, unless you have a "mathematical predisposition."  (I have a Master's in Math, so for me it was a fun and light "feasting on proofs"...  but a lot of people might feel differently!)  Nonetheless, this great course is quite worthwhile to at least START...

Anti-Aging Research: Science, not Hype

Last updated November 2021 Q: "How is aging a disease?" A: "It's a dynamic system that veers away from its homeostasis (normal equilibrium point): hence a form of slow-progressing illness. Labeling it as 'natural' is a surrender to our traditional state of ignorance and powerlessness, which fortunately is beginning to be changed!" The above is my standard answer to an oft-asked question. The science of aging is by all evidence very misunderstood by the general public.  Hype, misinformation and unquestioned assumptions often prevail, unfortunately. Aging as a systemic breakdown of the body, rather than a series of isolated events and conditions. This 2013 diagram from NIH is a good way to jump-start contemplating the big picture: The diagram originates from the Cell journal: The Hallmarks of Aging   Telomere shortening is perhaps the one most talked about - but just one of several processes.  As stated in the above paper: Each