Julian's Polymath Explorations

Some questions that arise in Machine Learning involve the prospect of using feedback from Higher-order areas (downstream) to Lower-order areas (upstream), and using Global Knowledge for Local Processing.  A desire to gain insight into those issues from Neuroscience ("how does the brain do it?") led me to some fascinating investigations into the Microcircuits of the Cerebral Cortex.  This blog entry is a broad review of the field, in the context of the original motivating questions from Machine Learning.   Starting out with a quote from the “bible of Neuroscience”: From Principles of Neural Science, 5th edn  (Online book location 1435.3 / 5867).  Emphasis and note added by me: Sensory pathways are not exclusively serial; in each functional pathway higher-order areas project back to the lower-order areas from which they receive input. In this way neurons in higher-order areas, sensitive to the global pattern of sensory input, can modulate the activity of neurons in lowe

The fascinating but terse 2017 paper on Machine Translation “ Attention Is All You Need ” , by Vaswani et al, has generated a lot of interest – and plenty of head-scratching to digest it!  As far as I can tell, the Attention Function , with its Key , Value and Query inputs, is one of the obstacles in wrapping one's head around the Deep-Learning " Transformer " architecture presented in that paper. In brief: whereas an RNN (Recursive Neural Network) processes inputs sequentially, and uses each one to update an internal state, the Transformer architecture takes in all inputs at once, and makes extensive use of “Key/Value/Query memory spaces” (called “Multi-Head Attention” in the paper.)  Advantages include faster speed, and overcoming the problem of “remembering early inputs” that affects RNN’s. That Attention Function , and related concepts, is what I will address in this blog entry.   I will NOT discuss anything else about the Transformer architecture... but at the

Image source Recently, a co-worker in her 20's was diagnosed with a brain tumor!  At times like these, the importance of medical imaging jumps to the fore! Most people have heard of CT ("CAT") scanners – at least enough to know that they don't actually involve cats – but less well-known are PET scanners (which likewise don't involve pets!), and the synergistic combination of the two. A Marriage Made in Heaven What do those scanners do?  And why are they being combined in single devices? Voted 2018 Breakthrough of the Year by a science magazine , the improved PET/CT combined scanner has been a game changer. The EXPLORER PET/CT scanner – the world’s first medical imaging system that can capture a 3D image of the entire human body simultaneously – has produced its first human images. Developed by UC Davis scientists and a multi-institutional consortium, EXPLORER can scan up to 40 times faster, or use up to 40 times less radiation dose, than

[Updated Mar. 2022] Interactomics today bears a certain resemblance to genomics in the  1990s...  Big gaps in knowledge, but an explosively-growing field of great promise. If you're unfamiliar with the terms, genomics is about deciphering the gene sequence of an organism, while interactomics is about describing all the relevant bio-molecules and their web of interactions. A Detective Story Think of a good police-detective story; typically there is a multitude of characters, and an impossible-to-remember number of relationships: A hates B, who loves C, who had a crush on D, who always steers clear of E, who was best friends with A until D arrived... Yes, just like those detective stories, things get very complex with our biological story!  Examples of webs of interactions, familiar to many who took intro biology, are the Krebs cycle for metabolism or the Calvin cycle to fix carbon into sugars in plant photosynthesis. Now, imagine vastly expanding those cycles of rea

 (Updated Nov. 2022) Wouldn't it be fantastic to have a "butler" to help us as we constantly face drowning in information? That need was crushingly pressing for me , as a polymath with a thirst for knowledge in several fields, not to mention numerous very technical jobs over the years, several complex research projects, old notes from college and grad school, an endless stream of online courses I take , a tech startup I founded and used to run, the many conferences I attend, life in general, and even hobbies that tend to generate abundant information (such as flying airplanes and studying multiple foreign languages!)   I was immensely eager for some sort of powerful assistance, something so helpful that I could poetically describe as an " annex " to my brain.. In this blog entry, I'll describe how deep frustration with existing software tools led to the start of the open-source BrainAnnex.org project, a web-based knowledge representation and manageme

Informative, fascinating class and a VERY entertaining professor, with a delightful understated humor and clear explanations, motivations and examples. I was expecting a class entirely about Mathematical Logic... so I was surprised (a good surprise, it turned out) when I discovered that this class roams much more broadly into the Philosophy of Logic.  I was satisfied that enough chapters were about Mathematical Logic - and actually I found the broader context interesting.  For example, I had long heard about Aristotle's syllogisms , but had never looked into them. While there are no Math prerequisites for this class, you'll probably find your head spin by the middle of the course, unless you have a "mathematical predisposition."  (I have a Master's in Math, so for me it was a fun and light "feasting on proofs"...  but a lot of people might feel differently!)  Nonetheless, this great course is quite worthwhile to at least START...

Last updated May 2023 Q: "How is aging a disease?" A: It's a dynamic system that veers away from its homeostasis (normal equilibrium point): hence a form of slow-progressing illness. Labeling it as 'natural' is a surrender to our traditional state of ignorance and powerlessness, which fortunately is beginning to be changed! Aging is "normal" only from the point of view of the "selfish gene", for whom the body is a disposable carrier. Individuals organisms - for whom self-preservation has a different meaning than for genes - have received scant help from evolution... with rare exceptions such as the T. dohrnii jellyfish (which I discuss here )... but now the time has finally arrived for our rational design to remedy some of the cellular flaws that evolution never bothered to correct!   The above is my standard answer to an oft-asked question. The science of aging is by all evidence very misunderstood by the general public.  Hype,