Julian's Polymath Explorations

Yes, we all wish for Quantum Computers... but in the meantime we need something here and now!  Could Photonic Computers fit that role? Just about everyone has heard of fiber optics – using light for data transmission – but did you know that light can also be used for computing? There's a new commercial product expected for early next year (2022) . I contacted the CEO, Nicholas Harris, of a 4-y.o. startup, Lightmatter , interviewed in April 2021 here . Photonic computers, at least in their first commercial appearance, are essentially accelerator cards for Linear Algebra - and so of special interest for Machine Learning and some types of simulations.    Their claims are remarkable: 10X faster than some of the best GPUs using 90% less energy can be used with existing software stacks, such as TensorFlow commercially available early next year (2022) a lot of future growth, as additional wavelengths of light get used in parallel My own inte...

The fascinating but terse 2017 paper on Machine Translation “ Attention Is All You Need ” , by Vaswani et al, has generated a lot of interest – and plenty of head-scratching to digest it!  As far as I can tell, the Attention Function , with its Key , Value and Query inputs, is one of the obstacles in wrapping one's head around the Deep-Learning " Transformer " architecture presented in that paper. In brief: whereas an RNN (Recursive Neural Network) processes inputs sequentially, and uses each one to update an internal state, the Transformer architecture takes in all inputs at once, and makes extensive use of “Key/Value/Query memory spaces” (called “Multi-Head Attention” in the paper.)  Advantages include faster speed, and overcoming the problem of “remembering early inputs” that affects RNN’s. That Attention Function , and related concepts, is what I will address in this blog entry.   I will NOT discuss anything else about the Transformer architecture... but at the ...

[Updated Mar. 2022] Interactomics today bears a certain resemblance to genomics in the  1990s...  Big gaps in knowledge, but an explosively-growing field of great promise. If you're unfamiliar with the terms, genomics is about deciphering the gene sequence of an organism, while interactomics is about describing all the relevant bio-molecules and their web of interactions. A Detective Story Think of a good police-detective story; typically there is a multitude of characters, and an impossible-to-remember number of relationships: A hates B, who loves C, who had a crush on D, who always steers clear of E, who was best friends with A until D arrived... Yes, just like those detective stories, things get very complex with our biological story!  Examples of webs of interactions, familiar to many who took intro biology, are the Krebs cycle for metabolism or the Calvin cycle to fix carbon into sugars in plant photosynthesis. Now, imagine vastly expanding those cyc...