I’ve become interested in how we might be able to bridge the gap between deep learning and the learning that humans are capable of.
Deep learning continues to amaze me, but to train a classifier you typically need hundreds of examples — even if you use a pretrained network.
I’ve been exploring how we could use the same architectures, but get the models to perform well with almost no training data.
I’ve got a paper and poster at NIPS in the workshop on Bayesian Deep Learning on an idea I explored with Zoubin Ghahramani. It’s remarkably simple and seems to get state-of-the-art performance. You take, say, a VGG16 trained on all 1000 ImageNet classes and freeze all but the last layer. You model the weight vectors to each of the 1000 softmax outputs as your favourite distribution (Gaussian) and use that as a prior on the distribution of any new weight vectors that need to be trained for any new classes. With each training example, you perform a Bayesian update on this weight distribution. This will be data efficient if you have only a few examples, and if you have a lot of data this weight distribution will converge on the point estimate you’d get with stochastic gradient ascent.
See my poster below and you can read the extended extract on arXiv.
I’ve written a report on the Rustat conference on bitcoin and blockchain technologies I attended in September. It features keynores and
discussions from Balaji Srinivasan, David Yermack, the Bank of England among others.
Figure drawn for the document, depicting how viable ‘killer apps’ for cryptocurrencies depend on the market penetration.
How different will the world be in 30 years? If we continue on our trajectory from the past 30, we may be only decades away from creating artificial intelligence vastly more powerful than humankind.
Creating AI would be the most important event of human history. Everything that civilisation has created, good and bad, has been the result of our intelligence. As technology acquires a mind of its own, and continues to expand, many predict we’ll enter a world where biological intelligence is no longer the dominant force in our universe. It sounds like science fiction, but should we take it seriously?
This prediction is based on extrapolating the exponential growth we have seen in information technologies. Moore’s law is one example, which states that the number of transistors on integrated circuits doubles approximately every two years. Over different time scales, the same is true of processing power, network bandwidth, storage capacity, brain scanning resolution and more. Importantly, the trend seems to extend beyond any one particular technology — the curve for calculations per second extends back to vacuum tubes and electro-mechanical devices.
The trend of exponential growth extends Moore’s law to earlier technologies and is expected to continue in to the future – Courtesy of Kurzweil Technologies, Inc. Creative Commons 1.0
Twitter’s best feature is something they try to make as obscure as possible: lists.
You can group together certain users and see a timeline of just their tweets and no ads.
These lists can even be private, so you don’t need to signal with who you follow. Problem is, it’s hidden behind at
least five clicks and it’s painful to use. So instead I made a web app to make managing them
Drag-and-Drop Simple™.
I use it to separate out my actual friends from all the prolific tech bloggers and news sites. I can look at my friends list and not miss anything they tweet (if I wanted).
I made it with Meteor – a JavaScript web framework that you should be excited to use. I’ve made If you’re interested in how, you can look through the source code on my github.
A beam supported on the left hand edge, with downward force applied to the bottom right. The optimisation algorithm distributes the material in the way to minimise the deflection.
You are watching an optimisation algorithm come up with the stiffest design completely automatically.
The outcome is the stiffest shape possible for a given amount of material. Amazingly, it’s a nuanced truss that isn’t far removed from the look of most motorway bridges. That’s pretty reassuring, actually.
The raspberry pi is fully functional credit card-sized computer that is cheap enough ($25) that it can be used just for a single purpose. With this hack the computer imitates an airplay speaker, making it possible to send songs over to an old stereo wirelessly from your phone.
The Raspberry Pi generated massive hype in nerdy circles this summer when it came out and we’re beginning now to see some amazing hacks from this tiny computer now.
I’ve had mine for a few months now but I hadn’t got around to using it yet. So I’ve now decided to try to make something that I’ve wanted for a while: a product to bring my good but dated speaker system into the 21st century by enabling wireless streaming of music to it.
A possible way to do this would be to buy an Airport Express or an Apple TV and connect the audio out to the stereo. But then I would feel like overpaying for features like video streaming or wireless routing that wouldn’t be used. Besides, those products cost around £80. Airplay enabled speakers cost £200 at the low end. This raspberry pi creation should easily come in under £30.
A chrome extension to replace low resolution twitter avatars with a sharper retina-ready equivalent.
Just to scratch my own minuscule itch that Twitter still looks so crap on retina displays I’ve created my very first chrome extension.
It’s super simple right now – it just replaces the 48px avatars with their 73px equivalent. There are probably a few other higher resolution gains up for grabs but that’s it for now. It looks a lot better.
