Neural Networks

Earlier this year (towards the end of my undergrad) I started to record some notes on [machine learning applications and the datasets that power them](/2018/feret-database/). While I did find some interesting artifacts (see the FERET database) and read some good articles (such as [The Body and the Archive](, I never managed to really train and use a neural network in practice.

I did experiment with some pre-built networks, such as DCGAN by and char-rnn, but I didn't really undestand what was happening under the surface. More importantly, I couldn't modify these networks to change their outputs (apart from tweaking some hyperparameters).

The focus of the next few months is going to be to change that. As with earlier projects, I will be taking notes here as I go along.

Ideas for a workshop

Sheena suggested I might run a workshop with Interpolate on the subject of machine learning. This comes after a discussion at Designing a Feminist Alexa. We agreed that there seems to be an awful lot of magical thinking among humanities-folk about what machine learning is. To me, it often feels like these debates are floating in thin air. Arguments seem to be based more on Black Mirror episodes and Ted Talks than the papers where the working mechanisms of neural networks are being developed.

The goal of the workshop would be to dispel some of those myths, and give people some low-level understanding of what they're talking about. I'm proposing to do this by having a group of people train a neural network by hand — using nothing but pen, paper, and maybe a basic calculator. Over the course of a few hours, we'd build up a network using string and index cards pinned to the wall. Once the network is sufficiently trained, we'd use it to generate some kind of outcome.

I'm imagining it like the scene in Apollo 13 where engineers in mission control are working out orbital mechanics using slide rules.

The main question is: What kind of task do we train this network on? It has to be simple enough to be accomplished by a small network with limited computing power, yet complex enough to keep people interested.

November 22, 2018: Some more specific workshop ideas

The learning algorithm is taken straight out of Rosenblatt (1958):

$$w_{i,j}^{\text{next step}} = w_{i,j} + \eta (y_i - \hat{y}_j)x_i$$

Once the model is trained, it can be evaluated:

Monday, December 17

Notes from running the workshop myself:

Things that my go in the reader