Less code, more neural networks

A big share of the world’s code is about to get replaced by neural networks. Not simply the writing of the code, which is increasingly generated by AI, but the code itself. Computations in software, to transform inputs into outputs, will be covered more and more by neural nets. This transition already started.

Neural networks, especially since transformers, are general programmable computers. You can think of neural network architectures as being like computer program templates, and their weights as the learned parameters that determine their specific behavior. When trained on adequate data and with enough compute, they can be taught to perform many operations and tasks, that are increasingly complex. It now includes being able to talk to a customer or to drive a car.

Machine learning and AI work with probabilities, which is a great tool to navigate and make decisions in uncertain environments. That is their nature: they are probabilistic. They introduce uncertainty into software, which has been historically deterministic. Such systems don’t output 0 or 1, they output 0 to 100% of probability. Evaluating them necessitates the use of statistical methods, in addition to deterministic testing.

Not every piece of software will be a neural network, many operations need to be rule-based and deterministic. There won’t be less code in terms of absolute quantity, but probably more, especially with AI’s ability to write rule-based code itself. But its share, in actual computations and output production, will decrease importantly for the profit of neural nets.

Software is shifting from code to neural networks. Applications are evolving from deterministic tools that people use to do things, to probabilistic agents that do things for people.