In Econ 05 and Econ 06 I covered what is unique about software as opposed to the physical world. I fleetingly mentioned the concept of the ‘creative-commons free market of open source AI’. Why is it creative commons? How is it a free market? What exactly is open source and how does it matter to AI? Let’s explore these questions.
Let’s start with the very idea of open source. Software is written in a programming language. The resulting code - the set of instructions that define how the software should behave (such as with a cash machine (ATM) under what conditions it should trigger the dispensing of cash) - forms part of the ‘intellectual property’ of the people who built or own it.
Within the realm of intellectual property law, computer code has been deemed as something that is to be copyrighted rather than patented, just as other creative work in literary, educational and other spheres. This means, as hinted above, the builders or owners of the software retain the right to copy, distribute and otherwise make use of such code as they possess under the copyright.
However, the question of software code as copyrighted intellectual property took a different form early on. There arose this notion, and one that was spearheaded by stalwarts such as Richard Stallman, that code should be free to share, modify and reuse. This led, for example, to the concept of copyleft, as opposed to copyright of course, which provided for such freedoms, still within the legal framework of copyright law within the common law system.
Richard Stallman pioneered the GNU General Public License with other similar licenses emerging, including the MIT license. This licensing mechanism, while working within the broader framework of copyright law in any jurisdiction, is particularly suited for application under common law - a bottom-up legal system where individuals and entities have the freedom to enter into contracts and covenants, which will then be enforced by the courts as law. This is the predominant legal system in most regions of the United States, and the United Kingdom, where it originated.
So how does it actually work? A key aspect of such open-source licenses is, apart from the said freedoms to share, modify and reuse, that any modified copy should retain the attribution to the originator, ensuring that credit is given for all the work done before it was copied and modified. This is what sustains the ecosystem of open source because such previous work that has been copied from does not go unrecognized.
This is what I’m broadly calling the creative commons. Specifically, ‘Creative Commons’ refers to a similar licensing approach, overseen by an entity so named, to encompass all creative work, and not just code. I’m using the term as a more generic concept, analogous to the town square which is shared by all citizens. And since this is fundamentally about the freedom to engage freely, within the legal framework of such open source licenses, I’m also calling it a free market, again a slightly broader definition than the one formally used in economics and market Capitalism.
There’s a lot more to be said about this, but with this general background let’s see how the open-source phenomenon impacted AI. The open source ecology, as relevant to the AI space, builds upon the existing conventional software open source foundation.
The greatest example of an open-source project is Linux, the work put into which over the years exceeds the GDP of several countries. Linux is what Google copied and modified to become Android, the basis of most (non-Apple) mobile phones out there, perhaps you have one in your pocket. Another great example within the creative-commons context is Wikipedia, where anyone can edit any page.
There is one key difference in the context of open source between conventional software and AI. As we have seen in prior articles, AI is by definition data-heavy: AI as we’ve seen, is not so much about rules (code) laid down, but an algorithm trained by copious amounts of data so it learns the domain it is to operate in. Thus the concept of open source somewhat extends here to the notion of sharing such data that was used in the training and/or the weights of the neural net - in essence, the resulting trained version of the neural net (the shape of the well-formed brain if you will.)
An important milestone in this regard was in the building of ImageNet, a large dataset curated around 2007-2010, containing millions of hand-annotated images of simple objects / categories (cat, balloon…), used to build a visual object-recognition AI system. This dataset was made freely available for other researchers and interested parties to use in their own AI experiments.
Two other notable releases following the 2017 breakthrough of the release of the Transformer Architecture whitepaper (see my article in the Concepts Series outlining how this works). These were BERT (Bidirectional Encoder Representations from Transformers) from Google where the architecture innovation happened, and shortly before that, OpenAI’s first GPT (Generative Pre-Trained Transformer-1) - yes, of ChatGPT fame!
Both of these were the first language models, bigger later versions of some of which have come to be qualified as ‘large’ - the LLMs I’ve so often referred to in these articles, underpinning much of the (Generative) AI phenomenon we’re discussing. Such developments led to a flurry of activity, thanks in large part to the fact that such models were released as open source and early research around them has been available to us within the creative commons.
It so transpired that some of these early labs, large software firms – seeing the growing power and potential of these LLMs, stopped making them open source, hence the models underlying ChatGPT (a UI wrapper over the LLMs), such as GPT-3.5 and later, are, despite being released by a company named ‘OpenAI’, closed source!
However, the genie was out of the bottle. These and other similar developments in Artificial Intelligence and Machine Learning demonstrated what was possible.
And while scaling these up to large models has necessitated huge amounts of data and compute power in building these models, which have thus only been within the reach of a small number of large technology firms such as Google, Microsoft and Meta (Facebook), the sheer vibrancy of the open source movement as made possible within the creative-commons free-market ecosystem has meant that open-source AI is actually not that far behind these proprietary AI developments. I will have a lot more to say about these in future articles.
—
Subscribe to FF Daily for more content from Ash Stuart.
