Attention and open source is all you need

2024-04-08

A topic of great impact

In 2017, a paper titled Attention is all you need by eight Google researchers marked the first brick of the (maybe still not complete) house on which a big part of present-days AI is built.

The concept introduced by Vaswani and colleagues, the transformer, became in recent years the base for the blossoming AI spring, and we will dive deeper in the architecture and in the applications of transformers in the upcoming weeks, in a series that we will entirely dedicate to that.

For now, we will just briefly take a look at one of the most known and beloved use cases that exploded in the last three years: Large Language Models (LLMs).

_config.yml

Photo by Google DeepMind from Pexels

We won’t see the technical details (we will explain them in next week’s post): we will rather focus on their impact on society and on the importance of keeping LLMs (an AI in general) open-source and accessible to everyone.

The first widely-known LLM was ChatGPT-3, made available by OpenAI to the great public in late november 2022: it leveraged transformers’ power to seemlessly generate human-like language, responding to prompts by its users. From then on, lots of larger and larger models were released, one better than the other, and day after day the race to unleash the generative power of AI started to permeate our everyday life.

GPT-4, Gemini, Mixtral, LLama… and all sort of other tools by big-tech companies (Google, Meta, e.g.) and rising startups were trained on an increasingly bigger quantity of data, performing better and forcing AI detectors to refine their weapons.

But all of this, obviously, came with a price.

The huge training and maintenance costs forced big companies to partially or fully lock the access to their LLMs behind a paid subscription, making it difficult to the large public of amateur users to enjoy the improvements made by generative models these days. The era of fully open-source AI had slowly, but steadily, started its sunset.

In spite of that, open-sourcedness should be a value that all AI companies want to pursue, in order to create a world of shared knowledge, open science and great educational opportunities.

Just think about it: a world where the most powerful AI models are at a few clicks from teachers, students, even in the most remote areas, with scarce learning resources. A world where everyone can work less and boost their productivity thanks to AI optimization, where repeatitive and monotonous tasks are left to AI and we can enjoy more freedom and a better quality work-life balance. A world where AI can help us solve thed biggest challenges of climate change, health and politics.

As utopistic as it may sound, if we stick on keeping AI knowledge and power open source and easily accessible, maybe one day all of this won’t be a mirage, but a solid reality.

Conclusion

Open-source AI holds immense promise for democratizing innovation, promoting inclusiveness, and empowering individuals to shape the future of technology. By embracing the principles of collaboration, sharing, and mutual benefit, we can collectively build a stronger, smarter, and fairer digital landscape.

So, what are you waiting for? Join the ranks of passionate contributors and advocates who believe in the power of open-source AI. Share your expertise, ideas, and enthusiasm with others, and watch as our collective efforts catalyze positive change. Together, we can make a difference and pave the way towards a more equitable tomorrow.

Let’s spread the word and inspire others to embrace the spirit of open-source AI!

Curious on how to get free access to open-source LLMs? Check in the Refences section!

References