Former OpenAI scientist Andrej Karpathy, in his recent keynote address at the UC Berkeley AI Awards Hackathon, inspired students not to give up on their side projects. Citing the example of GPT-4o, he said that it initially started off as a Reddit chatbot project.
“It’s very easy to dismiss these small snowballs because they’re so fragile in the beginning. But actually, this Reddit chatbot, which looked naive, is a language model trained on Reddit,” he said.
“When the Transformer came out, it was developed into something much better. Then the domain was expanded from just Reddit to many other web pages, and suddenly you get GPT-1, GPT-2, GPT-3, GPT-4, and then GPT-4o,” he added.
Karpathy said that while some projects may not succeed, others may have the potential to thrive. He urged students to maintain momentum in their work, suggesting that even small projects can add up to a really ‘big snowball’, which he finds inspiring to observe.
He himself has been actively exploring several generative AI tools, always engaged in building something new. His latest project involves a tech stack that generates visual stories using Anthropic Claude, Ideogram, Lumalabs, ElevenLabs, and VeedStudio.
In another experiment, Karpathy reproduced the smallest version of GPT-2, which has 124 million parameters, in just over four hours.
Interestingly, GitHub Copilot also started as an internal project and has gone on to become a powerful AI-powered code completion tool used by developers worldwide. Similarly, Midjourney began as a small research lab project but has grown into one of the leading AI image generation tools, competing with established players like DALL-E.
Valued at $2 billion, Hugging Face began as a chatbot app for teenagers. It shared selfies of its computer-generated face, cracked jokes, and gossiped about its crush on Siri. Despite its charm, it hardly made any money. Today, it has evolved into a major hub for open-source AI models and datasets.
The credit for NVIDIA’s success goes to Andrew Ng’s cat-recognition model (his passion project) which was successfully trained using just 12 NVIDIA GPUs, demonstrating the superiority of GPU over CPUs for processing for AI tasks.
This proved to be a significant moment in showcasing the potential of GPUs for deep learning.
Indians Like Jugaad
In India, we have seen several side projects undertaken by students and professionals turn into something big. For instance, Ravi Theja and Ramsri Goutham Golla, who built Navarasa, a Gemma 7B/2B instruction-tuned model supporting 15 Indian languages and English, were featured at the Google I/O 2024 Keynote.
“Take your side projects seriously! You might as well get featured in the Google I/O keynote on the main stage in California ;),” Golla posted on X.
Just a few days after Cognition Labs launched AI software engineer Devin, Mufeed VH (Hamzakutty), founder of Lyminal and Stition.AI, created an open-source passion project called Devika.
This Indian counterpart to Devin can understand human instructions, break them down into tasks, conduct research, and autonomously write code to achieve objectives. Devika has been well-received in the developer community. Today, Mufeed is part of the YC ‘24 batch.
A majority of Indic LLMs, such as Kannada Llama, Telugu Llama, and Tamil Llama, were primarily side projects developed without any intention of commercialisation.
At WWDC, Apple recently introduced a brand new Siri powered by Apple Intelligence. Vyshnav Karun Somasundaram, currently a CS student at Christ University, Bangalore, built a Siri alternative called Anjali.
“Anjali is now proudly Indic 🇮🇳 and can fully hold conversations and summarize documents in Hindi (हिंदी), Tamil (தமிழ்), Kannada (ಕನ್ನಡ), Telugu (తెలుగు), and Malayalam (മലയാളം),” said Somasundaram.
“I’m currently working on Anjali’s multimodal emotion recognition capabilities and her ability to recognize people when talking to them,” he added.
Another example is Bishal Saha, a dropout from Lovely Professional University who created Omniplex, an open-source alternative to Perplexity AI, over a weekend. Speaking to AIM, he said that despite initial job rejections from Perplexity AI, it was his tenacity that led him to creating this alternative, which is now gaining traction.
Dhravya Shah was barely 16 when he started programming and game development. At 18, he built Radish, an open-source alternative to Redis.
Two Indian engineering students, Rudransh Agnihotri and Manasvi Kapoor, launched Mayakriti, an image generation platform that uses advanced GenAI to create lifelike visuals — from photorealistic portraits and personalised creations as well as in a variety of art styles such as cartoons, anime, and abstract art.
The list goes on…
Fun fact: Narayana Murthy established Infosys in 1981 with six other engineers as an “experiment in entrepreneurship” – a company of the professionals, for the professionals, and by the professionals.
The message is clear, don’t give up on your passion projects, you never know when you might hit gold.
