About 100 Days of AI:
I am Nataraj, I decided to spend 100 days in starting Jan 1st 2024 to learn about AI. With 100 days of AI is my goal to learn more about AI specifically LLMs and share ideas, experiments, opinions, trends & learnings through my blog posts. You can follow along the journey here.
There is a lot happening in generative AI space. So much so that Nvidia the arms dealer of generative AI with its H100 chips is now surpassed Amazon’s market valuations. It is also a very dynamic space with lots of activity from startups and the big tech companies. In this post I will highlight three big trends that we might see in the next 12-18 months.
1 – Data Deals
For last two decades we kept saying data is the new oil. This phrase will have real implications this year. A lot of companies have accumulated unique sets of data and have found a real product market fit. Recently a news broke out the Reddit made a deal to access to its data to an unnamed company for $60M. The terms of the deal are unknown but that’s $60M new revenue for Reddit. Look out for more deals from companies that have unique sets of data.
2 – Focus on Open Source Models
Since Open AI launched chat gpt, the conversation was dominated by gpt series base models which are closed models, meaning no one except the Open AI team knows the details of how the model was trained, what data it was trained on & the parameters of the model. We will see more Open Source models take stage and capture attention. The main player in this arena to look out is Meta (Facebook). Meta is taking the page out of Google’s approach in mobile. While Apple’s mobile operating system was closed, Google strategy was to be an open source mobile operating system that fuels 80%+ of world’s mobile phones. So keep an eye out on Meta.
3 – Small Language Models
Open AI proved to the world what is the power of Large Language Models (LLMs). But as every company in AI is racing to build product ready products and features and trying get enterprises adopt AI for their business. LLMs might not always be the best solution. A smaller model which can take less compute to train and uses less but more quality data might actually be better. These models referred to as SLMs could be run on mobile phones which will increase the access of AI to wider audience. Some of the SLMs to check out are:
One of the main forcing function to adapt and innovate with SMLs is to reduce the compute cost and there by making AI more accessible.
4 – Focus on LLM Security
I am not talking about data security to train the LLM. I am referring to instances where users can hack LLMs to do something that they are not supposed to or break just them. There are a lot of security issues that are cropping up on how to hack LLM behaviors. Some include:
- Jailbreak to make LLM answer something that it shouldn’t like making a banned substance at home.
- Prompt Injection to hide prompt instructions that break LLMs
- Data poisoning is embedding bad data on the internet which will be used by LLMs and can later be misused.
There are more and more ways we are currently discovering that LLMs can be compromised as we fix the ones we know about. This will be one of the active areas where progress will be made in 2024 both on research & product side.
5 – Regulatory Implications
It pretty clear that AI is going to have huge implications on the society across all sectors. It will cross paths with every aspect of society and as we have seen every big company announce major AI initiatives and investments into AI during 2022-23.
- Microsoft invested in Open AI & Inflection
- Google & Amazon invested in Anthropic
- Jeff Bezos invested in Perplexity
- Airbnb acquired a stealth AI company called gameplanner.ai
Regulators were already focused on big tech which blocked them doing any acquisitions in last 3 years. With this in the background we will see regulators both in the U.S and E.U closely watch Ai investments and also potentially detangle some of the investments. The investments were made primarily to avoid acquisitions because people who are running big tech companies know that it is going to be an uphill battle to get them approved. But even some of these investments might also come under scrutiny.
That’s it for Day 14 of 100 Days of AI.
I write a newsletter called Above Average where I talk about the second order insights behind everything that is happening in big tech. If you are in tech and don’t want to be average, subscribe to it.
Follow me on Twitter, LinkedIn for latest updates on 100 days of AI. If you are in tech you might be interested in joining my community of tech professionals here.