The tech stack of Generative AI and its investment opportunities

In a rapidly evolving digital era, Generative AI marks a significant turning point, offering new opportunities and challenges for developers, businesses, and investors. Endeit follows the latest developments in this space and this blog navigates through the current tech stack landscape for generative AI applications.  

Revolutionizing software development with Generative AI 

While natural language models have been around for some time, OpenAI has unlocked an AI revolution. ChatGPT became the fastest-growing consumer app in history, hitting 100m users in just two months. I personally think LLM (large language models) has had the biggest impact on software development; what once required weeks or months of development can now be achieved in a single weekend hackathon. It’s pretty much the norm these days for software developers to use generative AI tools to help with coding. This shift in the way of working happened really quickly, almost overnight. 

The generative AI landscape is bustling with activity, driven by foundational models that continue to push the boundaries of what’s possible. Also, the swift advancements in the generative AI infrastructure layer empower more developers to develop LLM-based applications. This dynamic environment is a double-edged sword; it’s filled with more opportunities than ever before, but navigating it requires a keen sense of where the technology is headed. 

Applications of generative AI: horizontal and vertical  

The application layer can be categorized into horizontal and vertical-gen AI applications.  

• Horizontal applications are developed for specific use cases or workflows and across various industries. Think of content generators, AI-powered presentations, code-assistants, etc. However, as an investor, distinguishing the standout players in this now-crowded field poses a real challenge. 
• Vertical applications, on the other hand, offer targeted solutions for specific industries. Whether it’s enhancing legal research with Harvey or revolutionizing protein design with Cradle, these applications demonstrate the potential for sustainable growth and innovation in niche markets. They rely on deep, often proprietary data, making them particularly appealing from an investment perspective due to their tailored approach and potential for impact.    

Both horizontal and vertical applications are developed using application frameworks that make Gen AI more accessible to developers. Tools like LangChain provide an open-source toolkit that Python developers can use to bring their application visions to life.

The underlying generative AI tech stack 

Diving deeper into the generative AI tech stack, we explore foundational models, the data that powers them, and the processes for evaluation and deployment. The tech stack’s base layer, powered by models like GPT-4, is where the magic begins. To evaluate the investment opportunities in the tech stack market of generative AI, you have to make an assumption on what the end state of the foundational model market will look like. 

Investment paths: choosing your route  

When investing in the generative AI tech stack, envisioning the future market structure of the foundational model market is crucial. We’re looking at two main scenarios here: it will either become a market dominated by a few of the large tech firms or an open ecosystem. 

If you believe this market will have an oligopolistic structure and be dominated by OpenAI, Google, AWS, and Microsoft, it would be challenging, if not impossible, for smaller and innovative companies to make their mark. This scenario suggests a more cautious approach when investing in foundational models and when investing in the tooling layer as the big tech firms will then also dominate the tooling space. Investments in startups would then be directed towards the application layer instead.  

The alternative is a more open ecosystem that would result in a larger, more flexible, and more innovative market. Here, the limitations of one-size-fits-all models spark the creation of specialized solutions catering to unique needs across various industries. This open ecosystem fuels innovation and broadens the investors’ horizons, offering opportunities in foundational models, the tools that build AI applications, and the applications themselves. In my opinion, this would be the preferred scenario.  

The power of proprietary data 

Tech investors know that the data feeding the AI model is key for evaluating an AI business model. The uniqueness of a startup’s data can significantly differentiate it in the AI market. To create datasets both structured datasets (like ERP exports) and unstructured data (such as emails) are used to create unique datasets and to train and refine AI applications. Another term you often encounter when reading about Gen AI is the vector database. Vector databases store information as vectors, numerical representations of data objects clustered based on similarity. This design enables low-latency queries, which is necessary for generative AI applications.  

The evaluation stage is especially interesting as developers need to optimize for model performance, costs, and latency. Finding the optimal solution can be done by iterating on prompts, switching between foundational model providers, by finetuning the model. Finding the right balance between these factors can be challenging, and startups like Orquesta and Promptlayer, jump in this area by offering low/no code tools that facilitate the prompt iteration process and enable switching between foundational models to evaluate associated costs.  

Deployment and continuous observation

Once a model has passed the testing phase, it moves to deployment. The deployment can be done using third-party services or self-hosted with frameworks such as Fixie. The final, ongoing phase involves observability — closely monitoring the AI’s performance in a live environment to track its efficiency, cost-effectiveness, and responsiveness. Tools provided by companies like Arize and Observe are crucial at this stage, offering developers the insights needed to maintain and improve their AI applications post-launch.

For an overview of the Gen AI tech stack companies to watch, see the image below.  

Image 1. Gen AI tech stack companies.

Conclusion  

The generative AI sector is rapidly evolving, with new startups and solutions emerging at every layer of the tech stack. Keeping an eye on these developments and the players involved is essential for anyone active in the AI space, whether you’re a developer, investor, or tech enthusiast. The landscape is filled with opportunities for innovation, growth, and investment, highlighting the dynamic nature of the generative AI market and its potential to reshape industries.