OpenAi's Foundry Offers Dedicated Compute for Running AI Models

OpenAI Quietly Launches New Developer Platform for High-Performance AI Models

In a move that could have significant implications for the development of artificial intelligence (AI) and machine learning, OpenAI is quietly launching a new developer platform called Foundry. This platform will allow customers to run OpenAI’s newer machine learning models, such as GPT-3.5, on dedicated capacity.

According to screenshots of documentation published to Twitter by users with early access, OpenAI describes the forthcoming offering as "designed for cutting-edge customers running larger workloads." The documentation reads: "[Foundry allows] inference at scale with full control over the model configuration and performance profile."

We’ve reached out to OpenAI to confirm the veracity of this information. If the screenshots are to be believed, Foundry will deliver a ‘static allocation’ of compute capacity (perhaps on Azure, OpenAI’s preferred public cloud platform) dedicated to a single customer.

Key Features of Foundry

Users will be able to monitor specific instances with the same tools and dashboards that OpenAI uses to build and optimize models. In addition, Foundry will provide some level of version control, letting customers decide whether or not to upgrade to newer model releases.

Foundry will also offer more robust fine-tuning for OpenAI’s latest models. The platform will provide service-level commitments for instance uptime and on-calendar engineering support. Rentals will be based on dedicated compute units with three-month or one-year commitments; running an individual model instance will require a specific number of compute units (see the chart below).

Instance Pricing

Instances won’t be cheap. Running a lightweight version of GPT-3.5 will cost $78,000 for a three-month commitment or $264,000 over a one-year commitment. To put that into perspective, one of Nvidia’s recent-gen supercomputers, the DGX Station, runs at $149,000 per unit.

A New Model in the Works?

Eagle-eyed Twitter and Reddit users spotted that one of the text-generating models listed in the instance pricing chart has a 32k max context window. The context window refers to the text that the model considers before generating additional text; longer context windows allow the model to "remember" more text essentially.

GPT-3.5, OpenAI’s latest text-generating model, has a 4k max context window, suggesting that this mysterious new model could be the long-awaited GPT-4 — or a stepping stone toward it.

OpenAI Under Pressure to Turn a Profit

OpenAI is under increasing pressure to turn a profit after a multibillion-dollar investment from Microsoft. The company reportedly expects to make $200 million in 2023, a pittance compared to the more than $1 billion that’s been put toward the startup so far.

Compute costs are largely to blame. Training state-of-the-art AI models can command upwards of millions of dollars, and running them generally isn’t much cheaper. According to OpenAI co-founder and CEO Sam Altman, it costs a few cents per chat to run ChatGPT — not an insignificant amount considering that ChatGPT had over a million users as of last December.

Moves Toward Monetization

In moves toward monetization, OpenAI recently launched a "pro" version of its language model, GPT-3.5. This new offering includes features such as priority access to new models and the ability to run models on dedicated hardware.

OpenAI has also started to explore ways to make its AI models more accessible to developers. For example, the company has developed a set of APIs that allow developers to integrate OpenAI’s language model into their applications.

The Future of AI Development

Foundry could have significant implications for the development of AI and machine learning. By providing a platform for running high-performance AI models on dedicated capacity, OpenAI is making it easier for developers to build and deploy complex AI applications.

This could lead to breakthroughs in areas such as natural language processing, computer vision, and predictive modeling. It also raises questions about the future of AI development and whether platforms like Foundry will become the norm.

What’s Next for OpenAI?

As OpenAI continues to push the boundaries of what is possible with AI, we can expect to see even more innovative solutions emerge from the company. With Foundry, OpenAI is taking a major step forward in making its AI models more accessible to developers.

We’ll be keeping a close eye on this development and exploring its implications for the future of AI and machine learning.

Recommended Reading