Celestial AI raises $100 million to expand Photonic Fabric technology platform

Heavenly AI, an optical interconnect technology developer, announced a successful Series B funding round, raising $100 million for its Photonic Fabric technology platform. IAG Capital Partners, Koch Disruptive Technologies (KDT) and Temasek’s Xora Innovation fund led the investment.

Other participants included Samsung Catalyst, Smart Global Holdings (SGH), Porsche Automobil Holding SE, The Engine Fund, ImecXpand, M Ventures and Tyche Partners.

According to Celestial AI, their Photonic Fabric platform represents a significant advance in optical connectivity performance, surpassing existing technologies. The company has raised $165 million in total from seed funding through Series B.

Tackling the “memory wall” challenge.

Advanced artificial intelligence (AI) models, such as the widely used GPT-4 for ChatGPT and recommendation engines, require exponential increases in memory capacity and bandwidth. However, cloud service providers (CSPs) and hyperscale data centers face challenges due to the interdependence between memory scalability and compute, commonly referred to as the “memory wall” challenge.

The limitations of electrical interconnection, such as limited bandwidth, high latency, and high power consumption, hinder the growth of AI business models and advances in AI.

To address these challenges, Celestial AI partnered with hyper scalers, AI computing, and memory vendors to develop Photonic Fabric. The optical interconnect is designed for disaggregated, exascale, and memory clustering processing.

The company says its proprietary Optical Compute Interconnect (OCI) technology enables disaggregation of data center scalable memory and enables accelerated computing.

Memory capacity is a key issue

Celestial AI CEO Dave Lazovsky told VentureBeat, “The key issue going forward is memory capacity, bandwidth, and data movement (chip-to-chip interconnectivity) for large language models ( LLM) and recommendation engine workloads. Our Photonic Fabric technology allows you to integrate photonics directly into your silicon mold. A key benefit is that our solution allows you to deliver data anywhere on the silicon die all the way to the point of processing. Competitive solutions such as Co-Packaged Optics (CPO) cannot do this as they only provide data at the edge of the mold.”

Lazovsky says Photonic Fabric has successfully addressed the difficult beach problem by providing significantly higher bandwidth (1.8 Tbps/mm²) with nanosecond latencies. As a result, the platform offers fully photonic compute-to-compute and compute-to-memory links.

The recent round of financing has also attracted the attention of broadcom, who is collaborating on the development of Photonic Fabric prototypes based on Celestial AI designs. The company expects these prototypes to be ready to ship to customers within the next 18 months.

Enabling accelerated computation via optical interconnection

Lazovsky said that data rates also need to increase as the volume of data transferred within data centers increases. He explained that as these speeds increase, electrical interconnects encounter problems such as loss of signal fidelity and limited bandwidth that fails to scale as data grows, thus limiting overall system throughput.

According to Celestial AI, Photonic Fabric’s low-latency data transmission facilitates the connection and unbundling of significantly more servers than traditional electrical interconnects. This low latency also allows latency-sensitive applications to use remote memory, a possibility previously unattainable with traditional electrical interconnects.

“We enable hyperscalers and data centers to disaggregate memory and compute resources without compromising power, latency and performance,” Lazovsky told VentureBeat. “Inefficient use of server DRAM results in $100 million (if not billions) of waste between hyperscalers and enterprises. By enabling memory disaggregation and memory pooling, we not only help reduce the amount of memory spent, but also demonstrate memory usage.

Storage and processing of larger data sets

The company says its new offering can deliver data from anywhere on the silicon directly to the point of processing. Celestial AI claims that Photonic Fabric pushes the limits of silicon edge connectivity, providing a packet bandwidth of 1.8Tbps/mm², which is 25 times greater than what CPO offers. Furthermore, by delivering data directly to the point of processing instead of the edge, the company claims that Photonic Fabric achieves 10x lower latency.

Celestial AI aims to simplify enterprise computing for LLMs such as GPT-4, PaLM, and deep learning recommendation models (DLRM) that can range in size from 100 billion to over 1 trillion metrics.

Lazovsky explained that because AI processors (GPUs, ASICs) have a limited amount of high-bandwidth memory (32GB to 128GB), companies today need to connect hundreds to thousands of these processors to drive these models. However, this approach reduces system efficiency and increases costs.

“By increasing the addressable memory capacity of each high-bandwidth processor, Photonic Fabric allows each processor to store and process larger blocks of data, reducing the number of processors needed,” he added. “Providing fast chip-to-chip links allows the connected processor to process the model faster, increasing throughput and reducing cost.”

What’s next for Celestial AI?

Lazovsky said the money raised in this round will be used to accelerate production and commercialization of the Photonic Fabric technology platform by expanding Celestial AI’s engineering, sales and technical marketing teams.

“Given the growth in AI workloads due to LLMs and the pressures it places on current data center architectures, demand is rapidly increasing for optical connectivity to support the transition from general compute data center infrastructure to ‘acceleration of computation,” Lazovsky told VentureBeat. “We expect to increase headcount by approximately 30% by the end of 2023 to 130 employees.”

He said that as LLM usage expands into various applications, infrastructure costs will also increase commensurately, leading to negative margins for many Internet-scale software applications. Additionally, data centers are hitting power limits, limiting the amount of processing that can be added.

To address these challenges, Lazovsky aims to minimize reliance on expensive processors by providing high-bandwidth, low-latency chip-to-chip and chip-to-memory interconnect solutions. He said this approach is intended to reduce companies’ capital expenditures and improve the efficiency of their existing infrastructure.

“By breaking the memory wall and helping improve systems efficiency, we aim to help shape the future direction of AI model advancement and adoption through our new offerings,” he said. “If memory capacity and bandwidth are no longer a limiting factor, they will allow data scientists to experiment with larger or different model architectures to unlock new applications and use cases. We believe that by reducing the cost of adopting large models, more companies and applications would be able to adopt LLMs faster.”