By Nicholas Larsen, International Banker
While many see the next major bottleneck in the development of artificial intelligence (AI) emerging through limitations in computing power, others identify communication as the most crucial incoming challenge to be resolved, with the training of modern AI models now requiring tens of thousands of chips to communicate continuously being particularly problematic. As such, companies across semiconductor, networking and other key AI-related sectors are committing vast sums to the development of a technology once regarded as little more than a niche branch of optical engineering: photonics.
Despite the hundreds of billions of dollars committed to building powerful AI systems—including faster, larger and more specialised graphics processing units (GPUs)—one problem has proven inescapable: Processors can only be as effective as the networks connecting them. With the AI systems of the future expected to connect hundreds of thousands of processors simultaneously, more energy will be needed to transmit information between them, as well as between memory systems, storage devices and networking equipment. Electrical signals generate heat, lose efficiency over distance and face considerable limitations as data volumes swell.
Connectivity within AI servers is still mostly enabled by copper wires, the expense and speed limitations of which are being increasingly exposed as distances increase and traffic volumes rise. “One of the main bottlenecks for the performance of AI models is the speed of communication between chips and between chip servers,” Gil Luria, head of technology research at D.A. Davidson, explained to CNBC in late May.
Moving information between processors has thus become one of the most expensive, energy-intensive and technically challenging aspects of computing. Signal degradation, power consumption, heat generation and latency are significant constraints that are expected to weigh more heavily as AI systems expand.
With many of the most important technological ambitions of the next decade—hyperscale AI, fault-tolerant quantum computing, autonomous vehicles, advanced robotics and next-generation communications networks—depending significantly on moving and processing information not only more efficiently, but also beyond what today’s electronic systems can accommodate, alternative solutions to electronics are becoming necessary.
Instead of transmitting information via electrical currents, photonic systems use particles of light (photons) to carry large amounts of information.
Enter photonics, a technology that utilises light to offer a potentially superior approach to the problem. Instead of transmitting information via electrical currents, photonic systems use particles of light (photons) to carry large amounts of information. “While electronics use electrical signals to carry and process data, photonics leverages light—enabling significantly faster speeds, greater bandwidth, and lower energy consumption,” explained Roberto Piacentini Filho, director of marketing for design engineering software at Keysight.
This method generates less heat and consumes less energy than many conventional electronic communication systems. “The faster the communication, the faster the user can get their answer or their task executed,” Gil Luria asserted. “By moving the connections between chips and between servers to optical, the performance of the models can improve significantly.”
Photonics has been successfully applied in fibre-optic communications for several decades; indeed, today’s internet capabilities would not be possible without optical fibres that carry information across oceans and continents. Rather than simply connecting data centres with optical cables, the technology is being embedded more deeply into computing systems, with researchers increasingly exploring the use of light within servers and networking equipment.
AI’s rapid development has only accelerated the need for—and interest in—photonics. A data centre that can train models more quickly while consuming less energy benefits from lower operating costs, improved utilisation and potentially higher returns on capital. At hyperscale, these gains become meaningful, with the industry’s largest operators projected to spend hundreds of billions of dollars annually on AI infrastructure and capital expenditure.
But with modern AI training clusters already operating at a scale that would have seemed implausible only a few years ago, researchers are now observing that machine-intelligence workloads are colliding with the power, memory and interconnect limitations of conventional architectures. Often described as an interconnect bottleneck, the main challenge is that data movement increasingly dominates overall system performance. In such environments, reducing networking bottlenecks by even a small percentage can generate significant economic value, which, in turn, generates demand for alternative approaches capable of scaling beyond incremental transistor improvements.
With its remarkable ability to provide higher bandwidth and greater parallelism in data movement, integrated photonics is increasingly being viewed as a solution to this bottleneck. This interest is currently highlighted by the growing amounts of venture capital (VC) being committed to backing photonic-chip developers, optical-interconnect firms and quantum-computing startups.
With a stellar track record of producing powerful AI processors, Nvidia has also become increasingly vocal about the importance of optical networking and photonics. Jensen Huang, chief executive officer, previously described the physical development of AI capacity as “the single largest infrastructure buildout in human history”, underscoring why seemingly obscure technologies such as silicon photonics are attracting unprecedented attention.
Recent months have seen the Santa Clara-based tech giant investing at least $6.5 billion in companies involved with photonics development, including Lumentum, Coherent and Marvell, as well as Corning, which is focused on optical-connectivity solutions and which has received both equity investments and substantial prepayments from Nvidia to accelerate the construction of new optical-fibre manufacturing facilities.
“Photonics represents a way for Nvidia to scale their AI infrastructure without the energy costs that staying with electrical and copper will incur,” Alvin Nguyen, a senior analyst at Forrester, told CNBC. “By investing in photonics companies, Nvidia is making sure that advancements in photonics continue, and it will prevent them from hitting a scalability and performance wall that will occur if they remain on electrical and copper.”
Reports in early June also revealed that AI hardware and silicon photonics start-up Lightmatter had joined Nvidia’s NVLink Fusion ecosystem to accelerate the deployment of high-performance optical connectivity for AI infrastructure. According to the company, the collaboration will enable Lightmatter to deliver Co-Packaged Optics (CPO) and Near-Packaged Optics (NPO) products that are compatible with Nvidia’s optical and SerDes technologies, thereby creating “a unified platform for semi-custom AI factories while reducing fiber and connector requirements by 50%”.
Photonics’ appeal has been further illuminated by growing interest from China, which has launched several new initiatives as part of Beijing’s efforts to advance the country’s AI capabilities amid US-imposed semiconductor restrictions. China’s approach to investing in photonics is also distinctly broad, combining university research, venture funding, manufacturing support, industrial policy and commercial deployment efforts. Rather than viewing photonics as a standalone technology, Chinese policymakers increasingly regard it as part of a larger computing ecosystem.
Photonic computing is “an important pathway for achieving breakthroughs in computing power, offering advantages in bandwidth, latency, and energy efficiency.
In June, for example, China officially launched the Shanghai Key Laboratory of Integrated Photonic Computing Chips and Systems. Located at the city’s Jiao Tong University, the facility is the country’s first industry-academia platform dedicated to advancing integrated photonic chips, photonic architectures, optical components and AI applications, all in support of next-generation AI systems. Photonic computing is “an important pathway for achieving breakthroughs in computing power, offering advantages in bandwidth, latency, and energy efficiency,” according to Zou Weiwen, a director at the laboratory and a photonics professor at the university.
Although AI dominates current investment flows, quantum computing may ultimately prove equally important. If photonic architectures prove scalable, photonics could become one of the foundational technologies of the quantum era. Several of the industry’s most ambitious quantum-computing companies are pursuing photonic architectures. Companies such as PsiQuantum and Xanadu are developing systems that use photons as carriers of quantum information. Photons are attractive because they are less susceptible to some forms of environmental interference that complicate other quantum-computing approaches.
Manufacturing remains difficult. Producing photonic-integrated circuits at scale requires specialised fabrication techniques that remain less mature than those used in conventional semiconductor manufacturing. Cost is another challenge. Many photonic solutions still struggle to compete economically with established electronic systems outside high-performance applications. Integration presents a third hurdle. Modern computing infrastructure was designed around electronic architectures. Introducing photonic components often requires substantial redesign of systems, software and manufacturing processes.
There is also a risk that investor expectations outpace commercial reality. Technology markets have a history of extrapolating promising breakthroughs too aggressively. Photonics may eventually transform computing, but the timeline is unlikely to be measured in quarters.
The next major milestones are likely to include broader adoption of co-packaged optics, photonic-integrated circuits, optical-AI interconnects, photonic accelerators, quantum-networking infrastructure and increasingly sophisticated sensing systems. The most likely future is one in which photon and electron technologies increasingly converge. While electronic processors will continue to perform many computational tasks, photonic technologies will increasingly handle communications, specialised processing workloads, sensing applications and quantum-information transmission.