Table of Contents
As copper cables hit physical limits at 200G to 400G, co-packaged optics and the OCI-MSA are unlocking multi-rack AI scaling and speeding time-to-revenue for data centers.
In this episode of RCRTech AI TechTalk, Near Margalit, vice president and general manager of Broadcom’s Optical Systems Division, talks about the evolution from copper and traditional pluggable optics toward integrated optics (such as Co-Packaged Optics and the Optical Compute Interconnect standard from the OCI-MSA consortium founded by Broadcom, AMD, NVIDIA, OpenAI, Microsoft, and Meta).
In large-scale AI clusters, moving data between chips can consume 15% to 25% of total system power. When scaling across thousands of GPUs with high-speed SerDes, retimers, switches, or pluggable optical transceivers, the constraints of power and physical space grow rapidly. That’s why there’s such a keen interest in OCI-MSA and Co-Packaged Optics (CPO), which hold the promise to free up and redirect megawatts of electricity toward AI processors so that computing capacity can drive faster time-to-revenue. According to industry estimates, bandwidth and power efficiency demands could drive a 30%-37% CAGR for the CPO market.
“AI is exploding, and in large data centers, you have to connect thousands and even millions of GPUs,” says Margalit, explaining the physics wall of copper as data is pushed to 200G and 400G per lane, causing electrical signals in copper to degrade rapidly. After just 1 to 2 meters, the signal becomes too weak or distorted to read reliably without burning excessive power to boost it.
The first thing engineers did to connect those tightly integrated GPUs was focus on copper links and copper backplanes, Margalit explains. “But as they went to expand beyond a single rack, the physical reach of the copper interfaces became limited.”
That left data center designers stuck between two imperfect choices. While optical fibers solve copper’s distance limit, traditional pluggable modules proved too bulky, power-hungry, and hot for dense in-rack connections, with cost and reliability the “Achilles’ heel” of optical links, according to Margalit. That is the bottleneck Broadcom is attempting to resolve by moving the optical conversion right next to the switch chip—eliminating the long, power-hungry copper traces inside the server box.
With Co-Packaged Optics (CPO), optical engines sit directly inside the main AI processor (ASIC) package. That merger of silicon chipmaking and photonics eliminates long copper traces and lowers costs through wafer-level manufacturing. Using the Optical Compute Interconnect (OCI) standard to attach optics directly to switches or XPUs, data jumps straight from the chip onto parallel, low-speed NRZ optical lanes—cutting both heat and power consumption.
Solving those power and cost bottlenecks unlocks multi-rack scaling: “You are no longer constrained into an individual rack. You’ve instead allowed yourself to create something like a super-node or super-cluster that could be 8 or 16 racks – each with 72 GPUs. That means you could be at 1,000+ GPUs in a single effective node for the GPU.”
Yet reaching that 1,000-GPU scale requires data center operators to rethink how they manage hardware risk. Moving from pluggables to Near-Packaged Optics (NPO) and then to CPO involves an evolution in risk management and a change in mindset about how data centers can be engineered.
According to Margalit, CPO is where you start thinking about the optics not as an independent element, but as a core function of the ASICs, whether it’s switches or XPUs. He says “the evolution is happening, with different hyperscalers going at different speeds, but the ‘North Star’ is fully integrated, where optics are directly on the interposer or directly attached to the ASICs.”
Separating the laser from the processor
To make that deep integration practical without sacrificing reliability in the CPO architecture, Broadcom separates the fragile laser light source from the hot processor. The laser is placed outside of the package in a pluggable module called an ELSFP (external laser small form-factor pluggable), which sits on the front panel of a server or switch and supplies continuous laser light to CPO engines inside the box.
Separating the ELSFP from the main chip is a “game changer,” says Margalit, in that the external module acts like a central power supply. “The light is routed into the central chip and gets distributed like a power supply into the different lanes, which saves you precious space on the ASIC” and preserves the “replaceability of the light source if there are failures,” not to mention the opportunity to tap a multi-vendor system for the light source.
DWDM for NRZ-type signaling
Once that external laser light reaches the chip, the next challenge is encoding data onto it without burning extra power. Rather than pushing one lane to 200G or 400G using complex signaling and power-hungry digital signal processors (DSPs), OCI-MSA uses Dense Wavelength-Division Multiplexing (DWDM) to send multiple simpler, lower-speed NRZ streams down a single fiber simultaneously.
“With OCI-MSA, we say, hey, if we’re attaching the optics directly on the switches themselves, there’s no reason to generate that high-speed serial link. We can stay at low-speed, NRZ-type signaling, using multiple wavelengths to generate the amount of bandwidth we need,” says Margalit.
He likens this to a “gear shift down to a lower speed.” Instead of pushing a single serial lane to 400G, you can send four wavelengths at 100G each, delivering the same bandwidth but with far lower power consumption and better bit error rate (BER) performance. Whether downshifting by a factor of 4x or 8x, Margalit notes it is “incredibly powerful for both signal integrity and power consumption.”
Crucially, Broadcom isn’t keeping this architecture proprietary. “We’re trying to be as open as possible, because we want to create an ecosystem as opposed to creating a unique Broadcom technology,” Margalit concludes. “We’re trying to help other vendors come up to speed.”
The specification can also be modularized to welcome more ecosystem partners. “What’s on the fiber is always standardized, so whether it’s a module or fully integrated, you’re always interoperable from system A to system B,” Margalit explains. “That means if one company is making a GPU and wants to talk to a Broadcom switch, it’s going to be interoperable.”
While the standard makes direct ASIC integration easier, Margalit admits that modularizing sacrifices some power efficiency. Even so, it offers flexibility for operators that want to use an adapter module that converts standard IEEE 200 Gbps signals into an OCI fiber interface, which can communicate with a fully integrated switch.
Looking ahead, Margalit’s vision for the upcoming year is to see OCI-based links integrated directly onto Broadcom switches and XPUs. “That would be powerful and give clear direction to the industry that is manufacturable and… really great for the industry to take.”