Susana SchwartzTable of Contents
OCP Global Summit: SK hynix touts 3x-5x TTFT improvement
In AI infrastructure, Time-to-First-Token (TTFT) is onen of the most critical metrics shaping how hardware is built, how software is optimized, and how businesses buy compute.
At the OCP Global Summit in San Jose, SK hynix SVP Hoshik Kim introduced the Memory-Centric AI Rack-Scale System (MARS), a disaggregated architecture designed to seamlessly coordinate compute and memory resources. Kim highlighted that traditional processor-centric architectures suffer from severe structural limitations, such as idle memory that cannot be shared and rigid scalability that requires purchasing entire server nodes just to expand memory.
To solve these constraints, MARS decouples memory from traditional CPU/GPU limitations to establish a dedicated, massive memory pool, effectively shifting infrastructure from per-server memory headroom to pool-level efficiency. To demonstrate its real-world performance gains, SK hynix applied MARS to a multi-turn agentic AI inferencing service. The system stored and consumed exploding Key-Value (KV) caches within a shared memory system without exhausting GPU memory and storage, delivering a 3x to 5x improvement in TTFT.
Alongside MARS, the company also highlighted its progress on High Bandwidth Flash (HBF), an OCP-standardized technology capable of scaling storage capacity by up to 14x.
Susana Schwartz
Technology Editor
RCRTech
AI Infrastructure Top Stories
DT on AI uplink: Talking of Deutsche Telekom’s mobile-capacity Capex, SVP and incoming CFO Dhananjay Mirchandani said he doesn’t expect AI-related mobile uplink to require incremental radio-access-network investment.
Connectivity inside the DC: Nokia writes that high-performance XPUs’ effectiveness depends on more than compute: the network also influences the performance of distributed processing, model training, and compute resources.
AI Today: What You Need to Know
Tackling tokenomics: In Forbes “How two CIOs measure AI’s true ROI,” CIOs Paul Beswick and Rishi Tripathi talk about enterprise AI ROI and the “build versus buy” decisions they must make as cloud models’ hidden costs emerge.
Nokia multi-PON flexibility: Nokia has launched a 50G/25G/10GPON solution designed to accelerate AI-ready fiber broadband. The company says it’s the 1st with a 16-port line card solution supporting GPON, 10G, 25G, 50G PON on every port.
Anthropic warns of AI self replication: Anthropic has now warned that small AI models may be months away from being capable of self-replicating across internet-connected household devices (IoT) without needing a data center.
Russia – Ukraine DC war?: When Russia attacked Ukrainian data centers and telecom infrastructure, it reporedly did not expect widespread outages caused by Ukrainian drone strikes to its three largest Yandex-owned data centers.
Iran threatens U.S.-owned infrastructure: Iran’s security chief, Mohsen Rezaei, has issued a warning that Persian Gulf cables, data centers and AI facilities may be attacked if there are further U.S. attacks on Iranian civilian installations.
RCR Events
Industry Resources
Report: NTN in motion — evolving standards, expanding services
