0

Cerebras co-founder JP Fricker helped design the largest chip ever built by ‘making the naysayers his research team’ and considering confidence a contagion that can spread to investors, technical experts, new hires, and customers.

Jean-Philippe Fricker is a semiconductor visionary who succeeded in the audacious task of carving a single, massive processor from an entire silicon wafer — a feat that massive tech companies had attempted and failed to achieve for decades. Here is part one of a two-part interview that looks at how Cerebras co-founders pushed past the skeptics to spearhead wafer-scale computing with the Wafer Scale Engine (WSE), a“plate sized” silicon wafer that acts as a single, massive processor – the largest chip ever built. With that “crazy” idea, Cerebras just raised $5.55 billion in the largest IPO of the year.

To defy conventional semiconductor wisdom, one has to be very bold, very confident, and very “open” to criticism and expansion of an original idea. “JP” Fricker, co-founder and chief system architect at Cerebras, knew early on that cutting a very large chip out of a single 300mm wafer could “change the era of compute,” but he needed something that had many, many compute elements close to one another. That was quite a big problem 10 years ago when the idea was born, “as there weren’t that many options,” acknowledges Fricker. “We looked at the reconstruction of wafers, but pretty quickly, we found that that technology was not mature yet, so monolithic integration on a single wafer would be the way we had to go.” With that very novel idea, he began to hear a lot of disbelief from skeptics who’d say “no, no, other people have tried. They failed. You cannot do that.”

When it comes to the naysayers, Fricker says it was extremely important to “listen quite carefully,” and to believe intensely in the background experience and expertise of the founding team. That, he contends, helps you avoid the pitfalls that trapped those who came before you.“One pitfall is to just do what has been done before, and to get stuck in it. If you don’t think that you can do something ‘better’ and apply what you’ve done ‘better,’ you do not think ‘differently.’ You have to apply the reasoning that got you to where you are.”

To  avoid that pitfall, Fricker and his co-founders looked at the problem with an open mindset, repeatedly asking, “What is the fundamental problem we want to solve and what do we want to achieve?” Rather than get sucked into the trap of all the details that made people fail before, they avoided getting trapped in the particulars, like “who can I find that can package this for me? Or, what is the technology available today to actually package it?” Rather than tailor their thinking to the packaging that was available at the time, they approached the problem from the other direction.

“We had to reason with the fundamental physics related to making a very large chip, which doesn’t expand or contract at the same rate as the package around it.” Indeed, they had to accept that the chip would not expand with temperature in the same way the surrounding package would. They also knew that fact would make it very hard to package such a large chip. “Yes, bonding or soldering something to it would make it break. Those were the fundamental truths at the time,” admits Fricker. “To understand the fundamentals, you will have had to go through quite a bit of training, experimentation, and basic tinkering to figure out that what is in the textbook pertains to someone else’s expertise and experiences, not your own,” says Fricker, who posits that it’s important for innovators to realize that what’s in the textbook “is not all the expertise that exists. That’s how you go beyond the existing expertise, and that your own expertise can move you beyond. Do not be bound what was done in the past.”

By thinking in a way that is fundamental and yet slightly different, you can ensure you are not “bound by what someone has done in the past,” says Fricker, who advises that other potential founders look at the underlying physical components that might be the limit, but not at the expertise or experience or solutions that characterized the others that came before you.

Build belief in the people behind the idea

To have people believe in you, and to draw them into a problem so that they become a participant who then transfers their thinking and knowledge to you, it’s important to view challenges through other people’s eyes.

“Once you have convinced yourself other paths might be possible, you can start to convince others of the possibility by effectively communicating, so you don’t get treated like you are completely crazy.” Fricker adds that it’s important to support your vision with a recognition of the fundamental physical components and real-world constraints. “If you have a fil rouge of explanation of why it is possible…of why you want to do it and why it’s something that benefits the end product, it’s a lot easier to defend it [to a VC team] – not with all the details – but with the confidence that there is a technical path and steps to take.” Because investors and VC teams aren’t necessarily knowledgeable about physics or the AI domain, and because they see a huge number of ideas on any given day,  he posits it was important to first convey the big picture as opposed to defining in granular detail what a “wafer scale engine” would be.

“It’s really a belief in people that drives VCs and entrepreneurs to support and invest in novel ideas. You have to project confidence first to the VCs, and then again to the entrepreneurs who you are asking to fund your vision, and then again to the people you are going to hire to carry that vision forward.”

To pitch the idea, the Cerebras team had to convey who the people were behind the vision. “We put together a very short slide deck, with an ‘about us’ about who they’d be investing in. The team we were asking them to trust with their funds.” Fricker says it was an unusual “about us” for the VCs, because it was a larger team of founders than most VCs encounter: Andrew Feldman, co-founder and CEO, who Fricker says really knew how to tell the story, as well as Gary Lauterbach, who pioneered new ideas about computer architecture, and then what Fricker says were three “younger” people with complementary expertise: Sean Lie, who focused on ASIC design; Michael James, who was more focused on mathematics and software engineering; and Fricker on packaging and systems.

“That was the team the investors had to believe could complement one another to tackle the main problem in AI: the lack of enough compute. We presented what we wanted to do to solve it—a product 1,000x better than a GPU–and all without talking about wafer scale.”  At most, the people listening, some of whom may have had some software engineering backgrounds,” thought 5%,10%, 20% improvement of GPU was all that was possible. “But we came in and said ‘no, it’s going to be a 1000x better.’”

The ‘how’ doesn’t come into play until the due diligence phase, during which they hire ‘the best CTO on the planet’ to look at your idea, and to look at the team, to see if you’re worth the investment.” – JP Fricker, co-founder and chief system architect, Cerebras

To convince investors it was worth going to the due-diligence phase, Fricker says it was critical to emphasize the team’s expertise. “The ‘how’ doesn’t come into play until the due diligence phase, during which they hire ‘the best CTO on the planet’ to look at your idea, and to look at the team, to see if you’re worth the investment.” Fricker admits that was a week of very hard and very long interviews, where each team member had to describe the “what” and “how” to someone who understood the domain and the fundamental details, and who would be assessing whether the team was capable of pulling off the vision. According to Fricker, the vision alone is never enough to get someone to invest: “you have to communicate the ‘why’ and then the ‘how’ of the path that will take you there.”

He says once a start-up is given the funds to do the work, it’s important to surround yourself with “people who can ‘do,’ but also who can ‘think’ –  who can move in a similar direction but expand on what you think.” Fricker emphasizes that alone, no one has the power to bring big ideas to fruition. “Everyone who came on board at Cerebras added a little piece to the overall puzzle – to the wall of knowledge that drives new ideas forward.” He admits, that the expertise and knowledge are one thing, but resilience and endurance are another important factor. “There were so many people we hired that were, in the first two weeks, super excited, only to a few weeks later be asking ‘what the heck did I do?’ It sometimes feels like an impossible task.”

Nourishing confidence amid new hires

Fricker looks at confidence as a contagion that founders and leaders have to nourish and spread.  “As a founder, it’s important to nurture the idea that ‘yes, you can, and yes, it is possible, and you repeat the fundamentals that your team may have forgotten, or perhaps never had the opportunity to ask about.” Once the right questions are allowed to be asked, the level of confidence rises, and becomes contagious. “Others absorb the confidence, and then when I start to doubt, they can help me to keep up.”

As a founder, he says, you have to have the vision and the leadership, “but many times, it’s your team that together projects the vision of what you’re going to build going forward.”

Customers as the Engine

In deep tech, getting the funding and then actually engineering a breakthrough is a battle that is ultimately decided by the customer, the end user. Real-world customers either ignite or extinguish a flame, so when it comes to an ambitious architecture, communicating to customers and rolling their feedback into the design is essential.

“Once you get the feedback from potential customers at first, and then actual customers eventually, you boost validation that this is not only interesting, but actually useable and valuable,” says Fricker. “That gets you on the right path and pushes your ability to go further. OK, it’s validated, so let’s perfect it, improve it, make it cheaper, make it easier to build, make it more available.

Building a wafer-scale chip meant refusing to create a traditional fallback option. Early customers who signed on provided the necessary market proof that a massive, single-silicon architecture was actively needed to clear the AI training and inference bottleneck.

It then became a question of convincing a manufacturing company, in this case, TSMC, to alter its standard production lines for a product that didn’t exist yet. Demonstrating that customers were ready to consume a massive amount of compute gave TSMC the confidence to back Cerebras’ design. Since about 2016, the two companies have been co-developing the new manufacturing framework for the WSE.

In part II, we will delve into the fabric that allows for very fast communication among the WSE’s cores, and how the underlying mathematics that drive LLMs and the communication among processing elements and memory devices.