Returning to his senior year of high school, he went all in and made billions by buying at the botto

The next morning.

Palm Beach International Airport.

As originally planned, Chen Mo and Lu Jingyi will return directly to Hong Kong from here.

As soon as the convoy pulled into the airport, Chen Mo received a call from Ilya.

"Chen, you haven't left the United States yet, have you?"

He's at the airport.

"Don't go yet."

Ilya's voice was hoarse.

Chen Mo glanced at the time.

"What happened?"

"The first scaling training of the next-generation language model failed."

Is there a problem with the model structure?

"It's not a model."

The sound of keyboard typing came from the other end of the phone.

"Last night, we expanded the training scale to 512 GPUs as originally planned."

"Single-machine testing is normal, but the more machines connected, the lower the training efficiency becomes."

"The video memory is full of duplicate data, and a large number of GPUs are waiting for other nodes to complete the calculation. The time spent transferring parameters between machines even exceeds the actual time used for training."

Ilya paused for a moment.

"Finally, the system lost synchronization in the seventh hour, rendering all training results invalid."

Chen Mo did not speak immediately.

Just because one machine works very fast doesn't mean that putting hundreds of machines together will increase the speed by hundreds of times.

The model needs to be split across different GPUs.

After each GPU completes its own calculation, it also needs to exchange results with other nodes.

How is the data distributed?

How to utilize video memory.

When will we communicate?

If any link in the chain becomes blocked, a multi-million dollar GPU cluster will be put into a waiting state.

No matter how great the computing power on paper is, it cannot be converted into real training ability.

What is the current utilization rate?

"An average of 32 percent."

Ilya answered.

"At its worst, it was less than 20%."

OpenAI currently has more than two thousand GPUs.

If this efficiency is maintained, the actual computing power used will be less than half of the nominal size.

The problem will only worsen if the cluster is expanded fourfold in the future.

"Before the last meeting ended, you said there was only one way left."

Chen Mo recalled the conversation that had been interrupted when Musk pushed open the door.

"Is this it?"

"right."

Ilya said.

"Google has more machines, engineers, and data centers than we do."

"We will never catch up with them if we only purchase GPUs."

"OpenAI must build its own computing system to allow each GPU to do more work."

"We need people who truly understand CUDA, video memory, communication, and distributed systems."

"I've already asked Sam to look for suitable candidates in Silicon Valley."

"No need to look anymore," Chen Mo said.

Ilya was silent for two seconds.

Do you have any recommendations?

"I know one."

Chen Mo hung up the phone and looked at Lu Jingyi sitting next to him.

"Have the crew change the flight plan."

"Let's go to San Francisco first."

Lu Jingyi had obviously figured out the reason from the phone call just now.

"Lin Qihang?"

Chen Mo smiled.

"Do you still remember?"

"I looked at someone in the Nvidia corridor for half a minute and then they told me to write down their name."

Lu Jingyi took out her phone.

……

The Gulfstream changed course and flew back to the west coast of the United States.

After the plane reached cruising altitude, Chen Mo did not allow his team to contact Lin Qihang privately, but instead dialed Huang Renxun's number first.

"Mr. Chen?"

Huang Renxun was clearly somewhat surprised.

"I just saw the news about Falcon 9."

"Your trip to the United States seems to have been very exciting."

"There's one last thing to do."

"Is it related to Nvidia?"

"It's related to one of your engineers."

There was a brief silence on the phone.

"Who?"

"Lin Qihang".

Huang Renxun quickly remembered the name.

"So you called today to try and poach someone from Nvidia?"

"I'm going to invite him to join OpenAI."

Chen Mo didn't beat around the bush.

"Responsible for large-scale GPU clusters and distributed training systems."

"He only recently joined Nvidia."

"The CUDA kernel and memory optimization team also needs him."

"I know."

Chen Mo said.

"That's why I'm calling you first. If you just want to take the person away, I can have a headhunter contact you directly."

Huang Renxun gave a soft snort.

"Buy Nvidia stock first, then poach engineers from Nvidia."

"Mr. Chen, aren't your investment returns too high?"

"OpenAI plans to expand its GPU scale to more than 8,000 units in the next 18 months."

Chen Mo said.

"More will be added later."

"Lin Qihang joining OpenAI will not cause Nvidia to sell a single less GPU."

"If he can solve the problem of large-scale training efficiency, it will actually make us buy more."

"Are you using my engineers to cultivate my biggest future client?"

"almost."

Huang Renxun was so angry he laughed, but he was a businessman, and no businessman could refuse huge profits, so he didn't argue.

Chen Mo continued, "OpenAI can establish a joint computing systems lab with NVIDIA. NVIDIA will be in priority supply negotiations for the first batch of 8,000 GPUs."

Huang Renxun held his phone, deep in thought. For Nvidia, large-scale artificial intelligence models were still a very early market.

The vast majority of GPU revenue still comes from gaming and professional graphics.

If OpenAI is willing to help NVIDIA test large-scale clusters with real training tasks, the value would go far beyond a batch of hardware orders.

Moreover, Chen Mo called him directly, which, according to Chinese custom, is a matter of courtesy before resorting to force. If he didn't agree, AMD next door wasn't going to back down either.

When are you arriving in San Francisco?

Around one o'clock in the afternoon.

"Come to Nvidia at 3 o'clock."

Huang Renxun said.

"I'll arrange for you to meet."

Whether he wants to leave or not is up to you.

"That's enough."

"besides."

Huang Renxun added a sentence.

"Not a single one of those eight thousand cards can be missing."