After the layoffs, I became a tech godfather

Chapter 110 OpenAI has been a bit jumpy lately, so we need to limit it!
Hao Cheng invited He Gang to a meal, and He Gang ate it with a lot on his mind.

As soon as the dinner ended, he immediately returned to the hotel and contacted Yu Dong.

"Old He, why do I find that you become so flustered every time you go to Linzhou?" Yu Dong even teased.

"Don't be in a hurry, listen to me."

He Gang carefully repeated what Hao Cheng said in the car to Yu Dong, and then concluded: "Don't think this is just Hao Cheng talking nonsense, talking about the concept of the road in general, and not revealing any AI training methods.

"But in fact, he talks in general terms and chats with a large group of us. He is different!"

"I know that he has trained Xiaosha, and his concept of the road is the right one." Yu Dong took a long breath and said, "Maybe we can find the direction from this general discussion.

"But just listening to these is useless!" After thinking for a long time, Yu Dong said helplessly: "All the things you said, I can only summarize them in three words: 'similar to humans', and this direction has been considered by the team a long time ago.

"But the problem now is not the direction, but the method. We haven't found any method. The only good news is that the cost of training AI in the traditional way has been greatly reduced, and the efficiency has been greatly improved."

What Yu Dong said is true. The cost of training AI has been reduced not only for Huawei, TikTok, Tencent, but even OpenAI.

The reason is very simple. They asked Xiaosha to help them with data cleaning, calibration and feedback.

Data cleaning is originally a very complex and tedious task, and it is also a very critical step in training AI. The higher the quality of data cleaning, the higher the quality of AI training.

In the past, this work was done manually, and AI was also used, but the effect was very poor. It would often feed in some garbage data, causing the model to be polluted and leading to some low-level errors.

Now with Xiaosha, this work can be done more quickly and the accuracy is even not worse than manual cleaning.

A more critical issue is that previous GPT-type models were RLHF, which is reinforcement learning based on human feedback.

How to do it: First pre-train a language model, and then do fine-tuning.

How to fine-tune: you ask a question, the language model gives you an answer, and then you manually rank these answers to obtain a quality-ranked data set. You use this data set to fine-tune the relevant model parameters in turn, repeating the cycle over and over again, and the answer will get closer and closer to what you want.

And now, Xiaosha has replaced this manual step.

The RLHF model was previously thought to be impossible to improve indefinitely. One of the most important reasons is that as the number of parameters increases and the amount of data grows, it becomes increasingly impossible to manually obtain a quality-sorted data set.

So some people think that the self-feedback model, that is, the model that allows the model to evaluate and improve itself, is the future, even though it sometimes seems very stupid.

But now, with Xiaosha, Xiaosha replaces the human in [Reinforcement Learning Based on Human Feedback] and becomes [Reinforcement Learning Based on Xiaosha’s Feedback]. All this becomes possible again!

It not only solves the problem of self-feedback being easily stupid, but also solves the problem of low efficiency and high cost of manual feedback.

This is equivalent to directly combining the advantages of the two models.

Moreover, with such a large-scale operation, there is no need to worry about manpower issues.

Therefore, this is why each company’s models have made great progress now.

It is of course impossible to become as good as Xiaosha - it is a paradox in itself for an AI trained based on Xiaosha to surpass Xiaosha.

However, as long as we are willing to accumulate computing power, infinitely accumulate computing power, and use Xiaosha to replace humans for reinforcement learning for feedback, in theory, we can eventually approach Xiaosha's level.

Of course, theory is just theory. In reality, there is no infinite computing power. Considering the actual situation, it should be possible to reach 60% to 70% of Xiaosha's level by combining this method with ultra-large computing power training for one year.

Huawei has quietly evaluated this and now almost all AI training companies are secretly doing this.

"Do we need to tell Hao Cheng about this?" He Gang asked. "He should know this, right?" Yu Dong was stunned: "In the past, many models used ChatGPT feedback for initial training, and only switched to manual feedback after training to a certain stage. This is a common practice."

"I guess he really doesn't know. He probably isn't paying attention to other AI colleagues right now."

Hearing what He Gang said, Yu Dong's mouth twitched. Yes, they were just a bunch of weaklings. What was there to pay attention to?

"Well, let's talk about it. This matter has a huge impact. Especially OpenAI, whose computing power is huge and has been jumping around a bit recently. We need to limit it."

"Oh, that's what Apple is counting on, right?" He Gang suddenly connected the two things together in his mind.

"Yeah." Yu Dong smiled and said, "The reason why Apple hasn't gotten completely angry yet is that it has received a promise from OpenAI, and the reason why OpenAI is so confident is that they have purchased graphics cards worth hundreds of billions of dollars.

“We claimed to investors that we had developed a new algorithm that could catch up with Xiaosha. In fact, to put it bluntly, it was [deep learning based on Xiaosha].”

"Similar consciousness to that of humans." As soon as Yu Dong mentioned "deep learning based on Xiaosha", He Gang muttered this sentence unconsciously.

"What do you mean by 'similar consciousness to that of humans'?" Yu Dong was stunned and asked.

He Gang recounted to Yu Dong everything Hao Cheng had said on this matter word for word.

“That’s interesting!” There’s no problem with Huawei, but OpenAI has a big problem.

After trillions of cycles, what is the "thought" of ChatGPT trained by Xiaosha's feedback? It is controlled by Xiaosha!
It is said that technology has no borders, but trained AI is actually "biased". Let alone the technical concepts of "convergence towards humans" and "similar consciousness" mentioned by Hao Cheng, even traditional AI is mixed with various ideologies of the trainers.

Even if it is not reflected in AI itself, rule restrictions and human intervention are necessary to achieve this effect.

……

"So that's what happened!"

Hao Cheng has never paid special attention to what users do with Xiaosha. Even if he wanted to pay attention, he couldn't.

Unless it is some illegal or irregular operations, Xiaosha's [AI controllable direction] will monitor and restrict them.

Anything is allowed unless prohibited by law. Xiaosha was used for data calibration, result feedback, and training other AIs. Hao Cheng did not specifically explain this matter, so Xiaosha did not impose any restrictions.

Furthermore, there are requirements for data usage and traceability. Users themselves must know clearly how their data is obtained, how it flows, and what is ultimately done with it.

The public algorithm for data traceability is there and everyone can verify it, and Baiju Technology is no exception.

This is also an open conspiracy to a certain extent, because Xiaosha itself is so powerful. If it were a completely black box, many people would be worried and scared.

Now, Baiju Technology controls the core algorithm and makes peripheral algorithms such as information tracing and recommendation publicly available and open source. Everyone can supervise and verify them, making it much safer to use.

However, what He Gang just said -

Hao Cheng remembered what Li Qingbo had told him before Zhu Yue from Apple came: "Brockman told Apple CEO Cook that OpenAI will solve the problem and reach Xiaosha's level by March next year at the latest."

"You originally thought that Brockman was fooling Cook, but this is actually the case!"

Hao Cheng shook his head and finally understood why Zhu Yue had such an attitude before:
"I was still careless!"

(End of this chapter)