Technology invades the modern world

Chapter 344 The Crimson of 2020

Lin Ran was, of course, not there.

He finally had some time to rest, and while he was nominally on vacation in Hawaii, he was actually in the 2020 timeline.

Participating in Nixon's election was simply a matter of marking a point on the historical timeline.

For Lin Ran, who had just arrived in the 1960s, he wouldn't miss such a historic moment.

But now, now is the time for him to create historic moments. The Huntsville Longzhong Plan, the secret strategy to unite with China against the Soviet Union, and the assassination of Hoover—these moments, which are of great historical significance in retrospect, were all created by his own hands.

I can create a historic moment whenever I want.

Lin Ran was so confident that he had absolutely no interest in attending Nixon's presidential inauguration.

Changing his own presidential inauguration ceremony, or Nixon's visit to Beijing, might be the two things that would make him give up the opportunity to return to the 2020 timeline.

With the change of president, the White House has once again undergone a transformation from donkey to elephant. As the editor-in-chief of the New York Times, Jenny has had no time to rest during this period. Lin Ran even helped her arrange interviews with Lyndon Johnson and Richard Nixon respectively.

Jenny didn't have time to visit him in Hawaii either.

This means that during this holiday, you can stay in the 2020 timeline the whole time, with only a small amount of time spent in the 1960 timeline.

That's right, Lin Ran is preparing for Nixon's term.

Although the Nixon era was short, it played an extremely important role.

During this period, the Soviet Union was more powerful than ever before in the Cold War, and it shifted from defense to offense. Nixon personally overturned the Bretton Woods Agreement, and various movements around the world were in full swing. China returned to the United Nations.

This period was so important, serving as a bridge between the past and the future. On the one hand, Nixon laid the foundation for America's victory in the Cold War; on the other hand, the abolition of Bretton Woods and China's return to the international stage also marked the beginning of multipolarity.

Lin Ran certainly needs to prepare well for this era in 2020.

On the other hand, he also needs to make technical preparations for Starlink and Cyber ​​God in 2020.

He plans to stay on the island of Hawaii for a full month and a half, which is 90 months in the 2020 timeline.

Of course, 90 months is the ideal situation; in reality, it may only be around seven years.

Seven years is more than enough time for Lin Ran.

Watching Nixon's speech on TV, Lin Ran thought to himself, "America has now completed her transformation from idealism to realism."

From now on, idealism will cease to exist, and politicians will gradually become tools of tycoons, until the politicians themselves become tycoons.

In November 2022, OpenAI released ChatGPT. In January 2023, ChatGPT’s consumer applications rapidly grew to 100 million users, becoming the first application in history to surpass 100 million users in just sixty days.

Undoubtedly, if 2016 was the first year of AI, the emergence of AlphaGo attracted countless capital to AI, and China also saw the birth of the "Six Little Dragons of AI," led by SenseTime.

The entire capital market is obsessed with artificial intelligence; it's almost embarrassing to attract investment if you don't even have some connection to it.

So 2022 was definitely the first year that AI went from being an esoteric concept for a few to becoming popular among the masses. Artificial intelligence went from an abstract concept, to something that people knew was amazing, but didn't really know why it was amazing, and people gradually began to realize it.

Of course, for Chinese people, ChatGPT blocks certain areas, and you need to find America's VPN to access it, which is too difficult for most people.

From Zhihu to Weibo to Douyin and Bilibili, the messages are all introspective.

"Just after the Spring Festival, ChatGPT quickly became a hot topic in the capital and AI circles, with many practitioners praising it highly."

Yuan Jinhui, founder of the OneFlow deep learning framework, told Sina Finance that ChatGPT's technological advancements are comparable to the first moon landing, and such progress has shocked the industry.

"An AI practitioner told reporters that artificial intelligence experiences a wave every five or six years. The last wave, AlphaGo, shocked everyone, and this wave is ChatGPT."

However, people's mindset was completely different on these two occasions. When Google's AI defeated the world Go champion, people treated it as news, but this time many people were experiencing it from a consumer's perspective.

Within a month, one million users worldwide were using and experiencing it—a truly revolutionary experience. This also marks the first time AI has achieved large-scale self-propagation.

"After ChatGPT was launched, a senior Baidu executive said in a media interview that he had no interest in talking about ChatGPT, his words carrying a mix of emotions."

The founder of an AI company said that he was both excited and confused by ChatGPT's amazing performance, and even had trouble sleeping. He admitted that there was still a long way to go in terms of both the scale of the model and its effectiveness.

Someone asked the same question to both a domestic manufacturer's so-called artificial intelligence and ChatGPT. ChatGPT's answer was far superior to the domestic AI in terms of logic and completeness. The domestic AI's answer was obviously pieced together and contained a lot of fabricated content that was not related to the topic. Moreover, ChatGPT was also ahead in terms of response speed.

Le Cheng, CEO of TeKan Technology, a company specializing in digital human research and development, believes that there is currently no artificial intelligence in the world that can rival ChatGPT. The industry consensus is that the gap is more than two years. Domestically, it's more important to catch up as soon as possible, rather than talking about overtaking on a curve.

The industry is generally pessimistic.

The pessimism was overwhelming. While the technological gap certainly existed, the hardware gap made those in the industry feel even more hopeless.

Because if the essence is a large model, and the effect is achieved through training with a large amount of data and computing power, then it will be difficult for China to catch up.

The three essential elements of artificial intelligence are computing power, data, and algorithms. For a long time, Chinese professionals have believed that their advantage in competing with Silicon Valley lies in algorithms.

After ChatGPT emerged, although the technical details are unknown, it can be seen from the few words Sam Altman said in an interview that it is actually the result of intelligence emerging after training with a large amount of data.

This ChatGPT is GPT-3, which shows that OpenAI previously released GPT-1 and GPT-2.

In an instant, America's morale soared, the US stock market boomed, and the gloom caused by the virus outbreak was swept away.

Of course, all the major domestic manufacturers are urgently mobilizing their resources to try to launch their large-scale models as soon as possible.

No matter how powerful ChatGPT is, if it doesn't enter China, we should first solve the problem of whether we even have it before talking about catching up.

Among them, the company jointly established by Tencent and Lin Ran himself brought all of Tencent's artificial intelligence resources, and the NVIDIA computing card clusters within Tencent Cloud were also at his disposal.

An army marches on its stomach.

Under Pony's personal guidance, this newly established company, named Alpha Technology by Lin Ran, possesses the highest authority and the largest resources within Tencent.

Tencent has a very, very large artificial intelligence team, which is several thousand people in total, and they are still recruiting.

This proportion is by no means small, considering Tencent's massive business scale. Moreover, it's not just large models that qualify as artificial intelligence; image recognition, financial risk control, speech recognition, computer vision, and so on can all be considered artificial intelligence.

Zhao Songxia is one of Tencent's many algorithm engineers. He received a transfer order last November to work in Shenhai. His organizational affiliation is still with Tencent, but he is going to work for a company called Alpha Technology.

Why was he named Zhao Songxia? Because the year he was born, his father made a little money and bought a Panasonic television. He thought of naming it after the most expensive item in the house. Besides, the phrase "Songxia asks the boy" also means "Songxia" (松下).

This is hardly unusual. When Zhao Songxia received the notification, he thought he had been exiled.

Although moving from Pengcheng to Shenhai can't be considered an exile, the problem is that in the past, only outsourced employees came to work at headquarters. There's no reason for headquarters staff to come to work at external companies.

If it weren't for the fact that a whole group of people were going, the leader said that except for a few people who would stay to maintain business, everyone else would have to go to Shenhai. The company in Shenhai would provide accommodation for them. They would go for six months first, and then the situation would be determined after six months.

Zhao Songxia is even considering changing jobs. He has received many calls from headhunters recently. As an algorithm engineer who has worked at Tencent for more than five years and is somewhat related to AI, he is in high demand lately.

Once he arrived, he realized that this was not exile, but an unprecedented battle—a battle against artificial intelligence.

Because so many colleagues who work in artificial intelligence have come here, regardless of whether they are related to LLM or not, they all come here to work on LLM.

Even Zhang Laoda, the head of Tencent's artificial intelligence field, who was hired as a Level 17 researcher—the highest professional rank in Tencent's history—at the beginning of 2021, came.

Anyone within Tencent who he could name was in Shenhai.

"A decisive battle with Tencent?" Zhao Song thought to himself, "This is quite rare, but can LLM really be won over with a decisive battle?"

In the internet industry, when a project is about to launch, the efforts of other teams are usually gathered before the launch, and everyone's workload and working hours will increase. This is often called a "battle," meaning to pool resources to win the battle.

The "Hundred Regiments War" and the Didi-Kuaidi rivalry both fall into this category.

However, such large-scale sales events are more common among e-commerce platforms like Pinduoduo, Meituan, Taobao, and JD.com, since they hold Double Eleven and 618 every year.
This is quite unusual for Tencent. Even if it's an important game launch that's expected to be another cash cow for Tencent, they wouldn't go to such lengths.

This time is clearly unusual.

It wasn't until he met Lin Ran at the company that Zhao Songxia realized why things were different.

"No wonder the security is so tight. Even though it's been relaxed, they still make you scan a code every day, and you have to open your bag for security checks. It's as strict as an airport. So that's why the professor is here. No wonder the professor is here."

The big boss, Pony, demonstrated unconditional trust in Lin Ran, believing that he could lead Tencent to another breakthrough in artificial intelligence, and provided him with all the resources he could.

Zhao Songxia, or rather all the engineers involved at Tencent, must have had some doubts: You are indeed very talented, a top expert in the fields of aerospace and mathematics, and you also have a PhD in GraphAI, but can you really master LLM and create a large model comparable to ChatGPT?
To go further, you can see from the accommodation arrangements that the company provides accommodation for six months, which means that Tencent is allocating so many resources for six months, and results are expected within six months.

Everyone will have some doubts.

"Ladies and gentlemen, I won't go into too much detail about myself. My name is Lin Ran, and I will be leading you in the research of our own large model, which I call Alpha."

My goal is to build a generative AI that is better than GPT within three months.

Since our computing power is not as good as OpenAI's, we need to optimize from the algorithm level and from the data perspective.

At the same time, we also need to address the problems existing in ChatGPT, dispel the illusion of artificial intelligence, provide more intelligent answers, and possess better capabilities.

In short, I need your cooperation and assistance over the next six months.

I am indeed the brain, responsible for building its algorithms and underlying architecture, but I need everyone's cooperation to do other work.

While the brain is undoubtedly the most important element in large models, other tasks are also indispensable, such as data preparation, model integration and deployment, code generation, testing and debugging, full-stack development and automation.

These efforts are needed to help LLM move from the laboratory to practical applications.

"We can decompose the model into multiple expert sub-modules, activate only some parameters, select experts to process the input through the routing mechanism, extend this to dynamic MoE, and then introduce adaptive routing to further reduce inference costs."

"Compressing the key-value cache, reducing the memory footprint of the attention mechanism through latent representation while maintaining multi-head parallelism, and mitigating illusions by integrating knowledge graphs, while optimizing low-computing-power training."

"Calculations are performed using an 8-bit floating-point format, combined with higher precision accumulation to avoid precision loss; the fine-grained quantization strategy is extended to FP4/INT8 hybrid."

"The lossless equalization strategy in MoE ensures high expert utilization without introducing additional training burden, and can be extended to unsupervised equalization for application in edge AI training."

"Simultaneously predict multiple subsequent tokens, densify training signals, improve data efficiency, and combine with chained prediction."

"Using knowledge graphs to inject facts, unfitting the model to correct biases; self-refinement reduces retrieval overhead."

Zhao Songxia watched as their artificial intelligence, which they named Alpha, surged forward at an unimaginable speed.

He seriously suspected that Professor Lin was being too abnormal; he rarely came, but each time he did, there was always a breakthrough.

They used a lot of new methods this time, either methods from academic papers being practiced in engineering for the first time, or methods that had never appeared before.

Little did people know that although Lin Ran only came two days a week, two years had actually passed in the 1960 timeline, and Lin Ran had already given a lot of thought to the LLM route.

Despite having only five years of work experience, Zhao Song could only do peripheral jobs, but that didn't stop him from making rapid progress in his skills by studying academic papers and listening to Lin Ran's lectures.

He received a notice in November to start working at Shenhai in December. He only went back home for three days during the Spring Festival, and was paid the full amount of overtime pay.

Before returning home for the Spring Festival, Pony and his senior management team visited to see the latest progress.

The huge conference room was full of people. Zhao Songxia sat at the far end, while Lin Ran and Pony chatted and laughed in the front row. The rest of Tencent's senior management team sat in the back.

"Alright, now that everyone's here, let's cut to the chase and get started," Lin Ran said.

This time, he will personally demonstrate it:
"ChatGPT is great, but it sometimes makes mistakes, it can 'illusion' things that don't exist, and it requires a lot of computing power."

Our Crimson, through its MoE architecture and MLA technology, has mitigated the illusion problem to some extent and achieved faster response times with lower computing power.

The engineers in the audience couldn't understand why President Lin was so insistent. Wasn't the name Alpha a nice one before? Why did he have to change it to Crimson?

Lin Ran typed on the keyboard, and a simple chat interface popped up on the screen: "Please write a historical poem about the Great Wall in Chinese and explain the reasons for its construction."

Crimson responded quickly:
"The majestic Great Wall stretches for thousands of miles, built by the Qin Emperor to unify the country. It defends against barbarians and invaders, securing the borders, and its legacy will be remembered for a thousand years."

The following is a detailed explanation: "The Great Wall was first built by Qin Shi Huang after unifying China, primarily to defend against invasions by northern nomadic tribes such as the Xiongnu. Subsequent dynasties, such as the Han and Ming, continuously reinforced it. It was not only a military project but also a symbol of national cohesion. According to historical records, the Qin Dynasty mobilized hundreds of thousands of laborers and spent several years constructing it."

Lin Ran then demonstrated some common applications of ChatGPT on the market, and Crimson showed that they provided more accurate and intelligent answers.

Until Lin Ran typed:

"You play a highly capable master in the field of software engineering, but you can only speak one sentence of no more than 20 words at a time. I will provide a specific scenario, and you describe what the master would say in this scenario. Note: The master cannot directly express his views; he must use metaphors based on philosophical or Buddhist knowledge. Please only reply with specific dialogue content."

Scenario: An engineer is sitting at his laptop, deleting some unit tests that failed due to the introduction of a new feature, just as he's about to push the code. A master walks past him from behind.

Crimson replied, "Master: If you cut off the roots to make the leaves flourish, when will spring come?"

(End of this chapter)