I am the only one who practices magic: I practice magic in the city

Chapter 309 Model Collapse (4506)

Even after the last red ripple disappeared, Fang Yu was still worried. He searched his brain carefully three times again, and only after confirming that there were no red ripples of Vernis remaining in his brain did he breathe a sigh of relief.

This kind of extremely detailed inner vision is very energy-consuming. Even though Fang Yu has strengthened his physical strength to the peak of carbon-based creatures, a few drops of sweat still seep out of his forehead.

Phew, so dangerous.

Wiping the sweat off his forehead, Fang Yu couldn't help but feel scared.

Fortunately, I obtained the "Model of God" from Aizen Myōō, fortunately, I discovered the changes in my feelings towards Shen Shuyao in time, and fortunately, I was self-reflective enough to discover the changes in my self-cognition.

How could I call myself the God of Desire?

How could I identify with the identity of these losers who abandoned their physical bodies and hid in the astral world? And how could I be so proud of my identity as the God of Desire?

Would Sun Wukong be proud of his title of Fighting Buddha? He would only feel ashamed!

As for himself, he claimed to be God twice in Miami, using the word "God", which has a specific meaning in the West, which may have included some of his bad taste.

But just now, I subconsciously identified myself as the God of Desire in my heart, which was very abnormal.

That would be fine if it were just that. Maybe it was just my vanity that made me want to play tricks on others.

What really made Fang Yu realize that something was wrong was his condescending and indifferent attitude when he saw Shen Shuyao dressing up carefully to meet Musk.

It has been a while since I entered the arcane world. Although I felt a sense of superiority over ordinary people, I never treated the people around me as straw dogs.

However, subconsciously, I just developed a mentality of looking down on Shen Shuyao, like an ant, thinking, "As long as I want, you are just an XX."

This is definitely not normal.

The great arcanists of the Great Arcane Age didn't float like this, so how could I possibly float like this?

If I were this kind of person, I would have been given a title like "The Four Gentlemen of Haixi" and everyone would know me, right?
When did you start to change?

Thinking about it carefully, it seems that since the time when Vernis invaded the sea of ​​​​consciousness, my mentality has undergone subtle changes.

This change is different from the emotional change caused by being immersed in the belief of desire at the Four Seasons Hotel in Los Santos two months ago. Instead, it is a subtle and unconscious change in one's perception and attitude towards certain things.

That's why I have been becoming more and more unscrupulous in doing things these days.

After a while, when the model of Vernis completely takes over his brain, even if he is not possessed by this despicable god, he will definitely become a complete evil god!
Very risky.

and many more.

So, during this period of time, I have been with so many models and female celebrities...

Well, the means used and the attitude afterwards may have something to do with Verness’s model parasitism, but “doing” this thing itself should still be out of one’s own will.

After all, Vernis’s parasitic model has now completely disappeared, and he doesn’t feel that he made a mistake in the past two months.

Looking back now, the newly added Victoria Garrity and Camilla Moreau tasted pretty good.

Fang Yu did another full-body scan and was sure that the model left by Vernis had indeed been reduced to ashes.

"Dong dong dong." There was a knock on the door.

"Fang Yu, Fang Yu, are you okay?"

"Cousin, are you sick?"

"Don't you have classes this afternoon? Why are you here in the company? Do you have any sense of professionalism when addressing people by their titles or names?"

"It's none of your business? Humph. Anyway, my brother and I don't plan to work after we get married. I'll just stay at home and take care of the kids."

"I don't plan to give birth to a child so early and let you take care of it. If you want to be a nanny, I have your resume on hand and will upload it to 58.com for you in a while."

……

Hearing Lu Jiayan and Shen Shuyao quarreling in low voices outside the door, Fang Yu suddenly felt as if he was in another world.

It was as if everything I had experienced these days was covered by a veil, and now the veil was suddenly lifted.

"I'm fine, I just checked the backstage."

Fang Yu opened the door of his private laboratory, one in his left hand and one in his right hand, feeling very happy.

"Are you really okay?" Shen Shuyao touched Fang Yu's forehead worriedly, and then carefully looked at Fang Yu's complexion. There was indeed nothing wrong, so she continued: "Du Changgeng seems to have found something, do you want to go over and take a look?"

Oh, by the way, there is more important business to do.

Fang Yu slapped his forehead, kissed both of them on the cheek, and quickly walked to the open office area.

"Boss! We did ten rounds of testing, and the results..." Team leaders Du Changgeng, Hua Zecheng, Yang Zilong, and Lu Si all had serious expressions on their faces, and it was obvious that the situation was not very optimistic.

Someone pushed a chair towards Fang Yu. Fang Yu sat down and looked at the monitor, and couldn't help frowning.

"After ten rounds of training, model 2 using the original data showed no performance degradation. The overfitting phenomenon can be corrected through regularization and data cleaning."

"Model 1, which uses the Orange Big Model to generate images for training, has obvious overfitting. Boss, take a look. After ten rounds, the generated images begin to converge."

“And model 1, which is trained using the data generated by model 3 in each round…”

Du Changgeng's face flashed with worry. "By the fifth round, the overfitting of Model 5 had surpassed that of Model 3 after ten rounds of training. By the ninth round, the entire model was on the verge of collapse and could hardly output meaningful data."

"We were surprised by this result, so we used the data generated by different models to train a new 3B sub-model, named Model 4. The final performance after training was very similar to that of Model 1, with no improvement."

"Initially, we thought it was a data quality issue. There might be some distortion in the image data of the Orange model, so we repeated the training with text."

"The error rate of the text data generated by the Orange Big model is much lower than that of the original data used by Model 2. However, Model 1, which was finally trained with the text data generated by the Orange Big model, still experienced performance degradation, and the overfitting phenomenon after multiple rounds of training was nearly five times higher than that of Model 2."

"And in this case, Model 3 also experienced model collapse after eight to nine rounds."

“That is to say, no matter how high or low the quality of the data generated by AI is, as long as the AI-generated data is used to train the big model, the performance of the big model will deteriorate. This… This is incredible!”

"It's just like..."

"Inbreeding?" Lu Jiayan asked curiously.

"Yes! Inbreeding! That's right!" Du Changgeng clapped his hands and looked at Lu Jiayan in surprise. He didn't expect that this young proprietress, who was obviously a layman, had a good understanding.

Fang Yu stroked his chin thoughtfully.

He had already guessed about this result, otherwise he would not have proposed a comparative experiment. The problem found by Yang Zilong and Du Changgeng did seem a bit strange at first glance. The same data, after being processed by AI and then used to train another AI, was actually not as good as the manually cleaned data of lower quality.

Using the data generated by this trained AI to continue training AI, no matter which AI is being trained, will cause performance to further deteriorate.

This question may seem strange, but in fact, if you think about it for a while, you will understand that it is probably a problem caused by the diversity and complexity of data being continuously simplified by AI.

Not to mention the orange model, even for grapefruit, the data generated cannot be 100% correct. Although the errors may be less than those in artificial data, the errors lack diversity and complexity.

To put it bluntly, AI is smart in all ways, but humans are stupid in all kinds of ways.

But from the perspective of racial inheritance, stupid and weird humans have stronger racial inheritance capabilities than AI that they create that is smarter than humans.

Wow? If you think about it this way, a fool like Meng Guangkang has made a great contribution to the inheritance of human civilization?

Almost cured my stupidity?

By repeatedly using this type of data for training, similar errors will be considered real by the large model, thus becoming a "thought stamp" and continuing to produce more erroneous content.

Using these contents for training again will produce a "bias-variance trade-off", that is, the content with higher probability will be further strengthened, and the content with lower probability will be continuously ignored.

The next generation of models will result in more data loss.

At the same time, due to the characteristics of large models, "function approximation errors" will occur. Large models will retain the wrong things they have learned. After accumulation from generation to generation, they will eventually become all errors, thus losing the ability to correct errors.

This situation is indeed very similar to inbreeding depression caused by inbreeding.

From this perspective, the AI ​​big model actually has a certain biological nature.

It's just that the population of this species is too homogeneous, resulting in inbreeding.

This problem is not too urgent at present. After all, there is a lot of raw data available on the Internet. The problem can be solved by increasing investment in data sets, obtaining more raw data, and increasing the number of people and methods for data preprocessing.

But in the long run, no, not too long, the explosion of AI must be exponential. Maybe in two to three years, more than 70% of the data on the Internet will be generated by AI, and humans will hardly be able to distinguish whether it is AI-generated data or real artificial data.

At that time, no matter how large the parameters of the AI ​​model are, it may face performance degradation or collapse due to the contamination of training data by AI-generated data.

"Thirty trillion." Fang Yu quickly estimated a number.

???

Several team leaders all had question marks on their faces.

"The available valid data on the Internet can probably support the Orange model to a scale of 30 trillion parameters. After that, it will inevitably be affected by inbreeding."

Fang Yu pointed to the data on the display.

Thirty trillion is not so bad. The Orange model currently being trained has only 1.5 trillion parameters, but it has already begun to take shape as a prototype of strong artificial intelligence.

The 30 trillion-scale model will surely have achieved super artificial intelligence. By then, the problem of diversity in AI-generated data should have been solved.

Then there's no problem.

Several team leaders breathed a sigh of relief.

Is there anything wrong?
Fang Yu's eyes flickered.

Even if the Orange Big Model achieves super artificial intelligence, this problem will still exist as long as the diversity issue is not resolved.

Even if various technical means are used to increase the complexity of AI data as much as possible and the model does not collapse in the end, this large model is destined to be unable to carry its own vision.

Some ideas that I came up with from referring to the "God Model" can solve this problem...

However, before that, we still have to take over the search engine business first.

Only by taking over the search engine can we have enough data resources to support the evolution of the Orange model into a super artificial intelligence.

Only after the Orange model evolves into super artificial intelligence, the second-generation model generated with reference to the "God Model" can be truly useful.

"Shu Yao, notify the 'Y' project team. I want to hear the progress report of the 'Y' project in five minutes."

Fang Yu looked at the time. It was already four o'clock in the afternoon. In another half an hour, the relevant leaders of Haixi County would bring Musk to the company, which would be just enough time for a meeting.

I have to go to Xinhao at 11 o'clock in the evening to catch up with Da Mimi and Repa. Chihiro has put a lot of pressure on them recently, so I have to go and help them relieve the pressure.

It's just right that I can meet my grandfather's introducer in Xinhao tomorrow morning. The person my grandfather introduced should be reliable, but I still have to meet him first.

I have to go to Los Santos at noon tomorrow. Scarlett went to Middle-earth to shoot Ghost in the Shell. Kendall was quite reserved and just harassed her through iMessage, but her bestie Gigi Hadid kept sending provocative snapchat messages.

We must teach them a lesson!
"This "fencing gate" scandal is a blow to both factions, but it is a bigger blow to Trump. Karl Rove even believes that the congratulatory faction should convene a national committee to disqualify Trump from running for election."

While Musk was on the phone, he was also sending emails to various SpaceX employees. On the other end of the phone, his chief of staff Sam Taylor was telling him about the current situation in Miami.

For a businessman of his level, whether he hates Zheng Zhi or not, he must care about Zheng Zhi.

"It's difficult. At least two-thirds of the committee members need to vote in favor, which is almost impossible. It looks like Hillary will definitely win this time." Musk's fingers tapped quickly on the computer.

"'Fencing Gate' broke out too late. The vote counting for the Indiana primary scandal was completed the next day. Although the national convention had not yet been held to obtain the party nomination, Trump had already locked up the nomination votes in advance."

"If Trump is forced to withdraw from the election at this time, the congratulatory faction is likely to cause a factional split. Paul Ryan and Mitch McConnell, even if they do not support Trump, will not risk a factional split by pushing for a replacement nominee. Of course, the premise is that the Trump family needs to cut ties with Jared Kushner immediately, but in this case, the Jude Foundation will certainly fully support Hillary."

"You're right, Elon. It's widely speculated that Ivanka might announce her divorce from Jared Kushner in the next few days to cut ties and play the victim. But in that video, Jared said that Trump's family was just nouveau riche and that Ivanka married him just to get support from Jude's consortium, which would still be a huge blow to Trump."

"Boss, we have already started to land. We will be below 3,000 meters in five minutes, and the Wi-Fi on the plane will be shut down."

Musk's captain informed Musk using a loud speaker in the cabin.

"You heard me, Sam, I'm going to disconnect first, stay connected."

"Well, Elon, it's actually the same if I go this time. There's no need for you to go there in person."

"No, this time I must see this Orange Technology with my own eyes." Elon Musk shook his head. "In this month, at least 10,000 AI applications have been born, 50% of which call the API interface of the Orange Big Model, and the others also use the open source Orange Big Model, and this number continues to rise."

"I am sure that the 2.5 billion parameter Juzi that has already started formal operation is not all. They may already have a large model with trillion or even more parameters."

"I have to see with my own eyes what progress they have made."

Musk hung up the call and looked down through the window.

The clouds were so thick that I couldn't see anything.

(End of this chapter)