After the layoffs, I became a tech godfather

Chapter 138 The plan was successfully verified
"Wife, you take our daughter to play for a while. I'll go to the study to do some work. Just for a while!" Thoughts came to him, and Hao Cheng was about to break his promise of not working when he returned home.

"Ten o'clock!" What could Ning Hui do? She had to let him do what he wanted.

"It won't take long. It will take at most an hour!" Hao Cheng pointed at the lamp with a finger: "I promise!"

"What a shame!" Ning Hui cursed with a smile, and sighed in his heart: Just be busy, at least it seems a lot more cheerful than last year.

……

It really doesn't require much thought today, because the relevant knowledge has been sorted out in the past, but I didn't take it seriously at the time.

Coming to the study and flipping through what he had written down before, Hao Cheng's thoughts came flooding out like a spring.

If model training and improvement are addition, making the model bigger and stronger, then pruning, quantization, and distillation are subtraction.

Many people may wonder, if we do subtraction, doesn’t that make the model weaker?

This is indeed the theory, but sometimes, being weak or not is not particularly important. As long as you use the right strength, it can still make a big splash and play a big role.

For example, the weights of some neuron nodes are very small, and their impact on the model loading information is negligible. Cutting them off will not reduce the accuracy too much, but it can greatly reduce resource consumption.

In addition, the deployment of a fully omniscient and omnipotent model like Xiaosha and the corresponding calculation and reasoning each time require a large amount of computing resources. But in fact, is it really necessary every time?
Apparently not!
Take Xiaosha’s most widely used software development capabilities as an example. It only requires an accurate understanding of requirements and the ability to write code. As for the gradually enhanced ability mappings such as emotional mapping and subjective judgment, these are completely unnecessary!
Not only does it consume more computing power, but it also slows down the response speed.

And even the necessary parts can be handled in layers:
Taking software development capabilities as an example, at the most general level, it can be divided into two parts: one is understanding the requirements, and the other is developing based on the requirements.

As for the second part, only an industry model of "software development" is needed.

Therefore, we first decompose the user needs at the input end, and then cut Xiaosha into small models such as software development, legal services, security services, etc. as needed, and call one or several of them according to the decomposed needs.

For example, the first trial of Xiaosha, which was previously regarded as a classic case, only required three basic capabilities: semantic understanding, legal services, and software development capabilities.

Only in very rare cases is it necessary to use Xiaosha's full model.

Even if the full model is needed, the resources consumed by calling the full model throughout the process and calling the full model only at the necessary steps are very different, but the results may be exactly the same.

……

"I am stumped by training and improvement, but cutting and distilling are still easy!"

After studying in the late-night class for so long, Hao Cheng still has some skills. Although he is not good enough in model training of micro-consciousness body, there is still a way to compress Hao Cheng.

What's more, Hao Cheng has Xiaosha, an excellent auxiliary tool.

"Xiaosha, help me count and analyze all user behaviors, and list the capabilities and functions they use most frequently, sorting them from largest to smallest."

Originally, this amount of work would have been impossible to complete in less than ten days or half a month, but now with Xiaosha, the time required was suddenly shortened to an instant.

Hao Cheng's current idea is to use the complete form of Xiaosha as the full model, to hold the line, and not to move it out easily, and only when the problem really cannot be solved, then move here.

Then, we classify the small sand into several small models of their respective fields and professions. Later, user requests will be screened and processed by the small models in the corresponding fields.

For example, when interacting with people, the language model will of course take over first and then process it step by step.

As for the use of AI at the element level, AI ensuring network integrity, AI developing applications and software, etc., the first step can be omitted - the interaction between AI and machines also needs to be transferred through the language model, which is meaningless except for consuming resources.

……

The prototype of the first phase plan was quickly designed.

At this stage, Hao Cheng did not seek perfection, but only did one thing - cutting, that is, model pruning.

For example, the first small model to be made is: language recognition.

Then cut off all capabilities except language recognition. You only need to recognize speech and convert it into text. You don’t even need to understand it. That is the job of the second model.

According to this theory, under the guidance of Hao Cheng, Xiaoshaben took charge and one professional or complex small model after another was quickly born.

……

The next morning, as soon as he arrived at the company, Hao Cheng called Cao Song and said, "Free up the computing server in our office area. Just stop the load node and it will automatically be diverted to other servers."

This computing center was originally from Huawei, and its overall scale is relatively small. It does not have many tasks to undertake now. Hao Cheng plans to directly stop its external services and use it as an experiment for internal AI training.

We also take this opportunity to physically separate the small sand for our own use from the small sand that provides services to the outside world.

And today, what Hao Cheng wants to do is to try out the results of Xiaosha’s work last night.

"By the way, don't use Xiaosha, do it manually."

"Okay!" Cao Song responded. He was just wondering why he was asked to do this operation when Xiaosha could just do it with one word. After Hao Cheng's instructions, he suddenly realized:
According to the principle of the nearest node first, if you call Xiaosha here, it is likely that this server will be connected. Asking Xiaosha to stop the server on which it is running may easily cause unpredictable consequences.

About half an hour later, Hao Cheng received news that Cao Song had completed the process. He quickly asked Xiaosha to cooperate with him to deploy small models with professional capabilities one after another.

Then, the devices used by your mobile phone, including mobile phones, tablets, software development interface services, etc., will directly skip the distribution load and specify the IP entrance of this service at the Innovation and Entrepreneurship Center.

"Xiaosha, how are you?" Hao Cheng asked.

[Yes, everything is normal.] Xiaosha replied.

As usual, Hao Cheng called Xiaosha and asked him to help him handle various affairs and transfer various approvals and messages. After a morning, there was no perceptible difference from usual.

"The plan has been successfully verified!"

Hao Cheng has already reached this preliminary conclusion, and the next step is to conduct rigorous testing to see how much computing power resources it can save.

(End of this chapter)