I am the only one who practices magic: I practice magic in the city

Chapter 316 Y Search Out to Sea (6143)

Is this data crawled in real time? How is it possible?

How could Youzi Technology have such a large data center and bandwidth?

Not to mention Youzi Technology, which only received 10 billion Malaysian dollars in investment, even for Rice, whose cash flow has basically returned to positive, it would be a fantasy to invest in a search engine!

"Real-time crawling? Does Yuzu Technology have enough bandwidth and servers?"

Lei Jun couldn't figure out how Y-search was implemented by Youzi Technology.

The development of search engines today, whether it is Robert Lee's hyperlink technology or Google's pagerank technology, essentially starts from one or more well-known websites through web crawlers, and continuously crawls web pages and reads web page content through various web links.

The captured web page content is not used directly for search, but is analyzed and key information in the page, such as text content, title, keywords, links, etc., is extracted and stored in the search engine's index library.

This index library is like a directory of Internet content, helping search engines quickly find relevant pages when users initiate queries.

The difference between Robert Lee's hyperlink technology and PageRank is that Robert Lee solves the problem of crawling methods, while PageRank solves the problem of empowering web pages.

Two pages with the same content, a page from the White House and a personal page from a child in Africa, will obviously have different weights.

Google's PageRank algorithm weights these web pages and calculates which pages are more valuable, so that these pages are easier to search.

These two technologies are also the most basic technologies of today's search engines, and almost all search engines are built on these two technologies.

But this brings up a problem.

Bandwidth and extremely large database issues.

Bandwidth determines the search engine's crawling speed and user experience speed, while the database determines the accuracy and richness of search results.

Countless new web pages are created on the Internet every second. Where is the database of crawled links stored? How much server space is needed?
Although it only stores links and content indexes, the entire Internet web page volume is too large, and even this small part is not something that any small business can afford.

Google spends as much as 7 billion dollars every year on adding, updating, and maintaining servers, and this amount of money is increasing every year.

Both Google and Qianxun were entrepreneurs who entered this field in the wild days of the Internet. At the beginning, they did not need to invest too much server resources to crawl all the web links on the Internet.

But it is not like that now. After more than a decade of development, the Internet has become a behemoth. The number of Internet users has exceeded 34 billion, accounting for 45% of the world's population.

If we exclude preschool children who have not yet registered an Internet account and elderly people who have no knowledge of the Internet, the proportion may have exceeded 65%.

You can imagine how huge the amount of data on the Internet is today.

Search engine giants such as Google and Qianxun have grown step by step along with the Internet. Their revenue growth rate is even faster than the growth of the Internet. Naturally, they can continuously increase their investment to add new servers and respond to user needs.

This is also the reason why there are no new entrants in this industry.

This is a completely accumulation-based industry with a very deep moat that is simply not something that ordinary companies can cross.

If a company wants to overthrow the dominance of Google or Qianxun by relying on product strength indicators such as search experience, content richness, and search accuracy, the only way is for a giant or big boss to invest tens of billions of dollars regardless of returns, crawl the entire Internet for content, and use sophisticated algorithms to create a search engine that can compete with Qianxun or Google in terms of product strength.

This is just a test of strength. It is hard to say whether it can really surpass Qianxun and Gugou.

For this reason, based on cost considerations, search engines will not set a uniform crawling frequency for each web page.

The crawler will dynamically adjust the crawling frequency based on the importance of the web page, the update frequency, and the website's crawling strategy.

Important web pages, such as various news sites and the search engine's own news center, may be re-crawled every few minutes, while infrequently updated pages may be re-crawled every few days, weeks or even months.

But the Y search that Lei Jun and Zhou Shuzi just saw not only crawled some web pages that are generally believed that should not be crawled frequently, but the results captured were only a few minutes ago.

For example, there is a self-media article written by Dazui and published 5 minutes ago.

Generally speaking, this kind of self-media will be crawled by search engines very rarely. Unless it can be found through vertical searches such as searching for Toutiao accounts in Toutiao, it will not be found using Qianxun or Gugou.

Just like this webpage, due to the problem of crawling frequency, this article cannot be found by searching Qianxun and Gugou.

But Y search did find it, and the quality of this article is not low.

Could it be that Ysou just happened to crawl this link?
Isn't that just too much of a coincidence?

"Ysearch is not a completely real-time search. It is actually in two directions compared to traditional search engine technology." Fang Yu put out the cigarette in the ashtray.

He was not a heavy smoker and chose to meet Lei Jun and Zhou Shuzi outdoors because Lei Jun was a heavy smoker, smoking two packs a day. Smoking was strictly prohibited indoors in Xinhao, so a coffee shop with an outdoor area was more conducive to smokers discussing matters.

"The search technology used by YSearch is completely different from traditional search technology. Traditional search technology is to download links and then assign weights to the links and index them to build a database."

"YSearch uses a large model to analyze and learn the data connections of the 1.7 billion web pages on the Internet. It makes probabilistic judgments on which links may have higher quality and gives search results based on this probability."

“Therefore, YSearch does not need a lot of servers to store the specific data of these web pages. It’s just that the indexes of these links have been ‘learned’ by the big model. We only need to store the links.” (Note 1)

“When a user searches, the big model will automatically provide links that it believes meet the user’s needs based on the user’s intent or its own judgment.”

"As for the frequency of crawling, it's actually not that difficult. According to real-time data from internetlivestats, there are currently 13 billion web pages on the Internet, % of which are empty links or broken links."

"After removing these, there are only more than 600 million links. Among the 600 million links, nearly 400 million web pages are 'inactive websites'."

"The Orange algorithm makes judgments based on 'data tags'. If the 'data tags' that have been crawled have not changed, they will not be crawled again. After the 'data tags' are changed, the Orange big model will actively crawl the updated web pages to ensure that its own data is up to date, and then create a new 'data tag'."

"The benefit of this technology is that we don't need to build as many large data centers as Qianxun and Google."

"A single-story data center covering an area of ​​20,000 square meters should be enough to meet the search needs of all users in Da Zhou. The investment may be less than one percent of Google's. Currently, YSearch uses Ali Cloud."

"Of course, if we want to develop other businesses, such as the current cloud storage, encyclopedia, library, map, email and other functions of Qianxun and Gugou, we still need a large data center to support them."

"Another benefit of this technology is that it is very easy to review and filter. When the review and filtering rules are determined, YSearch can filter the information that needs to be reviewed more accurately to avoid accidental damage."

"In the AI ​​era, uncontaminated data is extremely important, but the Zhouwen data on the Dazhou Internet is now too polluted, and the effect of training large models is very poor."

"A considerable part of this is due to accidental review, which results in poor trainability of Zhouwen's data. Therefore, under the Y-search algorithm, we can accurately identify the search results that need to be filtered, reducing 97.98% of data accidental failure."

"Although this may not bring any significant results in the short term, over time it will have considerable benefits for the Internet data resources of the entire Great Zhou."

"The bandwidth required is not much different from Qianxun's current bandwidth requirements. After all, bandwidth is required for both data transmission and return. However, this part of the cost does not account for a large proportion for search engines."

“The biggest difficulty with this technology is that the changes of most web pages are difficult to accurately predict, and a reliable crawling strategy is needed to keep the data up to date and ensure the accuracy of the links and generated indexes.”

"But fortunately, we have made some breakthroughs in this regard. Of course, the specific algorithms are confidential, so I won't introduce them to you two."

"Because of the cost savings in all aspects, I can maintain the normal operation of this search engine even if YSearch does not go public."

Lei Jun looked at Fang Yu's phone screen as if he were looking at an alien: "You mean, YSearch is a big model disguised as a search engine?"

In just a few months, AI has revolutionized the search engine industry?
What kind of evolution speed is this!?
Is it possible?

If this is true, which industry will be disrupted next?
Lei Jun suddenly felt somewhat fortunate that his Xiaomi chose to start a hardware business and could become a carrier of AI.

If you had chosen to enter the field of mobile Internet software innovation, you would probably be worried and unable to sleep now, right?
Fang Yu immediately corrected Lei Jun: "No, it can only be regarded as a search engine integrated with AI functions."

Too much is as bad as too little. Integrating AI into search engines is one thing, but it is another thing for the search engine itself to be a large AI model.

Currently, most people are still at the stage where they know about AI but have not yet experienced it personally.

At this time, if they find that the operating logic of the search function they use daily has fundamentally changed, they will inevitably become wary of AI.

By then, you never know what might happen.

Fang Yu said earnestly: "This involves technical information that has not been made public. I told Mr. Lei because I trust that he is not a gossiper. Please keep it confidential for me."

Lei Jun smiled bitterly. He now really believed that Fang Yu really did not want to list Ysou.

Under this model, the threshold for operating a search engine that covers the entire Internet is greatly lowered. Even a startup company that has just entered the unicorn stage, such as Youzi Technology, can enter this field.

No, it cannot be considered as being lowered. Being able to build and pre-train such a large model is itself a threshold.

Especially the algorithms mentioned by Fang Yu, they are feasible in theory, but only in theory.

If these algorithms are so easy to make, what would be the point of Qianxun and Gugou? These two companies would have been overthrown long ago.

But it was actually developed by a small company like Youzi Technology!

Turning around to look at Zhou Shuzi again, Lei Jun saw an eagerness and anticipation in his little brother's eyes that he had never seen before.

Lei Jun sighed in his heart, but he didn't blame Zhou Shuzi.

It is impossible for anyone not to be tempted by this vision that completely subverts the future.

"Xiao Fang, if that's the case, then it doesn't have to be that way, right? If you don't go public, Shu Zi won't be able to use his talents. Qianxun and Gugou should have many more suitable talents."

Silently, Lei Jun changed the way he addressed Fang Yu and touched his pocket.

"By the way, I heard that Lu Qi from Weiruan has resigned now, and Qianxun is trying to contact him. If you contact him now, he should be very interested." "Qianxun's Yuan Shanjun and Liu Anlin are also said to be looking for opportunities outside. They are more familiar with the search engine business and are also the contributors to Qianxun's commercialization."

Yuan Shanjun? Liu Anlin? I forced these two guys to look for jobs. How could I possibly hire them?

Qianxun's technical staff is pretty good, but the management? Haha, forget it. If the top beam is not straight, the bottom beam will be crooked, and the road has gone astray a long time ago.

As for Lucy...

The operators of Pseudo Software Da Zhou love going to nightclubs too much and having sex with female colleagues too much, just like in the financial circle.

Although Lu Qi has been at the Pseudo-Soft headquarters, if he comes, there is no guarantee that he will not recruit a few senior executives from Pseudo-Soft Da Zhou.

The arrival of a few executives who like to have sex with female colleagues and go to nightclubs has set off a bad atmosphere.

I said that Chihiro is a crooked person, so I hope that in the end Ysou’s character will be worse than Chihiro’s.

Fang Yu is very dissatisfied with many professional managers in foreign companies.

These people claim to have an international perspective, but in fact they are just big talkers, working in a confined space, and maneuvering within the company's established structure. They rely on platform resources to do well and think that it is their own ability.

In fact, it's bullshit.

For a period of time, Fang Daqiang poached a lot of professional managers from several foreign companies. The salaries they received were basically double what they received in foreign companies, and some were tripled, and they were also given ample power.

As a result, after arriving, this group of people immediately started to form circles, exclude dissidents, and then started to make money.

It's not that foreign companies don't have strong people. These people's basic qualities and abilities are definitely much stronger than many professional managers in private companies, but that doesn't mean they can use these abilities in your company.

"If you think Qianxun's people are not good enough, you can also find someone from Google. Philip Schneider of Google is very good at operation management. I have met him before in Hamburg, Prussia."

Lei Jun looks like a nerd, but he actually has a very strong ability to read people's expressions. He vaguely sees that Fang Yu is not interested in these two people, and starts to recommend the vice president of Google.

Fang Yu smiled and handed Lei Jun another cigarette: "Boss Lei, Ysou doesn't recruit non-Zhou people for this position, but we don't plan to recruit Zhou people with a Great Zhou background either."

"To be honest, in addition to his outstanding abilities, Brother Shuzi's background is also a major reason why I wanted him to come to Ysou. Brother Shuzi, I have something to say, please forgive me if I have offended you."

After saying that, Fang Yu smiled apologetically at Zhou Shuzi.

Zhou Shuzi was a little confused.

Background? What background do I have? My wife does have some background, but it has nothing to do with IT.

An idea suddenly flashed through Lei Jun's mind: "You want to go out to sea!?"

Fang Yu snapped his fingers and laughed, "Bingo! As expected of Mr. Lei."

Lei Jun held the cigarette between two fingers and waved it. When the ash fell on his pants, he quickly brushed it off with his hand.

"No wonder you bought the why domain name after you got the Y domain name. It turns out you want to enter the international market."

Lei Jun sighed.

"If we are talking about going overseas, Shuzi is indeed a good candidate. His Lijiapo background is indeed suitable for developing the Southeast Asian and Bharat subcontinent markets."

Fang Yu smiled noncommittally and looked at Zhou Shuzi: "Brother Shuzi, how about it? Are you interested? At your level, I don't need to discuss any salary issues with you. What Mr. Lei can afford, I can afford too."

Zhou Shuzi was obviously very tempted, as this was a much more attractive job than operating a rice IPO!
If the rice is well-made, its market value will be around 100 billion yuan on the day of listing.

Moreover, as Sansang stops supplying rice, this year's rice production capacity issues and Mi 5's product strength issues will definitely cause rice sales to decline, and it will be hard to say what the valuation will be at that time.

But because of this, it would be a bit unfair to leave rice behind now.

If Mr. Lei disagrees and holds a grudge, it will be bad for his reputation.

Zhou Shuzi's eyes flashed and he looked at Lei Jun.

At the same time, Fang Yu also looked at Lei Jun who was supporting his chin with his wrist.

"Mr. Lei, the IPO is indeed very important to Dami, but this job is not something that only Brother Shuzi can do."

"As long as Dami can make a profit and show brand improvement and momentum to become the fourth pole in the mobile phone industry, there are plenty of professionals who can do this."

"I've said before that Mr. Lei is an entrepreneur and start-up that I have always admired. I don't want any grudges to exist between our cooperation, so I didn't communicate with Brother Shuzi in advance. Today, Brother Shuzi is a little embarrassed, and Mr. Lei is also a little embarrassed."

"How about this, Mr. Lei, I can make you a promise. In the future, when Youzi Technology cooperates with any other mobile phone brand on AI systematization, the price I give them will be 30%-50% higher than yours. We can sign a minimum price agreement, which will be valid for five years."

!!!

Lei Jun's body trembled, and he wanted to say something but didn't.

Fang Yu smiled knowingly: "Boss Lei, you can discuss it with Brother Shuzi. I will go back today. Brother Shuzi, if you have considered it, call me back. I will go to pay the bill first."

Fang Yu picked up his phone, stood up, turned around and was about to pay the bill, but suddenly he remembered something and slapped his forehead.

"Mr. Lei, have you decided on the spokesperson for the Mi Mix and Note 10 that you are going to release in October? Can you please give me a favor?"

As a core partner of Da Mi, Fang Yu certainly knows Da Mi’s product planning for the second half of the year.

Lei Jun was stunned. This was a matter for the brand strategy department. He had just listened to Li Wanqiang's report and had some impression of it.

"Note2 is mainly for business, and they are in contact with Liang Chaowei. Mix seems to want to find that guy, the one who just came back from Goryeo, he's quite handsome, Wu..."

"Mei Yeping." Zhou Shuzi reminded from the side.

Lei Jun patted his forehead and said self-deprecatingly: "Look at my memory, yes, that's right, it's him. He said that he has a lot of traffic now, young people like him very much, and he can help with the black technology settings of Mix."

Mei Yeping?
"Boss Lei, can you give Mix to Yang Mi and Note2 to Repa?"

When negotiating for Da Mimi, don’t forget Repa either. We need to treat everyone equally.

Fang Yu didn't say anything like: It's okay if we can't change the spokesperson, I'm just asking for help, these nonsense words.

For people of Fang Yu and Lei Jun's level, this kind of thing is not important at all, it's just a matter of a word.

It just depends on whether you are willing to say this.

Moreover, for Dami, it doesn’t matter who is chosen as the spokesperson.

Those who buy rice are looking for the price-performance ratio or the fans. To put it bluntly, the basic group is losers. Who would buy rice if they are a fan of a star?

I don't know who chose Mei Yeping. All the people who like him are women. If you choose him as your spokesperson, women will probably not buy your phone.

Your base of customers is young male losers. Choosing a beautiful woman as your spokesperson can at least give your customers something pleasing to the eye.

Choose Mei Yeping. Few men don’t hate him, and the number of people losing their core base is greater than the traffic he brings.

It would be great to choose Da Mi Mi, the main feature of your mix is ​​Yamato's black technology.

It’s okay for Mimi to be big, and there are a lot of black technologies on her face, which is more in line with the brand tone.

Sure enough, Lei Jun didn't take it seriously: "There's nothing wrong with Mix. The contract probably hasn't been signed yet. But doesn't this hot product you mentioned match the business tone of Note2?"

What kind of business tone does Note2 have? Who would use it for business purposes now?
Isn't this just throwing a wink at a blind man?

Besides, Liang Chaowei has no appeal among male customers, and men don’t think he has much business sense.

I guess it was done by the female fans of the brand department again.

If you really want to focus on business, you might as well find a few boss fans who bought your phone to endorse it. Although Dami doesn't have a business tone now, it has such a large user base that it is easy to find a few senior professional managers or private company bosses as fans.

If it doesn’t work, you can also get a few of your big friends to be spokespeople.

He Xiaopeng, the former boss of UC who is planning to build a car, Da Qiangzi, the husband of milk tea, Chen Nian, the boss of Fanke, and yourself, several big bosses are holding Note2, showing their side faces, with backlight, and as the lights move, the camera follows until the lens focuses on the bosses' pretentious postures and the Note2 in their hands.

The voice-over is a deep baritone, "Life is about breaking through your limits again and again. Xiaomi Note2, break through your limits and achieve yourself!"

Then, from time to time, take some street or daily photos of the bosses using Note2 to generate some hot searches.

Isn’t this better than hiring Liang Chaowei?
Lei Jun just thought about it for a while and said, "How about this, Redmi has three spokespersons, Wu Xiubo, Liu Shishi, and a young man who has become quite famous recently. I will replace one of them with the hot guy you mentioned."

Fang Yu smiled and said, "Thank you, Mr. Lei."

Note 1: What we learn is the web page metadata, not the web page content, so it does not contradict the data scarcity problem in the data crisis mentioned in the previous chapters.

To put it simply, using a book as an analogy, the book title is stored in the server, and then the big model learns the table of contents and at most a summary.

This technical idea is my original one. I checked the papers and found no relevant ones.



(End of this chapter)