Baidu Spider only crawls the home page of the website but not the inner pages. What's going on? Many people struggle with this issue, especially when a new website is launched and the number of entries has not increased for a long time, which makes them even more anxious.
First of all, let’s solve a problem. How do we know that “Baidu Spider only crawls the home page of the website but not the inner pages”?
The spider crawling situation can be viewed through the website IIS log. The log clearly records the spider's crawling time, crawled pages, crawled pages and other information.
What a horse? Can't read IIS logs?
There are many IIS log analysis tools on the Internet, check them out on Baidu. It is recommended to use the Lightyear IIS log analysis tool here, which is fast and easy to use.
Next, Nantang will analyze for you the problem of "Baidu spider only crawls the home page but not the inner pages".
The reasons for "Baidu Spider only crawls the home page of the website but not the inner pages" are as follows:
1. Robots.txt, incorrect operation, blocked inner pages.
2. Mass messaging and other cheating behaviors.
3. Server reasons.
4. There are too few links from the homepage to the inner pages of the website, the navigation structure is confusing, and the navigation is unclear.
5. The quality of the website is too bad and the weight is extremely low.
6. When maintaining the website, we spend three days fishing and two days drying the website.
7. The website is super new, and the new one is a mess.
8. The website is still in Baidu's sandbox. Baidu spiders crawled and crawled it, but did not put a snapshot of the page.
The fact that the homepage can be crawled by Baidu spiders shows that the website is not something that spiders are dismissive of.
That must be one or N of the above 7 reasons, which leads to "Baidu spider only crawls the home page of the website but not the inner pages."
This concludes the discussion. Finally, let’s break down the 8 reasons why “Baidu Spider only crawls the home page of the website but not the inner pages” above.
1-8 Check the websites in turn, exclude them one by one, and solve them one by one.
The eight major blasting plans are as follows:
1. Problem with robots.txt. Enter it in the browser and check it. It will be clear at a glance.
Breaking 2. As a website maintainer, you know this situation best. If there are mass SEO cheating techniques, it is very common for Baidu spiders to only crawl the home page of the website but not the inner pages.
Processing method:
Stop all SEO cheating methods that are sent in bulk.
Update the website content regularly and continuously. The website content should be original, and the secondary content should also be high-quality pseudo-original.
Make appropriate external links and friendly links to attract spiders.
In this case, you can only persist calmly and wait.
Break 3. You can query the "web page http status code" and analyze the code returned by the website to determine the cause of the problem.
There is also a situation where a website with the same IP server as yours is punished by Baidu, thus implicating your website.
Or you can calmly persist and wait. Or change the server.
Break 4. Organize the website navigation, straighten out the structure, and make the website navigation clearer.
Call the article title to the home page, add a "Latest Article", "Article Recommendation" and other sections, and add internal page entrances.
Breaking 5. The articles on the site are so rubbish that Baidu spiders are too lazy to crawl them. How do you still pray that the spiders will finish crawling the home page and crawl the inner pages?
Organize and modify the junk content that has been included by Baidu, and clean up the content that has not been included.
Then insist on updating website content with quality and quantity, and at the same time do a good job of off-site links.
Breaking 6, I update one article at the beginning of the month and one at the end of the month. I complain every day why "Baidu spiders only crawl the homepage of the website and not the inner pages."
Save your energy and update a few articles.
Baidu Spider is like a human being. It originally comes once a day, but your website is not updated. Then I come once every two days, and your website is still not updated. Then there are 5, 10, 15.
Baidu Spider also has its own behavioral habits. Although its intelligence is very low, it cannot be ignored.
Break 7. As a new website, keep a low profile and don’t struggle with the problem of “Baidu spiders only crawl the homepage of the website but not the inner pages” every day.
Develop your own website optimization plan, maintain and update the managed website, and avoid using any cheating techniques.
8. This reason is difficult to determine. If your website does not exist for the above reasons, and your website is also "Baidu spider only crawls the homepage of the website but not the inner pages."
Then, it may be that we are still in Baidu's sandbox.
This is Baidu’s testing period for new websites, or the observation period after a website is punished.
Face it calmly and actively implement the determined website optimization plan.
That’s all for today, I hope Nantang’s article can help you.
This article comes from: Nantang's website optimization blog, address: http://www.ba77.com/post/15.html , please indicate when reprinting!
Thank you Nantang for your contribution