WebSite X5Help Center

 
Terry  R.
Terry R.
User

Crawl errors  en

Author: Terry R.
Visited 730, Followers 1, Shared 0  

Hi.

I have built a website with x5 11. www.discountelements.com When I go into google webmaster and click on crawl errors. It brings this up. 1. downloads/ga-001-web.html, 2. products/indium/index.html, 3.products/index.html, 4.products/gallium/index.html, 5.products/lithium/index.html. Code 404 for all of them. but these urls are not in my website. What is it?

Regards.

Posted on the
6 ANSWERS
Incomedia
Claudio D.
Incomedia

Hello Terry,

In your sitemap file created by the program these pages are not present. Can it be these pages existed before and where submitted with another sitemap file?

If this is the case remove the other sitemap file you added in google webmaster and leave only the one created by WebSite X5.

Many thanks!

Read more
Posted on the from Claudio D.
Terry  R.
Terry R.
User
Author

Hi.

There is no other sitemap in my website files. The sitemap was created with x5 11 and uploaded by x5 11. I don't understand it.

Regards.

Read more
Posted on the from Terry R.
Paul M.
Paul M.
Moderator

Hello Terry,

Can you post a screenshot of what you see in Google Webmaster Tools?  It would be helpful to have more information.

What do you have in your robots.txt file?

Kind regards,

Paul

Read more
Posted on the from Paul M.
Paul M.
Paul M.
Moderator

Mystery solved...  your domain used to belong to someone else...  see the following link for the proof:

https://web.archive.org/web/20110202231428/http://discountelements.com/

You need to tidy up the robots.txt file to prevent the Googlebot trying to crawl these old pages which no longer exist.

This issue is not connected with WebSite X5 in any way.

Kind regards,

Paul

Read more
Posted on the from Paul M.
Terry  R.
Terry R.
User
Author

You know, that's exactly what I thought it might be, that someone else owned it.

I have attached the robots text file. I don't know how to resolve this issue.

Thanks.

Read more
Posted on the from Terry R.
Paul M.
Paul M.
Moderator

You need to append the following lines to the end of the 'robots.txt' file, Terry:

Disallow: /downloads/ga-001-web.html
Disallow: /products/indium/index.html
Disallow: /products/index.html
Disallow: /products/gallium/index.html
Disallow: /products/lithium/index.html

or a shorter, simpler version would be this:

Disallow: /downloads
Disallow: /products

The disadvantage of the shorter version is that it effectively prohibits you from using sub-folders called 'downloads' and 'products' in your X5 project, which might be a problem for an online shop!

Reupload the modified 'robots.txt' file to your server, and the next time the Googlebot comes around he won't try to follow those dead links  

Kind regards,

Paul

Read more
Posted on the from Paul M.