How to get my page crawled

[rhyswynne]

Hi everybody

One of my pages hasn't been crawled for at least a month now, and I've added a lot of content to it. It has a few promenant links on it (in my blog, on a few websites i own, squidoo lenses etc.), but no indexion. I think it is not indexed because my sitemap and .htaccess conflicted with each other, but would like to get the bots to pass over the site just to make sure.

But they haven't crawled it (according to Google Webmaster Tools).

Is that the figure in GWT the last time google CRAWLED (i.e. successfully passed over the site) or VISITED the site? Or does anybody know for sure?

 

[OldWelshGuy]

google spiders far more pages than it adds to the index. have you submitted a new sitemap? Are you positive that there is nothing blocking the page from being spidered?

 

[rhyswynne]

google spiders far more pages than it adds to the index. have you submitted a new sitemap? Are you positive that there is nothing blocking the page from being spidered?

The sitemap is being generated by a script linked to the site, will try another format. I have deleted the .htaccess and the robots.txt and the site still works, so I'm a bit confused by it.

It just the "last crawled on the Apr 26th 2008" line that's worrying me. I could be in the sandbox (it is an affiliate site, but with good information, certainly a lot more content compared to a lot of my other sites!), but I just don't think it's getting crawled.

Will continue to look into it, thanks OWG

 

[jitendraag]

Check your robots.txt too.

If nothing works try to put that page in forum signature and it will get indexed quickly

 

[OldWelshGuy]

The sitemap is being generated by a script linked to the site, will try another format. I have deleted the .htaccess and the robots.txt and the site still works, so I'm a bit confused by it.

It just the "last crawled on the Apr 26th 2008" line that's worrying me. I could be in the sandbox (it is an affiliate site, but with good information, certainly a lot more content compared to a lot of my other sites!), but I just don't think it's getting crawled.

Will continue to look into it, thanks OWG

he said he has deleted the robots TXT already, so that can't be it.

have a look at the page load times of the home page, make sure your internal links are pointing to domain/ make sure that your index page is returning a 200OK header response, and also that domain.com/ is doing the same.

it could be a server setup thing

 

[zurk]

Link building might help

 

[Garcia Coder]

well as for the knowledge I have I can say that link building.

 

[bochgoch]

What page is it? What links do you have to the page?

...I've had problems with pages that are linked to using PHP parameters ... i.e http://www .vvvvvv.com?P=page1 - G doesn't like ?'s in URL's

By the way Dating Site For Bloggers :wub: in your sig!!!! That sounds like a scary site :scared: ... bloggers shouldn't be allowed to procreate!

 

[pow-wow]

I suggest you build more backlinks on dofollow sites/blogs/forums/etc. If the backlinks are from frequently crawled sites - the bots will find your backlinks and crawl your site as well.

 

[rhyswynne]

By the way Dating Site For Bloggers :wub: in your sig!!!! That sounds like a scary site :scared: ... bloggers shouldn't be allowed to procreate!

I know , but I know xhan is on the site, which some of you lot find attractive

I'm pretty sure it's not the .htaccess file. All there in it is URL redirects. I'm checking a few things around (I've set it not to set a preferred domain to hopefully not get anything, and I haven't set a geographic target).

Anyway, the page in question is Bowling Balls, Shoes & Accessories from Ultrabowling.info. Is there a tool that mimics the googlebot to check how it crawls, as I'm getting completely clueless about it

 

[OldWelshGuy]

I can't se ANY pages spidered on that site in the google index, am I missing something here?

 

[rhyswynne]

I can't se ANY pages spidered on that site in the google index, am I missing something here?

That's the issue. No pages spidered, googlebot hasn't crawled it in nearly 2 months, and I'm clueless why. I have had sandboxed pages in the past, and even they are crawled - just not included.

If anybody has any suggestions, am I being really thick, or have I done something wrong?

 

[OldWelshGuy]

My bad, I thought you were saying a specific page hadn't been spidered.


Is the site hosted on the same server as any of your other sites, and are those spidered & indexed? If not then could be a server Ip blacklisting (Google will dump all the sites on a known spam address.)

 

[rhyswynne]

It's hosted on my reseller account with a number of my sites (such as The Gospel According To Rhys and Retro Garden), and they are getting listed fine, it's just this one that appears to not get listed .

 

[DJDave]

Officially this isnt true, but I believe I have seen it work a number of times....set up an Adwords campaign, and link it to the page you want crawled...and wait.

I've seen my target pages crawled 3 times a week after setting up an Adwords campaign, after a period of not being crawled for weeks!

give it a go!


------------------
Dave

 

[MakeMoneyOnline]

You should build about 20 blog comment links for each page of content that you produce. Try to use blogs that you do not own to promote your website. Also try to post comments in blogs that have the highest Page Rank and check and make sure the blog is do-follow and not no-follow variety. I like doing a keyword search for blogs that have similar content value to drop the link. I feel this will not only creat the back link but may create traffic.

Regards, Gregory Atlas Johnson from Homebizseo.com

 

[johnafrankllin]

Here is some information for you:

How to Get More Pages into Google's Index - Search Engine Watch

 

[markcijo]

back linking can help you crawl that page