View Full Version : will this spider?
schlottke
Apr 15th 2004, 5:21 pm
http://www.wrestling-connection.com/ez-article/index.php
digitalpoint
Apr 15th 2004, 5:28 pm
No, you are using session IDs in the URLs.
- Shawn
schlottke
Apr 15th 2004, 5:47 pm
Im sorry for being such a novice.. since there is no way to just 'turn them off' how would i change this?
rfuess
Jun 11th 2004, 1:24 am
These dont look to be session id's, but rather database id's for different articles. To test this theory: open up a browser and click on one of the articles. Bookmark it. If you can get there after you close down all browser sessions, then it is not a session id.
They will spider in most spiders. Some don't accept anything with a querystring (index.htm?name=value) that would show up as having a questionmark.
Many do.
You are currently redirecting to the same page, http://www.wrestling-connection.com/ez-article/index.php, but with a different article id. Some scripting in the back end will decide what to display.
The most ideal for the largest scope of spiders is to have it go to the default page in a directory like so: http://www.yoursite.com/ or http://www.yoursite.com/coolarticle1/ which would go to default.htm or index.htm
This is because they will look at this as the beginning or top page of your site.
The next best thing is to get it in a page without url encoding. (No question marks, no ampersands, . . . )
- Robert Fuess
Spiderweb Logic
ProductivePC
Jun 11th 2004, 6:20 pm
Those pages have a page rank of 1 so they have been spidered. For anyone interested in getting a website with session ID's spidered either use a mod_rewrite command with a unix clone or "ISAPI REWRITE" on a windows box. This will rewrite the URL so that the spiders can spider all of yoru pages.
Hope it Helps
Wayne
schlottke
Jun 11th 2004, 6:30 pm
Seems the pages are being spidered- super.
Owlcroft
Jun 12th 2004, 5:01 pm
To test this theory: open up a browser and click on one of the articles. Bookmark it. If you can get there after you close down all browser sessions, then it is not a session id.
I have no expertise here, but it seems to me that I can recall closing down, say, an Amazon session, and the whole browser, then going back later and being able to re-open that page with the URL (with session ID) as saved in the clipboard. I'm not swearing that is so, but it seems to me that I so recall it.
For the test to be fully reliable, would not some length of time, anything from (I'm guessing) 90 minutes to 24 hours, have passed before trying the bookmark?
nlopes
Jun 14th 2004, 6:22 am
you should change the IDs from md5 hash to something better. Google may think that those MD5 hashes are Session IDs.
The better for rankings is: /ez-article/index.php?id=some_description
-or even better-
/ez-article/article_title (done using mod_rewrite)
Help Desk
Jun 14th 2004, 7:00 am
Is that id that is in the url just an article reference ID?
nlopes
Jun 14th 2004, 7:16 am
it seems to be!
vBulletin® v3.8.4, Copyright ©2000-2009, Jelsoft Enterprises Ltd.