Social Buttons
  1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

Is this wierd for spiders ..!

Discussion in 'Google Sitemaps' started by eliasmoosa, Apr 8, 2006.

0
  1. #1
    hi,

    When I send a spider to spider www.mydomain.com/folder it does that properly , when i SEND IT TO SPIDER THE SAME FOLDER , but through subdomain.mydomain.com which is a subdomain attached to that same folder , it can't see except the index page !! although the subdomain.mydomain.com works perfect in the browser , why is that ???
    SEMrush
     
    eliasmoosa, Apr 8, 2006 IP
    SEMrush
  2. Jean-Luc

    Jean-Luc Peon

    Messages:
    601
    Likes Received:
    30
    Best Answers:
    0
    Trophy Points:
    0
    #2
    Basically a spider sees the same page as a browser. Maybe with your URL, it would be easier to help.

    By the way, what do you mean by 'When I send a spider...' ? Did you write you own spider script ?

    Jean-Luc
     
    Jean-Luc, Apr 8, 2006 IP
  3. eliasmoosa

    eliasmoosa Guest

    Messages:
    27
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #3
    Well,
    No actually I meant these spiders that comes with some scripts , like site map creators and so on .. but still I mean why they read www.domain.com/A

    BUT

    Can't read www.A.domain.com they are both the same folder !!!
     
    eliasmoosa, Apr 8, 2006 IP
  4. softplus

    softplus Peon

    Messages:
    79
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    0
    #4
    Then your webserver is configured wrong :)

    And seriously, why would you want to publish the same content under multiple URLs? It's just asking for trouble... Don't do it! Pick one and stick to it.
     
    softplus, Apr 11, 2006 IP