1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

Google crawl errors

Discussion in 'Google' started by gift-online, Oct 8, 2013.

0
  1. #1
    Hi DP members.

    I found crawl errors on my site in this format
    node/21
    node/25
    etc

    I suppose those pages were created without url and were indexed by Goolge.
    What is your opinion?
    How can I fix this problem?

    Thanks
     
    gift-online, Oct 8, 2013 IP
  2. bitronictech

    bitronictech Active Member

    Messages:
    74
    Likes Received:
    8
    Best Answers:
    0
    Trophy Points:
    58
    #2
    You can use robots.txt to block the indexing of files and folders you don't want indexed by search engines.

    - Quoted from robotstxt.org

    So if you wanted to block tose folders completely from public access you would be better off using .htaccess rules.
     
    bitronictech, Oct 16, 2013 IP
  3. almyty

    almyty Greenhorn

    Messages:
    64
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    6
    #3
    yeah use robots, to block in future, you can also use 301 redirection to proper pages/urls
     
    almyty, Oct 17, 2013 IP