Google Webmster can't crawl the site

Discussion in 'Site & Server Administration' started by longhand, Apr 22, 2007.

  1. #1
    Hi members!

    I've added to the Google Webmaster last night my new site but does it says:
    We can't currently access your home page because of an unreachable error.

    I can't decide if is this an temporal error at Google Webmaster side (10-12 hours ago?) or something wrong with my configuration in the hosting account.
    The site is on a HostGator shared hosting as a subdomain (addon).
    http://coupon-sites.info
    Joomla website.

    Its .htaccess file is the following:
    DirectoryIndex index.php
    RewriteEngine On
    RewriteBase /

    RewriteCond %{REQUEST_URI} ^(/component/option,com) [NC,OR]
    RewriteCond %{REQUEST_URI} (/|\.htm|\.php|\.html|/[^.]*)$ [NC]
    RewriteCond %{REQUEST_FILENAME} !-f
    RewriteCond %{REQUEST_FILENAME} !-d
    RewriteRule (.*) index.php


    Robots.txt file:
    User-agent: *
    Disallow: /administrator/
    Disallow: /cache/
    Disallow: /components/
    Disallow: /editor/
    Disallow: /help/
    Disallow: /images/
    Disallow: /includes/
    Disallow: /language/
    Disallow: /mambots/
    Disallow: /media/
    Disallow: /modules/
    Disallow: /templates/
    Disallow: /installation/


    In the root of account is installed PHPLinkDirectory can its co nfiguration cause the error?

    Many thanks in advance for your tips and helps!
     
    longhand, Apr 22, 2007 IP
  2. longhand

    longhand Peon

    Messages:
    255
    Likes Received:
    4
    Best Answers:
    0
    Trophy Points:
    0
    #2
    ps.:site verification went ok also adding to G analitycs.
     
    longhand, Apr 22, 2007 IP
  3. Monty

    Monty Peon

    Messages:
    1,363
    Likes Received:
    132
    Best Answers:
    0
    Trophy Points:
    0
    #3
    I don't see anything wrong that could stop GoogleBot.

    Did you choose a "prefered domain" in GWT?
    Because your .htaccess redirect to the non-www version. So if you have choosen the www version as prefered domain, that's might create a conflict.

    Just an hypothesis. But actually, I don't see why because the www version redirect properly to the non-www, maybe a temporary bug in GWT
     
    Monty, Apr 22, 2007 IP
  4. longhand

    longhand Peon

    Messages:
    255
    Likes Received:
    4
    Best Answers:
    0
    Trophy Points:
    0
    #4
    Thanks for Your reponse Monty.
    No, didn't choosen yet to any format of preferred domain.
     
    longhand, Apr 22, 2007 IP
  5. longhand

    longhand Peon

    Messages:
    255
    Likes Received:
    4
    Best Answers:
    0
    Trophy Points:
    0
    #5
    in the root PhpLinkDirecoty's .htaccess can cause problems?
     
    longhand, Apr 22, 2007 IP
  6. longhand

    longhand Peon

    Messages:
    255
    Likes Received:
    4
    Best Answers:
    0
    Trophy Points:
    0
    #6
    I paste here .htaccess from the root of public_html (the website is in a subfolder)

    # Prevent .htaccess and .htpasswd files from being viewed by web clients
    <Files "^\.ht">
    Order allow,deny
    Deny from all
    </Files>

    # Protect files
    <Files ~ "^(.*)\.(inc|inc\.php|tpl|sql)$">
    Order deny,allow
    Deny from all
    </Files>

    # Protect directories
    <Files ~ "^(backup|files|images|include|lang|libs(/.+)?|temp(/.+)?|templates(/.+)?|javascripts(/.+)?)$">
    Order deny,allow
    Deny from all
    </Files>

    # Disable directory browsing
    Options -Indexes

    # Follow symbolic links in this directory
    Options +FollowSymLinks

    # Override PHP settings that cannot be changed at runtime
    # (If your server supports PHP settings via htaccess you can comment following two lines off)
    # php_value register_globals 0
    # php_value session.auto_start 0

    # Customized error messages
    # ( If you are running in a subfolder please add it, example: #"directory/index.php?httpstatus=404" )
    ErrorDocument 404 index.php?httpstatus=404

    # Set the default handler
    DirectoryIndex index.php

    # URL rewrite rules
    <IfModule mod_rewrite.c>
    RewriteEngine On

    ## Details Link Page Rewrite##
    RewriteCond %{REQUEST_FILENAME} !-f
    RewriteCond %{REQUEST_FILENAME} !-d
    RewriteCond %{REQUEST_FILENAME} !-l
    RewriteRule (.*)detail/link-(.*).htm[l]?$ detail.php [QSA,NC]

    ## Pagination Rewrite
    RewriteCond %{REQUEST_FILENAME} !-f
    RewriteCond %{REQUEST_FILENAME} !-d
    RewriteCond %{REQUEST_FILENAME} !-l
    RewriteRule (.*)page-(\d+)\.htm[l]?(.*)$ $1/?p=$2 [PT,NC]

    ## Category redirect
    RewriteCond %{REQUEST_FILENAME} !-f
    RewriteCond %{REQUEST_FILENAME} !-d
    RewriteCond %{REQUEST_FILENAME} !-l
    RewriteRule ^(.*)$ index.php [QSA,L]

    </IfModule>
     
    longhand, Apr 22, 2007 IP
  7. longhand

    longhand Peon

    Messages:
    255
    Likes Received:
    4
    Best Answers:
    0
    Trophy Points:
    0
    #7
    weehhh.
    Submitted sitemap.xml

    Network unreachable: DNS issue
    We encountered an error while trying to access your Sitemap. Please ensure your Sitemap follows our guidelines and can be accessed at the location you provided and then resubmit. [?]
     
    longhand, Apr 22, 2007 IP