Online spiders dont crawl my wordpress

Discussion in 'Search Engine Optimization' started by maestria, Dec 7, 2014.

0
  1. #1
    Hi,

    I have a kb on my site like sitename/kb/
    But the online spider simulators dont consider the inside links(article lists) at all.
    It merely consider sitename/kb/ and not sitename/kb/article.php and so on.
    I have disabled the "Search Engine Visibility" checkbox.
    Any other thoughts?
     
    maestria, Dec 7, 2014 IP
  2. a-jani

    a-jani Banned

    Messages:
    60
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    43
    #2
    First search this in google search if the url does show in the search results.
    site:www(dot)sitename/kb/article.php
    If it is indexed in Google search then the spider will accept it, in the next crawl.
    If its not indexed then submit it to index manually in Google webmaster tool and manually crawl it.
    Update and resubmitted a XML sitemap to google through webmaster tools.
     
    Last edited: Dec 8, 2014
    a-jani, Dec 8, 2014 IP
  3. patco

    patco Well-Known Member

    Messages:
    2,035
    Likes Received:
    47
    Best Answers:
    17
    Trophy Points:
    100
    #3
    Do you have a link to the article page from your homepage (or any other inner page of your website that is already indexed)?
     
    patco, Dec 9, 2014 IP
  4. mdkreischer

    mdkreischer Active Member

    Messages:
    127
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    98
    #4
    You have to make sure you do not have an orphaned link to start.. If it is not indexed then it will not be crawled and you should make new page and start over...IMO
     
    mdkreischer, Dec 10, 2014 IP
  5. WebLab

    WebLab Active Member

    Messages:
    229
    Likes Received:
    7
    Best Answers:
    2
    Trophy Points:
    65
    #5
    So you want 'online spider simulators' to find it and not search engine ? is that ?
     
    WebLab, Dec 10, 2014 IP