Robots.txt mistake now can't get indexed

Discussion in 'Search Engine Optimization' started by glorybeme, Aug 30, 2009.

  1. #1
    I am new to wordpress, still learning. I downloaded a theme, added some content and thought the site was running okay. I did a press release, articles, and started building backlinks. I kept checking google and I wasn't indexed and couldn't understand why so I did some checking and somehow the robots.txt was blocking the site, so I changed it and made a google xml sitemap went into google webmaster tools and submitted it.

    Now when I look google says success for accessing the robots.txt and shows 13 urls but there is no site: or cache: in google and it has been almost a week. Do I just need to wait it out? Will I eventually get indexed? Any other suggestions?

    This is for the tanning lotion site in my sig.
     
    glorybeme, Aug 30, 2009 IP
  2. shworth

    shworth Peon

    Messages:
    75
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #2
    Your robots.txt doesn't look right, try:

    instead of

    I'm not sure if it make a difference but never seen a robots.txt with just the one line before.
     
    shworth, Aug 30, 2009 IP
  3. glorybeme

    glorybeme Well-Known Member

    Messages:
    814
    Likes Received:
    31
    Best Answers:
    0
    Trophy Points:
    138
    #3
    Thanks I fixed it. I had it like that and I just removed the disallow all together since the I had disallow /
     
    glorybeme, Aug 30, 2009 IP
  4. Natheen2000

    Natheen2000 Peon

    Messages:
    154
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    0
    #4
    Try to get more backlinks, specially from here DP :p
    As well as get some more bookmarks. You will get index soon, However, when I search for site:bestratedtanninglotions.com, I have got one result.... So, get more backlinks.
     
    Natheen2000, Aug 30, 2009 IP
  5. farangkwaiyai

    farangkwaiyai Guest

    Messages:
    317
    Likes Received:
    4
    Best Answers:
    0
    Trophy Points:
    0
    #5
    If you used a robot.txt to tell google not to index your site IMO it will obey this for 90 days and come back.

    Just get more links in to each page of your site.
     
    farangkwaiyai, Aug 30, 2009 IP
  6. glorybeme

    glorybeme Well-Known Member

    Messages:
    814
    Likes Received:
    31
    Best Answers:
    0
    Trophy Points:
    138
    #6
    If you do site:bestratedtanninglotions.com and click contain the term you can see that I have submitted articles, a press release et.

    Of course I will still continue to build links. 90 days is a long time to wait. Oh well at least I learned a new lesson. Thanks to all for the help
     
    glorybeme, Aug 30, 2009 IP
  7. jitendraag

    jitendraag Notable Member

    Messages:
    3,982
    Likes Received:
    324
    Best Answers:
    1
    Trophy Points:
    270
    #7
    Wordpress blogs have a privacy setting where it will generate a robots.txt with disallow and also use meta noindex attribute. Just clearing the robots.txt may not work sometimes.
     
    jitendraag, Aug 30, 2009 IP
  8. farangkwaiyai

    farangkwaiyai Guest

    Messages:
    317
    Likes Received:
    4
    Best Answers:
    0
    Trophy Points:
    0
    #8
    ah good point,

    You have this in your code

    <meta name='robots' content='noindex,nofollow' />

    Thats the problem.
     
    farangkwaiyai, Aug 30, 2009 IP
  9. sydzapp

    sydzapp Well-Known Member

    Messages:
    703
    Likes Received:
    12
    Best Answers:
    0
    Trophy Points:
    100
    #9
    sydzapp, Aug 30, 2009 IP
  10. glorybeme

    glorybeme Well-Known Member

    Messages:
    814
    Likes Received:
    31
    Best Answers:
    0
    Trophy Points:
    138
    #10
    How do I fix that? Thank you so much for catching that!
     
    glorybeme, Aug 30, 2009 IP
  11. glorybeme

    glorybeme Well-Known Member

    Messages:
    814
    Likes Received:
    31
    Best Answers:
    0
    Trophy Points:
    138
    #11
    Found it under privacy settings in my dashboard. It said block search engines. Again thanks for catching that!
     
    glorybeme, Aug 30, 2009 IP