My content but my website is not coming up in search results because of another one

Discussion in 'Search Engine Optimization' started by thegunninghawk, Oct 11, 2009.

  1. #1
    Good morning friends.

    When I publish articles in my blog and I go in Google to see if they come up(by copying a part of the title and search for it), I notice that it does come up but under one of the many websites which spider my content on their news aggregator websites.

    How can I prevent this? To give you an example, search for "Kieran Gibbs scores at both ends" in the last 24 hours. The three results that come up are not my website.

    I have to specify my site to see the results:
    "Kieran Gibbs scores at both ends" site:thegunninghawk.com

    How come?

    Thank you.
     
    thegunninghawk, Oct 11, 2009 IP
  2. Dan Schulz

    Dan Schulz Peon

    Messages:
    6,032
    Likes Received:
    437
    Best Answers:
    0
    Trophy Points:
    0
    #2
    Find out what the spiders' names are and add them to your robots.txt file as separate Disallow: directives.
     
    Dan Schulz, Oct 11, 2009 IP
  3. thegunninghawk

    thegunninghawk Peon

    Messages:
    37
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #3
    Thanks for the reply but although I do have a knowledge of the robots.txt file, I'm not quite sure what you mean by "spiders' names".

    Is that something I need to find in the three websites coming up in the search results?

    Thanks again.
     
    thegunninghawk, Oct 11, 2009 IP
  4. Dan Schulz

    Dan Schulz Peon

    Messages:
    6,032
    Likes Received:
    437
    Best Answers:
    0
    Trophy Points:
    0
    #4
    They'll be in your server logs under robots.

    Let's say for example that Digital Point had its own search engine that was crawling your Web site named dpbot. In your server logs you'd look for the section that says what bots have been crawling your site. To block this (obviously fake) bot, look for dpbot in there and add the name dpbot to your robots.txt file.
     
    Dan Schulz, Oct 11, 2009 IP
  5. thegunninghawk

    thegunninghawk Peon

    Messages:
    37
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #5
    I think I understand.

    So these websites, which are taking my content from my RSS feed, should be in my server logs and I should be able to identify them, correct? You know what the problem could be ... my RSS feed is with feedburner, NOT on my server so I'm afraid I'll find no trace of these websites :(

    Just for the record, I gave permissions to these websites to take content from my RSS feed, I just never knew they would be preferred by Google, that's all.

    Thank you once more.
     
    thegunninghawk, Oct 11, 2009 IP
  6. Dan Schulz

    Dan Schulz Peon

    Messages:
    6,032
    Likes Received:
    437
    Best Answers:
    0
    Trophy Points:
    0
    #6
    Oh. You gave them permission. Then add a link back to the original blog entry to your feeds. I'm sure there's a plugin that does that.
     
    Dan Schulz, Oct 11, 2009 IP
  7. thegunninghawk

    thegunninghawk Peon

    Messages:
    37
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #7
    To be fair, they provide a link back to my blog, a "read the entire story" type of thing but I'm very confused as to what is beneficial to me. I mean, at the moment people search terms, end up in these websites but not sure they actually click this link. I prefer if they fall of mine straight away, without any intermediate levels, especially since I have CPM banners on my blog.

    On the other hand, if I ask these websites to stop crawling me, I will lose backlinks and maybe - in terms of Google - I would end up damaged the same way. Having said that, I noticed that feedburner masks my links with http://feedproxy.google.com ... etc. so I believe the first thing to do is to actually how to instruct feedburner to show my domain link and not mask it via http://feedproxy.google.com

    :)
     
    thegunninghawk, Oct 11, 2009 IP
  8. newlogo

    newlogo Peon

    Messages:
    3,931
    Likes Received:
    11
    Best Answers:
    0
    Trophy Points:
    0
    #8
    u need to get xml sitemap and interlink and high pr back link
     
    newlogo, Oct 12, 2009 IP
  9. dsouza

    dsouza Peon

    Messages:
    335
    Likes Received:
    12
    Best Answers:
    0
    Trophy Points:
    0
    #9
    Put some extra effort, to bring the keywords of your site in serp.
     
    dsouza, Oct 12, 2009 IP