Vbulletin 4 robots txt?

Discussion in 'robots.txt' started by rants1990, Sep 19, 2017.

0
  1. #1
    Hello guys i using vbulletin 4 so i asking for details what i have to write there to allow search engine index all my content in site??
     
    Solved! View solution.
    rants1990, Sep 19, 2017 IP
  2. #2
    Robots.txt is more about requesting bots stay away, they'll come anyway.

    If you specifically want to summon the bots look at Google's xml feed that you can submit into their webmaster tools. Bing does one too.

    https://support.google.com/merchants/answer/7503577
     
    sarahk, Sep 19, 2017 IP
  3. rants1990

    rants1990 Greenhorn

    Messages:
    14
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    11
    #3
    so that means if i dont have robots.txt bots will be index my posts ??
     
    rants1990, Sep 19, 2017 IP
  4. sarahk

    sarahk iTamer Staff

    Messages:
    28,797
    Likes Received:
    4,531
    Best Answers:
    123
    Trophy Points:
    665
    #4
    Yes, once they find your site they'll index it. That doesn't mean they'll add it to the search engine results but they'll crawl all of your site. You may want them to return more quickly which is what the feed is all about.
     
    sarahk, Sep 19, 2017 IP