Google questions about crawled pages

Discussion in 'Search Engine Optimization' started by thetechguy, Feb 8, 2011.

0
  1. #1
    Hello,

    I need some help from you all Google crawl my entire magento website the following folder on my website was crawl and showing up on google

    1. /customer/
    2. /wishlist/
    3 /sales/

    How is the best way remove these urls?


    Thanks in advance!

    WM
     
    thetechguy, Feb 8, 2011 IP
  2. Kiranraj

    Kiranraj Well-Known Member

    Messages:
    1,583
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    128
    #2
    create a robot.txt file with restrict that url's so that Google can't crawl that url's...
     
    Kiranraj, Feb 8, 2011 IP
    aira likes this.
  3. aira

    aira Active Member

    Messages:
    1,175
    Likes Received:
    7
    Best Answers:
    0
    Trophy Points:
    78
    #3
    Kiranraj's right. Create a robot.txt file and indicate the files or directories that you want SE's crawler to ignore
     
    aira, Feb 9, 2011 IP