below is my robots it is good #website User-agent: * Disallow: /cgi-bin/ Disallow: /tmp/ Disallow: website/forums/members/list/ User-agent: * Allow: website/dbseo_sitemap_index.xml User-agent: NinjaBot Allow: / User-Agent: bingbot Allow: / User-agent: bing Allow: / User-agent: Googlebot-Image Allow: /gallery User-agent: adsbot-Google Allow: / Website is actually domain