I just found weird problem with Google sitemap. They crawl my site but not index them. I already had links linked to them but google still not cache. After I analyzer robot.txt, they said my title tag has a problem because "Syntax not understood". What happened and how can I solve this one?
Well, <title> tag is not part of robots.txt Can we see links to your: * website * robots.txt * sitemap.xml (if you have one?)
Thanks, I have fixed it. It's my fault to misunderstood about statement. I dunno what happened but my robots.txt has command to disallow bot to crawl my <title> tag. I am sure that I didn't add that weird for sure so I just upload my empty robots.txt to replace the old one and my site got indexed now.