No for me no visitors returned. I am asking so as to check if it is worth it wasting bandwidth on this bot sort of
Gigablast is worthless. It ate up gigabytes of bandwidth and brought nothing to my site. If you have a dynamic site, like a forum I would definitely block their bot and even their ip address for a while. Try including this in your robots.txt file: User-agent: Gigabot Disallow: /directory you want to deny/ You can also use metatags: <meta name=robots content=noindex> // Gigabot will not index page <meta name=gigabot content=noindex> // Gigabot will not index page <meta name=robots content=noarchive> // Gigabot will not cache page <meta name=gigabot content=noarchive> // Gigabot will not cache page <meta name=robots content=nofollow> // Gigabot will not follow links on page <meta name=gigabot content=nofollow> // Gigabot will not follow links on page Code (markup):
Seems these tags, in particular, would stop all spiders from indexing, caching, or following the page?
That's correct. This was from Gigablast's own website as to what tags they supposedly obey. You can just use this tag to just block GigaBalst. <meta name="gigabot" content="noindex,nofollow" /> Code (markup):
Gigabot seems more agressive than MSNbot. Makes me wonder if it's another organization, using the data some other way, or checking for cloaked pages.
I actually quite like the GigaBlast search engine. Not enough to use it I'm afraid, but hey I liked it Is this the chicken-and-the-egg question? Webmasters won't allow the bot because its a waste of bandwidth and they get no traffic from the search engine, then months later people try to use the search engine and theres no results because everyone blocked it But no, we get crawled quite a bit by Gigabot and we haven't seen any traffic to speak of.
i saw this spidering my forum the other day, do you have to have signed up somewhere? or does it just follow links?
I haven't seen GigaBlast spider my site at all...I tried submitting my site for inclusion in their search engine but its said for about a month now that the Add URL feature is disabled. Gigablast used to be in the news 2-4 years ago a lot...I thought it could be a strong competitor to Google but in time its popularity decreased I guess because of the results they were giving users.. You search for something on Gigablast and sometimes its irrelavent, or the description for the site is simply not there...it just lists keywords that you searched for from the site.
Gigablast also has a section or site for blogs, and if you submit there then the bot will spider your site. For me it picked up my forum and would not stop spidering it. This cost me too much bandwidth, so I had to shut them out for a while.
They do seem to have lost their spider-leash. They spider mine like crazy but I can't say I've seen any visitors. Maybe they are just warming up for something big later.
I've never submitted any of my sites to Gigablast, but they seem to be discovered and crawled from Wikipedia, DMOZ and other well-known "resource hubs".
I saw a small bug in GigaBot's method of crawling yesterday - it was treating callto: links (Skype) as relative URIs. I e-mailed GigaBlast's support yesterday and already they've come back with a response. Good customer service I say!