Does anyone have tips on how to circumvent cloaking?

The scenario:

A competitor uses cloaked redirects and other unknown techniques to achieve high rankings in google. The competitors pages are not cached.

There are thousands of IBLs to the competitors site, from other pages on the competitors domain which are also not cached.

I would like to study all these pages to get a better idea of how this all works.

Since there is no cache, I try wgetting the domain. But wget
a) also tries to follow the redirects
and b) croaks on some kind of cryptic, I think, cookie related issue.

Nothing gets wgetted.

Does anyone have any suggestions on other tools / methods I could use to get to a competitors cloaked content?

Thanks in advance!

Thomas.

 

See

http://www.google.de/search?hl=de&ie=UTF-8&q="user+agent+spoofing"+cloaking&meta=
http://www.google.de/search?hl=de&ie=UTF-8&q=decloaking+"user+agent"&meta=

http://cal.phonelosers.org/forum/news170.htm
http://www.webmasterworld.com/forum24/377-1-10.htm
http://www.wannabrowser.com
http://www.gritechnologies.com/tools/spider.go

Following is a discussion about how to configure apache to lock out undesired user agents. IE, the same problem viewed from the other side of the table.

http://www.webmasterworld.com/forum13/687.htm

IF you can speak German this is good

http://board.protecus.de/showtopic.php?threadid=9838

including tips that ie6 user agents can be spoofed with proxomitron.

http://www.proxomitron.info



Supposedly this can be done with wget to rip an entire site. Haven't gotten this working yet, but it's on the to-do list. (I believe explained on the cal.phonelosers link.)

t.