KellyKeeton dot Com v3.0 reverse engineering life

10Jul/070

a blog i found

googled me

googled me
today.found a guy with a blog that looks like his, I say his because I use the theme hecreated and I found a problem with IE7 and helpedhim fix it
helpedhim fix it
. Well I post this because he has a goodtech-nerd-IT blog just like me. So add his to your RSS

10Jul/070

browse sites as a bot and get free info

The following trick will allow you to view some BBS services with out having to register,or some technical sites that require a log-in you might be able to access the dataso you dont need to have accounts all over the intertubes. When this gets banned outjust change to any other bots see the last bit of the post for all bot info I tookfrom the dasblog source code.

alter your settings to the following:

User Agent: Googlebot/2.1
Compatible: http://www.googlebot.com/bot.html

You can do so in Opera with ease. Firefox offers anextension which is downloadable from the official website.

to set up the plugin, click on add, the name your 'agent" something like: google

in the description text box. For the "User Agent" field, put this: Googlebot/2.1(http://www.googlebot.com/bot.html)

save it, then to acces the plugin, go to tools (next to help on the menubar in firefox) mouse over Agent Switcher, and select google, surf away. A word ofwarning, some sites will ban you if they do an IP range check, or a reverse DNS checkand your IP doesnt match that of their stored googlebot IP addy or DNS

For Internet Explorer you need to change registry entries.

Windows Registry Editor Version 5.00 [HKEY_LOCAL_MACHINESOFTWAREMicrosoftWindowsCurrentVersionInternetSettings5.0User Agent] @="Googlebot/2.1" "Compatible"="+http://www.googlebot.com/bot.html"

Save this as bot.reg and execute.

To revert the changes back, you need the following:

Windows Registry Editor Version 5.00 [HKEY_LOCAL_MACHINESOFTWAREMicrosoftWindowsCurrentVersionInternetSettings5.0User Agent] @="Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"

Save as nobot.reg and execute.

<UserAgents>
  <string>msnbot-Products/1.0 (+http://search.msn.com/msnbot.htm)</string>
  <string>MJ12bot/v1.0.8 (
http://majestic12.co.uk/bot.php?+)>
  <string>ISC Systems iRc Search 2.1</string>
  <string>ichiro/2.0 (
http://help.goo.ne.jp/door/crawler.html)>
  <string>Mozilla/4.5 (compatible; HTTrack 3.0x; Windows 98)</string>
  <string>asterias/2.0</string>
  <string>www.adressendeutschland.de</string>
  <string>NutchCVS/0.7.1 (Nutch;
http://lucene.apache.org/nutch/bot.html; raphael@unterreuth.de)>
  <string>Snapbot/1.0</string>
  <string>msnbot/1.0 (+http://search.msn.com/msnbot.htm)</string>
  <string>Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)</string>
  <string>Mozilla/5.0 (compatible; BecomeBot/2.3; MSIE 6.0 compatible;+http://www.become.com/site_owners.html)</string>
  <string>RufusBot (Rufus Web Miner;
http://64.124.122.252/feedback.html)>
  <string>Mozilla/5.0 (compatible; Yahoo! Slurp;
http://help.yahoo.com/help/us/ysearch/slurp)>
  <string>Gigabot/2.0/gigablast.com/spider.html</string>
  <string>TurnitinBot/2.0
http://www.turnitin.com/robot/crawlerinfo.html>
  <string>Mozilla/5.0 (compatible; BecomeBot/3.0; MSIE 6.0 compatible;+http://www.become.com/site_owners.html)</string>
  <string>Sphere Scout&amp;v4.0 (beta) - scout at sphere dot com</string>
  <string>Gigabot/2.0;
http://www.gigablast.com/spider.html>
  <string>msnbot/0.9 (+http://search.msn.com/msnbot.htm)</string>
 </UserAgents>

 So, how do you beat all 5 major types of cloaking?

1. Beat IP Delivery: Use GoogleTranslate as a Proxy, translating from spanish->english even though the siteis already in English.
2. Beat User-Agent Cloaking: Use the FirefoxUser-AgentSwitcher

User-AgentSwitcher
to spoof as GoogleBot
3. Beat Javascript Detection: Use the Firefox WebDeveloper Toolbar to turn off javascript.
4. Beat Cookie Detection: Use the Firefox WebDeveloper Toolbar to turn off cookies.
5. Beat Referer Detection: Use the Firefox RefControlExtension to prevent referer from being sent.

Using these in conjunction can be extremely effective, even at pay-for-informationsites.
Doing this may be against the terms of service of the site you are visiting.There are plenty of popular sites out their that cloak content which is normally onlyavailable to paying members. While these techniques work on those sites too, be careful.

Good browsing!

Filed under: Internet No Comments