I installed Squid earlier this week to track all the http traffic.... effectively web scrapper. I stumbled on one issue that took more than 2 hours to debug...
http_access allow localhost
was needed so that my local machine traffic can be peroxided.
After the traffic started flowing, stumbled on issue that the 'dynamic' web content is not cached. While this can make sense as proxy, Squid is not usable for my use. I wish this feature was provided as charity at-least.
The data is binary - not plain text. This makes it difficult to parse easily...
I had to resort to Firefox extension slogger on firefox 2.0 (instead of the latest as of today 3.0.10). This combo did work and got my task done.
Blog Archive
-
▼
2009
(51)
-
▼
May
(17)
- Groups & Teams
- Social Bookmarking sites
- Business Search / Evaluation / InfoTools
- open source bookmark Platforms
- Social Softwares
- Web Advertisement Server
- Membership Management
- Social Network Service features
- squid localhost
- online Browser 'emulators'
- Java vs C/C++
- Local advertisers directory from Indian monthly ma...
- Free stuff adverticements
- Cholan features
- URL Shortenning Space
- Web 2.0 revenue stream
- cool Web2.0 free technologies
-
▼
May
(17)