Reply to thread

I have found Bottopia (just google it) to be quite useful for scraping search engine urls and also youtube urls. There are two programs which are free but work pretty much like the paid ones.  The free ones come with advertisements and the paid ones without. They have got good proxy settings as well...These programs can harvest up to 1000 urls. But it will take some time though, if you load it with multiple keywords and push it to the limit.

Warning: Make sure you have a good updated list of proxy servers before attempting these. You don't want your ip to get banned.

P.S: There is one from Aroussi [aroussi[dot]com/freebies/scraper] as well which can scrape Google, Bing and Yahoo. This one's a self-hosted. So, if you have a website then you can download the scraper and upload it to your cpanel. Then u can access it from your site.

Both of these are helping me to harvest urls quickly.