07-26-2015, 09:39 AM
Hi
I am a newbie so please excuse me if Im missing something simple.
Last week I began successfully scraping a directory for emails (thomsonlocal.com). I first scraped the directory for thousands of internal pages then scraped those pages for external links. Then scraped those links for emails.
This process worked fine and i managed to get quite a few emails. Then suddenly I found that I could no longer get outbound links from the thomson local pages. I received the error 502.
I tried changing my proxies and this time including a 30 sec delay with only 1 connection, but i receive the same error. I have changed my proxies numerous times now informing buyproxies.com of my problem, they have told me that as long as the proxies are google passed, this should be ok.
I would really appreciate any help in this as I am completely stuck.
I look forward to hearing from you.
I am a newbie so please excuse me if Im missing something simple.
Last week I began successfully scraping a directory for emails (thomsonlocal.com). I first scraped the directory for thousands of internal pages then scraped those pages for external links. Then scraped those links for emails.
This process worked fine and i managed to get quite a few emails. Then suddenly I found that I could no longer get outbound links from the thomson local pages. I received the error 502.
I tried changing my proxies and this time including a 30 sec delay with only 1 connection, but i receive the same error. I have changed my proxies numerous times now informing buyproxies.com of my problem, they have told me that as long as the proxies are google passed, this should be ok.
I would really appreciate any help in this as I am completely stuck.
I look forward to hearing from you.