07-14-2019, 09:06 PM
(This post was last modified: 07-14-2019, 09:07 PM by rightintwo.)
Hi Guys,
I am trying to scrape LinkedIn profiles for a certain job titles. When I run it, it stops at 300 results every time. The search that I am trying to scrape Google says has 2 million results, but I can only get to 300 reults total.
Here are the details:
URL I am scraping: site:linkedin.com (inurl:in OR inurl:pub) -intitle:directory -inurlalaries -inurl:dir -inurl:jobs "Director of IT" Note: the forum post is converting "colon salaries" to an emoji.
Proxies: Using proxies harvested through SB proxy manager. Currently have 168 proxies that passed the Google Proxy test.
I've tried using the detailed harvest and the custom harvester. Same result. I also reloaded the custom harvester settings to see if anything had changed.
Anyone have any suggestions?
I am trying to scrape LinkedIn profiles for a certain job titles. When I run it, it stops at 300 results every time. The search that I am trying to scrape Google says has 2 million results, but I can only get to 300 reults total.
Here are the details:
URL I am scraping: site:linkedin.com (inurl:in OR inurl:pub) -intitle:directory -inurlalaries -inurl:dir -inurl:jobs "Director of IT" Note: the forum post is converting "colon salaries" to an emoji.
Proxies: Using proxies harvested through SB proxy manager. Currently have 168 proxies that passed the Google Proxy test.
I've tried using the detailed harvest and the custom harvester. Same result. I also reloaded the custom harvester settings to see if anything had changed.
Anyone have any suggestions?