Use Multiple LinkedIn Accounts to Scrape 80,000 Profiles
- or -
Post a project like this- Posted:
- Proposals: 32
- Remote
- #2537840
- OPPORTUNITY
- Expired
Description
I'm not entirely certain of the output structure that I want yet - whether that be as an object array with each job as a node on the array with child elements within each node for the company name, title, years of employment, location, description, etc. - or if I would want it to be linear in typical CSV format with 5 columns for each listed job, going from most recent to oldest, 5 columns at a time, up to however many jobs are listed on the LinkedIn profile.
The difficult aspect of this project is the scale. There will initially be 80,000 LinkedIn profiles to do this full employment scrape for and then potentially an additional 120,000 profiles afterward.
If you have experience scraping LinkedIn at scale, you are likely already familiar with their commercial use limits for viewing profiles, even through the access of a LinkedIn premium account like Sales Navigator which allows at most 500 profile views per day.
I'm looking to get the first 80,000 profiles done within the span of 1 month so I'm looking for someone who will have the right strategy and approach to accomplish this and get around LinkedIn's limitations - whether that be through the use of some 3rd party API, through the use of many dummy accounts or some other method.
Ideally, you would use 6 separate LinkedIn accounts to each scrape the maximum 500 profiles per day to be able to scrape all 80,000 profiles within one month's time. Anything that will take longer than 1 month will be considered unacceptable.
Let me know your thoughts, questions, ballpark pricing, timeline, anything else you think is relevant. I'm looking for proposals that are well thought out from freelancers that seem credible and experienced in such work. Any generic proposals or boilerplate promises to somehow get the work done will be ignored. This is a serious job and I am ready to pay appropriately for the work.
Attached is a sample of the kind of profiles that will need to be scraped.
They are all lawyers/attorneys, primarly located in the United States.
Please write the name of your favorite food at the top of your proposal to help weed out robo-proposals.
Many thanks and I look forward to discussing the work with you.
Mark M.
100% (5)New Proposal
Login to your account and send a proposal now to get this project.
Log inClarification Board Ask a Question
-
Hi Mark,
have you already found someone for this? I'll have it done manually just to ensure the quality of the data. Is the deadline a month away from when you awarded the job or from when you posted the job? Might need to get started soon? Let me know, thanks!
cheers,
AgnesMark M.16 Sep 2019There's no way to do 80,000 profiles manually in 1 month. It must be automated.
Agnes L.17 Sep 2019Hi Mark,
thanks for getting back to me!
Yes of course, but to spread it across 6 accounts, 6 peoples? I think that would be fine?!
Let me know, thanks!
cheers,
Agnes -
you need data of specific region or from anywhere from the globe?
-
Question - Do you want script to scrap profile? and do you want the public profile data to scrap as the logged in data can't be scrapped.
Mark M.14 Sep 2019Thank you for the question. I am not looking for scripts to use. Rather, I am looking for a freelancer who will do the scraping work themselves (by whatever means they choose) and provide target output to me. I have the regular LinkedIn profile URLs (not Sales Nav profiles) and they need to be scraped while logged in (not public) so as to be able to get the employment history.
Reena K.14 Sep 2019Thanks for your reply.
Question - As you mentioned you need employment details of user. The public profile has employment details of user - About, Experience, Education, Volunteer Experience, Licenses & Certifications, Honors & Awards, Languages. What else you want to fetch from profile?