Website Scraping - Using python or preferably Scrapy
- or -
Post a project like this2145
$100
- Posted:
- Proposals: 10
- Remote
- #2029589
- Awarded
Python Development, Web Development, Web Scraping, Data Mining, Automation, Web Crawling
Cairo
664984221760112407741355437140240114932381539295173041519326142238235
Description
Experience Level: Expert
Estimated project duration: 1 day or less
Hi,
We need someone to help us scrape a few sites. We currently use scrapy, hosted on our own server and would like to add a few more sites to our inventory. We want the scrapy scripts and not just the data.
1. koshermeat.co.za - everything under the Shop navigation
2. clicks.co.za - all categories
Secondly, we have a script that is written for another site, it works perfectly, however, we keep getting blocked after a certain number of requests so we would like to add maybe Tor functionality so that it can change identity every so many calls it makes. Alternatively if we cant use Tor, we are open to going the proxy route.
We need someone to help us scrape a few sites. We currently use scrapy, hosted on our own server and would like to add a few more sites to our inventory. We want the scrapy scripts and not just the data.
1. koshermeat.co.za - everything under the Shop navigation
2. clicks.co.za - all categories
Secondly, we have a script that is written for another site, it works perfectly, however, we keep getting blocked after a certain number of requests so we would like to add maybe Tor functionality so that it can change identity every so many calls it makes. Alternatively if we cant use Tor, we are open to going the proxy route.
Lynton P.
100% (5)Projects Completed
3
Freelancers worked with
3
Projects awarded
100%
Last project
1 Aug 2019
South Africa
New Proposal
Login to your account and send a proposal now to get this project.
Log inClarification Board Ask a Question
-
There are no clarification messages.
We collect cookies to enable the proper functioning and security of our website, and to enhance your experience. By clicking on 'Accept All Cookies', you consent to the use of these cookies. You can change your 'Cookies Settings' at any time. For more information, please read ourCookie Policy
Cookie Settings
Accept All Cookies