Python/Selenium Coding for Scraper
- or -
Post a project like this1903
£55(approx. $69)
- Posted:
- Proposals: 5
- Remote
- #2269530
- Awarded
Description
Experience Level: Intermediate
I already have a Selenium coded web scraper that i need some extra code / functionality adding to it.
The script i have currently works as follows:
- It has a .csv file with multiple website urls
- it goes to each url and scrapes data
- it saves the data to a csv file
- the script has settings and functions for waiting randomly between functions
I require the following functions adding to my script:
1) IP Rotation - The script must have settings / parameters for cycling through different IPs that are stored in a .csv file. The settings must include a random time function, where the script chooses a random time before the script moves to the next IP. How long to wait after moving to the next IP until the script starts again. After moving to a new IP the script must start where it paused.
2) User-agent spoofing
3) Error logging - must save either a .csv file or .txt file with any errors encountered while the script is running. As well as how many results were found while the script was scraping and how long the script ran for
4) Headless browsing
I will provide the selenium code i already have, the new functionality needs to be added to it and tested.
The script i have currently works as follows:
- It has a .csv file with multiple website urls
- it goes to each url and scrapes data
- it saves the data to a csv file
- the script has settings and functions for waiting randomly between functions
I require the following functions adding to my script:
1) IP Rotation - The script must have settings / parameters for cycling through different IPs that are stored in a .csv file. The settings must include a random time function, where the script chooses a random time before the script moves to the next IP. How long to wait after moving to the next IP until the script starts again. After moving to a new IP the script must start where it paused.
2) User-agent spoofing
3) Error logging - must save either a .csv file or .txt file with any errors encountered while the script is running. As well as how many results were found while the script was scraping and how long the script ran for
4) Headless browsing
I will provide the selenium code i already have, the new functionality needs to be added to it and tested.
Matt L.
100% (17)Projects Completed
7
Freelancers worked with
7
Projects awarded
50%
Last project
17 Apr 2020
United Kingdom
New Proposal
Login to your account and send a proposal now to get this project.
Log inClarification Board Ask a Question
-
There are no clarification messages.
We collect cookies to enable the proper functioning and security of our website, and to enhance your experience. By clicking on 'Accept All Cookies', you consent to the use of these cookies. You can change your 'Cookies Settings' at any time. For more information, please read ourCookie Policy
Cookie Settings
Accept All Cookies