
Web crawler scrapping development tool
- or -
Post a project like this3513
$$
- Posted:
- Proposals: 3
- Remote
- #869889
- Completed
Description
Experience Level: Intermediate
Hi!
I need to extract raw data from certain pages of sale of second-hand products and store in any structured storage (db better than csv).
Then we need to crawl it periodically to get new global copies of al the data to know the new products that people are selling, to get the changes in the existing products and to get notice of the ads that have been deleted (I understand that if an ad is removed is because the product has been sold, and i want to know it).
With this we could get the price evolution of certain types of products, grouping this information by the other fields that i extract from the product detail page.
There are some webs that we need to crawl and extract info.
One of this is http://www.agriaffaires.es/
There are several categories and subcategories. Just a subset of this have information that we need. But every categorie have, finally, a product detail page (like this http://www.agriaffaires.es/usado/tractores-agricolas/6105520/john-deere-7530-premium.html ) that has information in different fields about the product that they are selling.
for all this (the need to update this data periodically), i need the development of a tool that can do that installed in my own servers..
Could you develope some tool like this? (maybe using any of the frameworks that exists , like http://scrapy.org/ or ssoomething like this)
Thanks in advice
I need to extract raw data from certain pages of sale of second-hand products and store in any structured storage (db better than csv).
Then we need to crawl it periodically to get new global copies of al the data to know the new products that people are selling, to get the changes in the existing products and to get notice of the ads that have been deleted (I understand that if an ad is removed is because the product has been sold, and i want to know it).
With this we could get the price evolution of certain types of products, grouping this information by the other fields that i extract from the product detail page.
There are some webs that we need to crawl and extract info.
One of this is http://www.agriaffaires.es/
There are several categories and subcategories. Just a subset of this have information that we need. But every categorie have, finally, a product detail page (like this http://www.agriaffaires.es/usado/tractores-agricolas/6105520/john-deere-7530-premium.html ) that has information in different fields about the product that they are selling.
for all this (the need to update this data periodically), i need the development of a tool that can do that installed in my own servers..
Could you develope some tool like this? (maybe using any of the frameworks that exists , like http://scrapy.org/ or ssoomething like this)
Thanks in advice

Juanjo N.
100% (46)Projects Completed
46
Freelancers worked with
40
Projects awarded
30%
Last project
1 Jun 2024
Spain
New Proposal
Login to your account and send a proposal now to get this project.
Log inClarification Board Ask a Question
-
There are no clarification messages.
We collect cookies to enable the proper functioning and security of our website, and to enhance your experience. By clicking on 'Accept All Cookies', you consent to the use of these cookies. You can change your 'Cookies Settings' at any time. For more information, please read ourCookie Policy
Cookie Settings
Accept All Cookies