Post Project
  • Search
    • Buyers can
    • Search offers to buy now
    • Search freelancers to request a proposal
    • Freelancers can
    • Search projects to quote on
  • How it works
  • Log in
  • Sign up
  • Freelancer?
Browse by Category
    Technology & ProgrammingWriting & TranslationDesignDigital MarketingVideo, Photo & ImageBusinessMusic & AudioMarketing, Branding & SalesSocial Media

    Web Scrapping for 3 car sites

    - or -

    Post a project like this
    09/08/2016
    £200(approx. $245)
    • Posted: 7 years ago
    • Proposals: 6
    • Remote
    • #1213567
    • Expired
    Kaycelyn G.
    Kaycelyn G.
    Virtual Assistant / Data Encoder
    Philippines Tayabas
    Satish G.
    Satish G.
    LinkedIn Lead Expert, Web Scraping, Data Entry, Data mining, Web Crawling, Data extraction and Email Lists
    Top Endorsed
    India Gondia
    UMAIR K.
    UMAIR K.
    Web Scraping & WordPress Expert
    Top Endorsed
    Pakistan Lahore
    A.S.M R.
    A.S.M R.
    Web Researcher | Data Scraper | Data Entry |Digital Marketing | Lead Generation | Content Upload | Social Media
    Top Endorsed
    Bangladesh Tangail
    Jitendra Y.
    Jitendra Y.
    Mobile & Web (Wordpress, eCommerce, Shopify, Laravel , Yii , CakePhp, Symfony, Drupal, Android/iOS) | UI/UX Design | Graphic | SEO
    India Ahmedabad
    Avanish M.
    Avanish M.
    Scraper / crawling / data extraction
    India New Delhi
    129073512859199203348223959011301331203
    Kaycelyn G.Satish G.UMAIR K.A.S.M R.Jitendra Y. + 1 other have already sent a proposal.
    • 0
    • 0

    Description

    Experience Level: Expert
    General information for the website: Web scraper for three car sites for data
    Description of requirements/features: We require a tool to scrape carsales.com.au, carsguide.com.au, and redbook.com.au websites for relevant information regarding specific cars and models. Keep in mind we may need to incorporate more sites in the future.

    As we understand both redbook.com.au and carsales.com.au are under anti scraping systems.

    For each website, we need to set up a MySQL database containing each of the scraped data with each field in its own individual column. You will need to also write an aggregation interface to work with the result.

    The tool should be made available for us to:
    - initiate the scrape and run against the sites and populate the database with new data periodically until no more results are present
    - configure how often to scrape the sites i.e. daily, weekly, monthly should we want it to continously run it until all results on the page(s) are scraped.
    - import CSV (or Excel) of cars

    The data to scrape for carsguide.com.au include the following:
    - URL
    - Price (integer)
    - Model
    - Make
    - Variant
    - Series
    - Colour ext
    - Body type
    - Seats
    - Doors
    - Transmission
    - Engine
    - Drive Type
    - Fuel Type
    - Fuel Consumption
    - Kilometres (integer)
    - Marketing Year (MY)
    - Year
    - Features

    For example
    - URL : http://www.carsguide.com.au/cars-for-sale/NEW_UCJ16D_Mystic_Violet/HOLDEN--SPARK------Hatchback?searchKey=cg_s.38ffc6de7550de6aec952b87c417c3ac#pos3
    - Price : 16690
    - Model : Holden
    - Make : Spark
    - Variant : LS
    - Series : MP
    - Colour ext : Mystic Violet
    - Body type : Hatchback
    - Seats : 5
    - Doors : 5
    - Transmission : Automatic
    - Engine : 4 cyl, 1.4 L
    - Drive Type : Front
    - Fuel Type : Unleaded
    - Fuel Consumption : 5.8 L / 100 km
    - Kilometres : (not showing on some cars, but when it does extract it)
    - Marketing Year (MY) : 16
    - Year : 2016
    - Features : (everything under the features tab, comma separated)


    The data to scrape for carsales.com.au include the following:
    - URL
    - Price
    - Model
    - Make
    - Badge
    - Series
    - Colour
    - Body style
    - Seat Capacity
    - Doors
    - Transmission
    - Engine
    - Drive Type
    - Fuel Type
    - Fuel Consumption
    - Kilometres
    - Marketing Year (MY)
    - Year
    - Features


    For example
    - URL : http://www.carsales.com.au/bnc/details/Holden-Spark-2016/OAG-AD-12588414/?gts=OAG-AD-12588414&gtssaleid=OAG-AD-12588414
    - Price : 14990
    - Model : Holden
    - Make : Spark
    - Badge : LS
    - Series : MP
    - Colour : Solar Red
    - Body style : Hatch
    - Seat Capacity : 5
    - Doors : 5
    - Transmission : Manual
    - Engine : 4cyl 1.4L Petrol
    - Drive Type : Front
    - Fuel Type : Petrol - Unleaded ULP
    - Fuel Consumption : 5.2 (L/100km)
    - Kilometres : (not showing on some cars, but when it does extract it)
    - Marketing Year (MY) : 16
    - Year : 2016
    - Features : (everything under the features tab, comma separated, without the headings)

    The makes and models are in the excel sheet provided under the Makes and Models sheet. When scraping, we require every listing from the search result on the make and model.

    The examples show the field names that are on the site but we will need to also normalize the database field to have the same column name regardless of the website we are scraping.

    The scraper needs to be 100% tested without issues of anti spam detection.
    Extra notes:
    •  
    Genial Web Techologies
    Genial Web Techologies
    98% (9)
    Projects Completed
    9
    Freelancers worked with
    9
    Projects awarded
    16%
    Last project
    6 Feb 2021
    India

    New Proposal

    Login to your account and send a proposal now to get this project.

    Log in

    Clarification Board Ask a Question

    • 11 Jul 2016David B.

      Are you in a rush or flexible on deadline?

    • 11 Jul 2016David B.

      Are you in a rush or flexible on deadline?

    5514155101
    09/08/2016
    £200(approx. $245)

    - or -

    Post a project like this
    Genial Web Techologies
    Genial Web Techologies
    98% (9)
    Projects Completed
    9
    Freelancers worked with
    9
    Projects awarded
    16%
    Last project
    6 Feb 2021
    India

    Related project Searches


    data scraping mysql comma separated values(csv) database extraction model making web scraping

    Product

    • About
    • Team
    • Careers

    Support

    • How it works
    • Trust & Safety
    • Help Centre

    Discover

    • GuidesStoriesNews

    Resources

    • Customer Stories
    • Business Cost Calculator
    • Startup Cities

    Browse

    • Freelance Services
    • Freelance Services By Country
    • Freelance Skills
    • Terms
    • Privacy
    • Sitemap
    • Company Details
    • © 2023 People Per Hour Ltd
    We collect cookies to enable the proper functioning and security of our website, and to enhance your experience. By clicking on 'Accept All Cookies', you consent to the use of these cookies. You can change your 'Cookies Settings' at any time. For more information, please read ourCookie Policy
    Cookie Settings
    Accept All Cookies