Data Scraping Development Projects
Looking for freelance Data Scraping Development jobs and project work? PeoplePerHour has you covered.
featured
Seeking Testers for Social Media App - iOS/Android Edition
I am seeking 3 testers from the USA, Canada, and UK for a social media app available on both iOS and Android platforms. About the App: The app is a free social media platform designed for property owners or investors. To participate in the testing process, candidates should either own properties themselves or obtain approval from property vendors. Requirements: 1.Testers can sign up for the app using usernames and are not required to disclose the real names of property vendors. 2.Upon sign-up, candidates should engage in social interactions within their neighborhoods to endorse a proposal on the app. This proposal could be for a property mega lot sale (collective sale) campaign or a petition for rezoning in the local community. 3.To qualify full amount payment, a tester must secure at least one endorsement from their neighborhood to support the proposed campaign or petition. 4.Candidates must create a proposal with valid data in accordance with the instructions provided on the app's YouTube channel. For guidance on how to create a campaign proposal, please refer to the "How to Create a Campaign Proposal?" video on the channel named "Crowdzoning." 5.Delivery Time:30 days;
Xero - data developer
Hello, our company is looking for a Xero Accredited Developer who could help us connect our custom Access database (called Switchboard) to Xero so we can use it for one click invoicing. We currently have it set up with Sage but we are in a process of moving our accounting to Xero. Please get in touch is you could help. Many thanks, Jana Auton
17 hours ago4 proposalsRemoteFix error in Python-based web scraper wit GUI
Hello Freelancers, I'm searching for a developer familiar with web scraping and Python to fix an existing web scraper which scrapes product data from products from category links from italian ecommerce-website www.yeppon.it The script basically works, but it gives an error on certain points when scraping, I think because of a light change in the structure of the website which causes an error when the script tries to scrape a products text description. Goal of this project is to fix the errors so the script works like it used to again, scraping data of products from given category-URLs from the website and giving out the data in csv-files. I think this won't be too much of an effort because it is basically this one error which needs to be located and fixed, everything else still seems to work fine. Price can be discussed. Some facts: 1. The web scraper is based on Python with a GUI. It's final version comes as an exe file (therefore I can't attach it in the project description, I will send it in the messages or work stream). 2. It scrapes certain product data (like product name, price, description, image links) by category links which can be entered into the GUI. The GUI also has some input fields, these are just for fixed strings which can be entered into the fields and will be given out in the CSV files the script gives the product data in. 3. The scraper technically still works, however, it gives an error when scraping certain categories. You can check this by running the tool, filling out the given input fields with the data explained in the "Instructions" tab of the tool and then start scraping. It will produce this error (can be found in the log file): -------------------------------------------------------------------------------------------------------------- 2024-04-24 12:44:24,765:ERROR:'descriptionHtml' Traceback (most recent call last): File "async_scraper.py", line 739, in scrape description_html = pdata["pageProps"]["product"][ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ KeyError: 'descriptionHtml' 2024-04-24 12:44:24,765:INFO: Finally 2024-04-24 12:44:24,765:INFO: No data found! ------------------------------------------------------------------------------------------------------------------ 4. The error seems to occur when scraping a products text description. The text description consists of three possible elements: - bulletpoints formatted into an ul element - a text description which is cleaned/has HTML code removed/replaced - scraping data from a table on the website and putting it into a given HTML structure It was developed by a freelancer from PPH for a colleague of mine, unfortunately I can't reach my colleague for quite some time now to ask for all the details or the freelancers name, so I will post this to the public. Scraping some categories will result in the error mentioned above, for example: https://www.yeppon.it/c/elettrodomestici/grandi-elettrodomestici/asciugabiancheria or https://www.yeppon.it/c/elettrodomestici/grandi-elettrodomestici/frigoriferi Others work just fine, like: https://www.yeppon.it/c/telefonia/smartphone/smart-phone I will attach the files I have about this project from my colleague. As I can't attach exe or rar files, I attached: - a first version of the Python code (it is a beta version which will give another error which is solved in the final exe file and not the final code, just to give you an impression), as well as the code of the GUI and the requirements. These are async_scraper.txt, gui.txt and requirements.txt
8 hours ago12 proposalsRemoteData Scraping from LinkedIn search results
I need a data entry personnel to search out and scrape 200 relevant posts on LinkedIn using the keywords provided, filter the search to the United States, and add data to an Excel sheet. Data needed include: 1. Profile name 2. LinkedIn profile link 3. Post link Recorded a loom to help guide the process: https://www.loom.com/share/1191b16f8a124767a3c1d1f68fb6616b?sid=b8d5833f-3a3d-4e44-bd82-13170a35e165 Document containing keywords to search: https://docs.google.com/document/d/19pXU0P6IvUVO_PrVSre36MmOBQ-CBFLt7oIq3x7FmOM/edit?usp=sharing
8 days ago70 proposalsRemoteWeb Scraping
Hi, I want to scrape data from justdial.com. It is showing only 100 records per 1 query but I need to bypass the security and scrape 1k-2k records instead of 100. Can anyone help? FYI, It is a business directory like yelp, yellowpages etc.
2 days ago24 proposalsRemoteSelenium-based data scrape of targeted LI profiles
The creation of a SQL database by scraping data from ~345,000 records from LinkedIn. - Selenium scripts to capture multiple fields from LI profiles (including Summary, Role Descriptions, Dates in Roles) - The storing of the data in an SQL database - Detailed technical documentation about each script
a day ago29 proposalsRemoteSeeking Web Scraper for Adjustments to Existing Real Est. Script
I am a Python enthusiast in need of a freelance web scraper for a personal project to make adjustments to an existing Python script that extracts data from Idealista.com. The goal is to refine the script to handle the website’s anti-scraping measures effectively. Project Overview: Script Adjustment: Update and optimize an existing WORKING script to improve data extraction reliability and efficiency. But issues with anti scrapping defence from the website. Anti-Scraping Strategy: Implement simple and cost-effective strategies to overcome scraping blocks. Requirements: Proven experience in modifying existing web scraping scripts. Familiarity with Python and scraping libraries like BeautifulSoup, or others. Understanding of techniques to bypass or handle anti-scraping technologies. Ability to work within a tight budget and deliver quick solutions. Deliverables: An updated, fully functional scraping script. Brief documentation on the changes made and how to handle potential future issues. Project Duration: Quick turnaround expected. Budget: Since this is an adjustment to an existing script, I am looking for a cost-effective solution. Please provide your fixed-rate offer for this adjustment. I can provide my existing code, so you can evaluate changes needed. If you are interested, please respond with: Any initial thoughts or specific techniques you think could be effective. I am looking to start as soon as possible and appreciate your swift response. Thank you!
3 days ago19 proposalsRemoteopportunity
Data Extraction and Input
Hi all, I am looking for help to pull some price modelling data from an website and place it into an excel file that replicates how it is shown on the web page. There is a lot of data and I do not think it can be exported, it needs to be done manually. I do not think it is viable to scrape the data. Once we have pulled the data we then need to build charts / data filtering to display the data correctly. My feeling is that the freelancer that does the job will need to be an expert an MS excel. It would be easier to do a Zoom call to show the freelancer the system and explain fully what is required so the price below is a placeholder. Thanks!
6 days ago66 proposalsRemoteWeb scraping
I am looking at Web scraping software, based on input; to produce a spreadsheet of URL, email address and phone numbers extracted from the contact us pages,using legitimate methods. A URL scrapper will also need to be built into the application. To be able to facilitate the above process.
8 days ago30 proposalsRemoteData scrape consolation
An expert consultation is sought regarding modern artificial intelligence techniques for efficient data scraping. For over nine years, traditional scraping methods using common websites have been employed to gather information from Google Maps and LinkedIn. However, newer AI technologies promise scraping capabilities at significantly reduced costs compared to established solutions. Rather than using code or tools directly, the consultant would identify and briefly describe the ten best scraping platforms that leverage artificial intelligence to deliver high-quality data rivaling industry leaders, but at optimized pricing. In return for a10 minute consultation providing these top scraping website recommendations, a £10 payment will be provided. The aim is to uncover the highest data value possible through cutting-edge AI scraping alternatives.
a month ago28 proposalsRemotePython Developer - Scraping Expert
Hi All I have a python code that work but needs abit of editing. Is this something you can help with today? Thanks Phil
23 days ago31 proposalsRemoteopportunity
I need News Bot in C or C++, Phyton Language
Hello, I need a News Bots written in C, C++ or Phyton language that has features. I need 30 News bots that will scrape Latest News some Big News Sites's Category Pages or Rss Pages, Translate scraped data from source site language to target site language and post the news articles with image and features like News Title , Tags ect in Wordpress Sites with same or different categories via crons. Bot's main system will be same for all 30 bots, you can create one bot scraping from 30 different sites or you can create 30 different bots. I will give extra details in workstream But I will refuse freelancers who give low price in proposal and ask extra money later. I have fixed budget. You can pass this task if this money doesnt make you happy.
8 days ago23 proposalsRemoteWordpress expert familiar with API and datadriven templates
We are in search of a web developer for WordPress who is familiar with the WordPress API and the concept of data-driven pages. We need page templates built which will show data from a separate system.
19 hours ago80 proposalsRemotepre-funded
I would like some pages scraped which are behind reCAPTCHA
I have a list of 50k webpages of the same design on the same website, each of which includes a piece of text I want to be scraped and exported. The problem is that the page is protected by "I am not a robot" reCAPTCHA. So I would like someone to take my list of 50k URLs and return the the missing line of text for each page in a csv or google sheet. Note that I do not need to code you use - just the end result. Here is a redirect to an example of the pages I need scraped https://www.temporary-url.com/9C6C0 Please check the page first and explain in your answer what the site and the info I need are (this is to make sure people have read the ad fully and checked out the task. Without this, your reply will be marked as spam). Please give me the cost and time it would take to provide them all.
16 days ago29 proposalsRemoteSeeking No-Code Architect for Automation Project
We are seeking a talented no-code architect to assist us with a project focused on web scraping, content filtering, and process automation. The ideal candidate will have expertise in identifying and configuring the best no-code tools and platforms for our requirements, enabling us to automate processes and extract valuable insights without traditional coding. Responsibilities: Evaluate project requirements and objectives for web scraping, content filtering, and automation. Research and assess available no-code platforms, tools, and integrations suitable for the project's needs. Recommend and select the most suitable no-code solutions for web scraping, data filtering, and automation tasks. Configure and set up integrations between selected no-code tools and platforms to achieve seamless automation. Provide guidance and best practices for leveraging no-code solutions effectively to meet project goals. Requirements: Proven experience as a no-code architect or consultant, with a strong track record of implementing successful no-code solutions. Comprehensive knowledge of various no-code platforms, tools, and integrations, particularly those suitable for web scraping and automation. Familiarity with web scraping techniques, data extraction methods, and content filtering processes. Strong problem-solving skills and the ability to assess project requirements to recommend suitable no-code solutions. Excellent communication skills and the ability to collaborate effectively with project stakeholders. If you're passionate about harnessing the power of no-code tools to automate processes and drive efficiency, we'd love to hear from you! Please provide details of your relevant experience and examples of previous projects involving no-code architecture and automation. Budget: Flexible, depending on experience and scope of work. Duration: Short-term project with potential for ongoing collaboration. Deadline for Applications: [Insert deadline or state "Open until filled"] We look forward to reviewing your proposals and discussing how we can work together to achieve our project goals. Thank you for considering this opportunity!
7 days ago13 proposalsRemoteDatabase Design for Fleet Management
Summary: We are looking for an experienced freelance Database Engineer to create a high-performance database tailored for managing operational data from our truck fleet. The role involves collaborating with the Original Equipment Manufacturer (OEM) of the trucks to integrate our internal server with theirs. This integration will facilitate the transfer of real-time operational data. The primary goal is to store this data in our internal database, enabling data analytics to optimize fleet operations. +++ Scope of Work: The project will be executed in four main stages, with the initial contract focusing on the first two stages: 1. Database Design and Development: Design and implement a high-performance database hosted on our internal servers. This database will be linked to real-time data on the servers of the truck's OEM, ensuring seamless synchronization and data integrity. 2. Data Visualization and Reporting: Develop basic visualization and reporting capabilities to allow for easy interpretation and analysis of the collected data. This will include the creation of dashboards or reports that highlight key metrics and trends. (Future Stages to be Scoped Separately) 3. Develop a web portal, hosting data reports developed in stages 1 and 2. 4. Develop a mobile app, hosting data reports developed in stages 1 and 2. +++ Deliverables: 1. Develop and host a database on an internal server that maintains a consistent and complete dataset of real-time operational data. 2. Establish and document automatic synchronization between the internal database and the servers hosted by the OEM. 3. Create a basic dashboard for visualizing the operational data stored in the database, featuring essential metrics and real-time updates. +++ Key Responsibilities: • Analyze and understand the structure of the existing data on the OEM servers. • Design a scalable and secure database architecture. • Implement the database on our internal servers, ensuring real-time data synchronization with the OEM servers. • Develop basic tools or dashboards for data visualization and reporting. +++ Skills and Qualifications: • Proven experience in database design and development, especially with large and complex datasets. • Proficiency in database management systems (e.g., SQL, NoSQL) and data visualization tools. • Excellent analytical and problem-solving skills. • Ability to work independently and in collaboration with cross-functional teams. • Effective communication skills, with the ability to document and explain technical details clearly. +++ Project Timeline: The initial scope of the project (stages 1 and 2) is expected to be completed over the course of eight (8) full time days within a one-month timeframe (April 29th – May 31st). This period allows for actual development, with additional time allocated for necessary engagements with external stakeholders at the OEM. +++ Application Process: Interested candidates are invited to submit a proposal outlining their approach to the project, including a timeline, budget estimate, and a portfolio of relevant work. Shortlisted candidates may be invited for an interview to discuss the project in further detail. +++ About Us: At Bisedge we provide, maintain and operate mainly electric material and cargo handling equipment for companies in Africa.
3 days ago37 proposalsRemoteI need text scraped form 8000 URLs
I have an Excel with 8000 URLs in column A In Column B I need the text scraped for each of those websites (in plain text)
6 days ago36 proposalsRemoteWebsite development
We're seeking a talented web developer to modernize our existing website. Our current site served us well, but we recognize the need to refresh its design, enhance functionality, and optimize performance to meet the evolving needs of our users. As such, we're looking for a freelancer who can breathe new life into our online presence and deliver a seamless, user-friendly experience.
2 days ago43 proposalsRemoteSave HTML Page Python Script
Hi All I have a python code that work but needs abit of editing. Is this something you can help with today and need to scrape the site today Thanks Phil
10 days ago15 proposalsRemoteRecover my data.
Hello, my google drive has been hacked and the password changed. This is apparently quite common, but google provide no way or support to fix the issue. I have an old list of contacts, but without email and mobile phone numbers. Any proposals on how to recover the missing data please?
9 days ago18 proposalsRemoteopportunity
Expert Ruby on Rails Developer Needed for Project Development
We are seeking a highly skilled Ruby on Rails developer to lead the development of our new marketplace platform in sharetribe GO. The ideal candidate will have extensive experience with Sharetribe Go, including both front-end and back-end development, to help us build a fully functional marketplace from scratch. Project Overview: We aim to create a user-friendly, efficient, and scalable marketplace platform using Sharetribe Go. We will provide the HTML design templates for the front end. Your primary responsibility will be to integrate these templates with Sharetribe, ensuring a seamless and visually appealing user interface. Furthermore, you will configure and customize the Sharetribe backend to meet our specific business requirements, ensuring that all Sharetribe functionalities are perfectly implemented and operational. Key Responsibilities: Full integration of provided HTML design templates with Sharetribe Go, ensuring the front end is responsive and compatible across all devices and browsers. Configuration and customization of the Sharetribe backend, including but not limited to user registration, product listings, search functionality, payment processing, and user feedback systems. Ensure the platform is scalable, with efficient code that allows for easy future modifications and additions. Implement security measures to protect user data and transactions. Provide technical support and maintenance post-launch, including troubleshooting and adding new features as required. Requirements: Proven experience with Sharetribe Go development, including both front-end and back-end tasks. Strong knowledge of HTML, CSS, JavaScript, and Ruby on Rails. Experience in integrating HTML/CSS templates with Ruby on Rails applications. Ability to work independently, manage time effectively, and deliver projects on deadline. Excellent problem-solving skills and attention to detail. Good communication skills, as you will be required to provide regular updates and feedback. Project Timeline: We are looking to get this project up and running as soon as possible. Please provide an estimated timeline for the completion of the project along with your application. If you are passionate about creating high-quality, impactful work and have the skills required for this project, we would love to hear from you. Please apply with your portfolio showcasing previous Sharetribe projects, a brief introduction about yourself, and your proposed approach to this project.
a day ago23 proposalsRemote