Scraping websites to create a database of information.
- or -
Post a project like this£250(approx. $313)
- Posted:
- Proposals: 32
- Remote
- #4159458
- OPPORTUNITY
- Expired
Experienced Virtual Assistant with Expertise in Data Mining, Lead Generation, and Customer Support
Bay Minette
LinkedIn B2B and B2C Lead Generation, Internet Research, Company Research, Data collection, Data Scraping, Data mining, Virtual Assistant, Excel remote Data Entry
Satkhira
Data Entry Expert | Excel/Googlesheet Ninja | Google Analytics| GA4|GTM|Tag Manager Google Ads|Python Expert
London
1052136367069627939282844514386441289091268914323293158159977324110011871097152010931228
Description
Experience Level: Intermediate
I am looking to obtain a list of all councils in England, together with the some corresponding data.
I need column entries for the name of the council, the website, and the number of households within that council. Each council has to be cross referenced to its respective council up or down the hierarchy, and have a reference number attributed to it. For Parish and Town Councils I need the email address for the clerk, always given on their website.
There is a hierarchy of councils in the UK, at the top are County, City, and Unitary councils. Beneath County councils there are district councils, and then Parish and Town councils. Parish and Town councils are the same thing. Beneath Unitary Councils there is no district, just Parish and Town councils.
List of all county and district councils here:
https://www.local.gov.uk/our-support/guidance-and-resources/communications-support/digital-councils/social-media/go-further/a-z-councils-online
Each link can be clicked through to navigate to a website of a City, County, or Unitary authority. Those websites usually contain a list of the parish councils within that larger council zone.
Sometimes a county council will have district councils beneath it, and will list parish councils within the County, Unitary or District area.
City councils will most often not have any parishes within them
Sometimes website info and other data will be given, other times not, which would need a different form of search, possibly manual, or using a different website or reference tool
The tab named "Parishes within the UK" has a long list that includes all of the UK, so some will be in Wales and Scotland that we don't need right now. The reference numbers can be used though, and it makes a useful point of reference to double check if any have been left out.
If website data isn't present, a google search may be required to find out what the website is, or it can be extracted from an email address if it is domain related email.
There maybe some data that is still left blank after automating scrapers to gain the bulk of the info, this may need some manual work to fill in those blanks.
This data may well be freely available somewhere online with a bit of searching, or it might be buried in a subfolder of a government website. You may be able to set up a script to trawl the internet for this info, up to you, but as long as an excel doc can be created with this info that is all that I need. I have made my preliminary research on this and set up an excel sheet with some data added as guidance, I would like this format to be used, although you can use one tab and Parish and Town councils so they are in one long list.
There are around 10,480 town and parish councils according to Wikipedia.
This is a task that will involve a mixture of manual work, scripts and data scrapers. It is all possible and could be automated if you wished, but there will be a fair amount of work to get it right. I would like to see a sample sent of just one area with the top level council and the parishes in that area, as a test. After that, the main body of work can be carried out. I would like it done within 10 days please.
Please read the description and do not bid unless you understand the work involved. Preference will be given to bidders that describe their method and have clearly understood the task.
Reposted due to timewaster.
I have set a guide price, but please bid your final price, whatever you feel that it ought to cost.
UPDATE: Use case document now uploaded
I need column entries for the name of the council, the website, and the number of households within that council. Each council has to be cross referenced to its respective council up or down the hierarchy, and have a reference number attributed to it. For Parish and Town Councils I need the email address for the clerk, always given on their website.
There is a hierarchy of councils in the UK, at the top are County, City, and Unitary councils. Beneath County councils there are district councils, and then Parish and Town councils. Parish and Town councils are the same thing. Beneath Unitary Councils there is no district, just Parish and Town councils.
List of all county and district councils here:
https://www.local.gov.uk/our-support/guidance-and-resources/communications-support/digital-councils/social-media/go-further/a-z-councils-online
Each link can be clicked through to navigate to a website of a City, County, or Unitary authority. Those websites usually contain a list of the parish councils within that larger council zone.
Sometimes a county council will have district councils beneath it, and will list parish councils within the County, Unitary or District area.
City councils will most often not have any parishes within them
Sometimes website info and other data will be given, other times not, which would need a different form of search, possibly manual, or using a different website or reference tool
The tab named "Parishes within the UK" has a long list that includes all of the UK, so some will be in Wales and Scotland that we don't need right now. The reference numbers can be used though, and it makes a useful point of reference to double check if any have been left out.
If website data isn't present, a google search may be required to find out what the website is, or it can be extracted from an email address if it is domain related email.
There maybe some data that is still left blank after automating scrapers to gain the bulk of the info, this may need some manual work to fill in those blanks.
This data may well be freely available somewhere online with a bit of searching, or it might be buried in a subfolder of a government website. You may be able to set up a script to trawl the internet for this info, up to you, but as long as an excel doc can be created with this info that is all that I need. I have made my preliminary research on this and set up an excel sheet with some data added as guidance, I would like this format to be used, although you can use one tab and Parish and Town councils so they are in one long list.
There are around 10,480 town and parish councils according to Wikipedia.
This is a task that will involve a mixture of manual work, scripts and data scrapers. It is all possible and could be automated if you wished, but there will be a fair amount of work to get it right. I would like to see a sample sent of just one area with the top level council and the parishes in that area, as a test. After that, the main body of work can be carried out. I would like it done within 10 days please.
Please read the description and do not bid unless you understand the work involved. Preference will be given to bidders that describe their method and have clearly understood the task.
Reposted due to timewaster.
I have set a guide price, but please bid your final price, whatever you feel that it ought to cost.
UPDATE: Use case document now uploaded
Insides
100% (25)Projects Completed
18
Freelancers worked with
15
Projects awarded
40%
Last project
8 Feb 2024
United Kingdom
New Proposal
Login to your account and send a proposal now to get this project.
Log inClarification Board Ask a Question
-
Can you please confirm the deadline for this job?
1087670
We collect cookies to enable the proper functioning and security of our website, and to enhance your experience. By clicking on 'Accept All Cookies', you consent to the use of these cookies. You can change your 'Cookies Settings' at any time. For more information, please read ourCookie Policy
Cookie Settings
Accept All Cookies