Need help with pyspark and aws s3
- or -
Post a project like this£25/hr(approx. $31/hr)
- Posted:
- Proposals: 0
- Remote
- #2165422
- Expired
Description
Experience Level: Expert
Business Domain: Estate management, service apartments
Project Background: All the transaction data from the CRM is stored in oracle. The project is to build a Data lake with this data in AWS S3. There is a one time data migration followed by daily incremental feeds.
Data types: There is one time historical data migration followed by daily incremental extracts in CSV format
Tools/Technologies: Python, Spark, AWS glue, Athena and S3
Data modules: Customer, Bookings, Payments, Properties, movements (some of the customers shift to a different service apartments), Staff, Incidents (any issues at property like thefts, repairs)
Need someone to convert the scripts from Spark SQL to Pyspark.
Project Background: All the transaction data from the CRM is stored in oracle. The project is to build a Data lake with this data in AWS S3. There is a one time data migration followed by daily incremental feeds.
Data types: There is one time historical data migration followed by daily incremental extracts in CSV format
Tools/Technologies: Python, Spark, AWS glue, Athena and S3
Data modules: Customer, Bookings, Payments, Properties, movements (some of the customers shift to a different service apartments), Staff, Incidents (any issues at property like thefts, repairs)
Need someone to convert the scripts from Spark SQL to Pyspark.
Pavan C.
0% (0)Projects Completed
-
Freelancers worked with
-
Projects awarded
0%
Last project
10 May 2024
United Kingdom
New Proposal
Login to your account and send a proposal now to get this project.
Log inClarification Board Ask a Question
-
There are no clarification messages.
We collect cookies to enable the proper functioning and security of our website, and to enhance your experience. By clicking on 'Accept All Cookies', you consent to the use of these cookies. You can change your 'Cookies Settings' at any time. For more information, please read ourCookie Policy
Cookie Settings
Accept All Cookies