Data warehouse design and development
- or -
Post a project like this1891
£1.0k(approx. $1.2k)
- Posted:
- Proposals: 19
- Remote
- #2283481
- OPPORTUNITY
- Awarded
Android/iOS Apps & Games. PHP/MySQL. Expert Software Developer. 21 Yrs Experience
Middlesbrough
1494978205833375062325880922515715126480425443482446425254422599158175691463151
Description
Experience Level: Expert
We have a front end application which provides access to real-time data - at the same time, behind this we have a number of history tables which are used to record the state of an object when a change is made.
For example, a person is updated on 1st January - we first write a history record of their current attributes to one of 3 tables and then change the live record. In the change tables the change date is shown as 1st January.
We now have a need to be able to report on a consolidated view of the historical data for analysis - for example, we need to know the average salary of all managers in New York on 13th December 2017 and how that has changed each month for 12 months and so on.
We are looking for someone who can work with us to look at our data and make recommendations on the best architecture to make this work most effectively and flexibly.
We need to literally be able to report on ANY aggregate of any combination of "field" at any point in time, show how that has changed either prior or after that date. This might be something simple like above or could be something more complex like show me the average age of all female full time directors in finance in London who have worked here for 2 years who are paid more than 50k per year and who are left handed and show how that has changed by month for the last 3 years.
The next step beyond this will be to look at some sort of neural processing to allow us to try and find patterns but that is not in scope right now.
Our current thought process is we should create a data warehouse and push this data into there - the unknown is how we design this and how we get the data into it. Should we create a record for every person for every day or should we just create a record for month end, for example?
The front end application runs PHP and MySQL 5.7. Neither of these can change - the actual architecture for the data warehouse can be anything within reason.
For example, a person is updated on 1st January - we first write a history record of their current attributes to one of 3 tables and then change the live record. In the change tables the change date is shown as 1st January.
We now have a need to be able to report on a consolidated view of the historical data for analysis - for example, we need to know the average salary of all managers in New York on 13th December 2017 and how that has changed each month for 12 months and so on.
We are looking for someone who can work with us to look at our data and make recommendations on the best architecture to make this work most effectively and flexibly.
We need to literally be able to report on ANY aggregate of any combination of "field" at any point in time, show how that has changed either prior or after that date. This might be something simple like above or could be something more complex like show me the average age of all female full time directors in finance in London who have worked here for 2 years who are paid more than 50k per year and who are left handed and show how that has changed by month for the last 3 years.
The next step beyond this will be to look at some sort of neural processing to allow us to try and find patterns but that is not in scope right now.
Our current thought process is we should create a data warehouse and push this data into there - the unknown is how we design this and how we get the data into it. Should we create a record for every person for every day or should we just create a record for month end, for example?
The front end application runs PHP and MySQL 5.7. Neither of these can change - the actual architecture for the data warehouse can be anything within reason.
Jason D.
100% (67)Projects Completed
113
Freelancers worked with
97
Projects awarded
33%
Last project
19 Nov 2021
United Kingdom
New Proposal
Login to your account and send a proposal now to get this project.
Log inClarification Board Ask a Question
-
There are no clarification messages.
We collect cookies to enable the proper functioning and security of our website, and to enhance your experience. By clicking on 'Accept All Cookies', you consent to the use of these cookies. You can change your 'Cookies Settings' at any time. For more information, please read ourCookie Policy
Cookie Settings
Accept All Cookies