
Parse files into mysql database in parallel using multiple cores
- or -
Post a project like this2076
€200(approx. $227)
- Posted:
- Proposals: 13
- Remote
- #2475635
- OPPORTUNITY
- Awarded
IT Consultant |Virtual Assistant| graphics| | Website design | social media management .

Web Developers and Designers | E commerce | Java | Mobile Apps | Automation Testing | Quality Assurance
1244165461086261915518807352098966255109722614152681362255284162887024668942275501





Description
Experience Level: Expert
Hi,
I am looking to parse a few thousand text files into mysql database, below are some of the parser criteria
1. Since file are large in number parsing should be done in parallel
2. The parser should go into each file and build a schema when parsing for the first time
3. subsequent parsing the parser should truncate all existing data and be able to find new fields and add to tables if found in the files
4. tables should be indexed
5. While parsing the parser should display time taken to parse each file
6. Once parsing is finished total parsing time should be displayed
7. all tables and fields should have a name prefix (eg. t_ for tables and f_ for fields) to avoid mysql reserved names. The prefix and database connection may be specified in a config file
8. the parser will be run on a linux machine with 16 cores. parser should be built using java
Sample files and parsing requirements will be provided
we currently have a similar java parser for xml files (can provide source code if required)
I am looking to parse a few thousand text files into mysql database, below are some of the parser criteria
1. Since file are large in number parsing should be done in parallel
2. The parser should go into each file and build a schema when parsing for the first time
3. subsequent parsing the parser should truncate all existing data and be able to find new fields and add to tables if found in the files
4. tables should be indexed
5. While parsing the parser should display time taken to parse each file
6. Once parsing is finished total parsing time should be displayed
7. all tables and fields should have a name prefix (eg. t_ for tables and f_ for fields) to avoid mysql reserved names. The prefix and database connection may be specified in a config file
8. the parser will be run on a linux machine with 16 cores. parser should be built using java
Sample files and parsing requirements will be provided
we currently have a similar java parser for xml files (can provide source code if required)

Three (PPH Corporate) A.
100% (41)Projects Completed
36
Freelancers worked with
18
Projects awarded
62%
Last project
15 Apr 2021
Ireland
New Proposal
Login to your account and send a proposal now to get this project.
Log inClarification Board Ask a Question
-
Three
Is this a one off or cron need to be setup and is your budget flexible?
831083
We collect cookies to enable the proper functioning and security of our website, and to enhance your experience. By clicking on 'Accept All Cookies', you consent to the use of these cookies. You can change your 'Cookies Settings' at any time. For more information, please read ourCookie Policy
Cookie Settings
Accept All Cookies