Parse large number of text files to mysql database

  • Posted:
  • Proposals: 12
  • Remote
  • #2588546
  • Awarded
Ibrahim Thoaif A.
M. C.Muhammad F.Atef A.Yavor D. + 7 others have already sent a proposal.
  • 5


Experience Level: Expert
I am looking for parsing over 1000 files. Each 25 mb in size. The parser should work in linux environment. Parsing should be done in under 5 min (speed is of importance here). We have a system with 16 core and 30 gb ram with ssd storage.

The parser should parallel process all files in one folder and populated database. I would need the parser to write to mysql db. The parser should be able to create the schema on first run. On subsequent runs if any new fields are found then the parser should be able to include this to the db. The parser should also use multi threading .I have attached as example one such file and one table created.

Tables should be indexed, there are a few other details which I will provide later. The developer must be willing to tune the parser to customer satisfaction. If interested please let me know how you plan to implement this and timeframe you need

New Proposal

Create an account now and send a proposal now to get this project.

Sign up

Clarification Board Ask a Question

    There are no clarification messages.