Mongodb - populate MLabs database and create update script
- or -
Post a project like this- Posted:
- Proposals: 9
- Remote
- #3306841
- PRE-FUNDED
- Awarded
Description
Part 1; populate Mlabs mongodb database with NVD data: note, this is available in two ways;
a) API - this will require about 120 requests with 1000 items per page. Please read the NIST API attached;
or;
b) NVD files are available zipped for download (json/cve) @ https://nvd.nist.gov/vuln/data-feeds however, each file unzipped are apporxmtely 100mb (orgnaised by year) so would need to be broken down to insert into mongodb.
I do not mind how winning bidder wants to complete this task; outcome should be all available items from the NVD database are in mongodb.
Part 2: using the NIST NVD API - create and supply a script that will;
a) check mongodb for the latest/last entry for 'lastModifiedDate' field and then;
b) genrate an API request with 'modStartDate' API option using 'lastModifiedDate' to obtain all data from that date/time up to current time/date. ( Note Instructions are in the NIST NVD API document attached.)
c) The Script should then use the results from the API request (json results), process and insert into mongodb.
d) Update Script shall be supplied and should work on a linux VPS as a cron job / task to run regular updates.
(NIST update their database apporximately every two hours. Its likely there would only be 1 to 3 items in results API requests.)
Please ensure you vist the NIST website https://nvd.nist.gov/vuln/data-feeds and read the NIST API document prior to bidding.
I do not know what sort of budget is necessary for this activity. Please submit offers once you have read the requirements above.
thanks
Timur M.
100% (45)New Proposal
Login to your account and send a proposal now to get this project.
Log inClarification Board Ask a Question
-
Dear Sir,
We have gone through the NIST API documentation and we can write script in js to populates the data to our MongoDB. but it require time/expertise.
please share the deadline for this work ?
and are you flexible with the budget as proposed budget is very low for quality turnaround ?
so we can send your our valuable proposal for your consideration.
Awaiting your reply.Timur M.16 Jun 2021Hello,
re: "please share the deadline for this work ?"
no set deadline other than asap. once project is awarded, we appreciate the initial upload may take some time. We would ask the winner for the project to provide some guidance on how long they think they need to set expectations on both sides.
Re: "and are you flexible with the budget as proposed budget is very low for quality turnaround ?"
per the project description, we have indicated that we are not certain what budget is and have asked bidders to provide their best offers so yes its flexible, PPH unfortuntely forces us to pick a budget even if we dont know the level of effort/cost.
thanks -
I have an question! My script installs NVD data by downloading from the site (by parts) and then populate in db by default and then checks API for any changes and updates the db! My question is it works within VPS/any *nix based OS, is it hard requirement to use mLabs MongoDB connection?
Timur M.16 Jun 2021hello,
You are free to use any script you have to connect to Mlabs instance in order to populate the database . I dont mind how you do that. We will simply provide mlabs db credentials for your scripts.
The 2nd part of the requirement to create an update script does need to be able to run on linux vps as a cron.
thanks