Command-line tool for database diffs
- or -
Post a project like this3463
$250
- Posted:
- Proposals: 2
- Remote
- #595803
- Awarded
Description
Experience Level: Expert
Estimated project duration: 1 - 2 weeks
------------------------------------------------------------------
Opportunity
------------------------------------------------------------------
This is a fantastic opportunity for a seasoned software developer to work on a project which will have an impact in the open source community. All the source code from this project will be released for free for other developers to use and adapt. There may also be ongoing paid work on this project to create updates and add features
------------------------------------------------------------------
Developer Requirements
------------------------------------------------------------------
We are looking for someone who:
* Uses Git regularly for version control
* Is familiar with command-line (CLI) tools
* Is familiar with MySQL/SQL syntax
* Knows a little bit about database migrations, or is willing to learn!
* Puts in a proper bid with solid examples of previous software work done. Generic proposals/messages will be rejected
------------------------------------------------------------------
The Problem
------------------------------------------------------------------
I've found a few solid migration tools like Flyway (https://github.com/flyway/flyway) and Simple DB Migrate (https://github.com/guilhermechapiewski/simple-db-migrate), the latter being my preference for it's simplicity but the former having a lot more commits and contributors. However, these are just migrators and do not help produce the actual diff/migration to be versioned, which I would not like to do manually if it can be automated.
I found a diff tool by MySQL called mysqldiff (http://dev.mysql.com/doc/mysql-utilities/1.3/en/mysqldiff.html), which outputs SQL for (some) schema and data changes but it shockingly doesn't produce valid SQL!!!
I must say though everything the mysqldiff tool offers is great, doing the diff by connecting directly to a source and target database (can be locally or on another server) and doing a series of tests/checks (all of which can be skipped if need be), then checking both the schema and the data - but we just need it to produce valid SQL and I think it also had trouble in validating the schema fully.
I actually looked through A LOT of different schema and data diff tools (took me a whole weekend) and they all either 1. only do schema diffs, 2. are just too complex to get working or 3. just don't produce valid SQL.
------------------------------------------------------------------
The Solution
------------------------------------------------------------------
I think a solid migration tool mixed with an automated schema and data diff tool would be a great contribution to the open source community and exactly what we need for our project now. Can just work for MySQL for now.
Let’s call this tool the dbdiff CLI.
------------------------------------------------------------------
Specification
------------------------------------------------------------------
In summary, we would like a tool which can:
* Work on the Linux & Mac command-line/Terminal (can be developed in your choice of programming language e.g. Node.js, Java, Python etc. and can include as many other open source plugins as you like, but cannot include any commercial plugins)
* Connect to a source and target database, or two SQL files made by mysqldump, to do the comparison diff
* Diffs can include changes to the schema, data, both in valid SQL to bring the target up-to-date with the source
* Since this diff tool is being used for migrations, it must be able to provide up and down SQL in the same file e.g. The up SQL may include adding a row of data, but the down SQL would involve deleting that same row
* Work with existing migration tools like Flyway and Simple DB Migrate by specifying output template files/formats, for example, Simple DB Migrate may work with simple-db-migrate.tmpl which includes: SQL_UP = u""" {{ up }} """ SQL_DOWN = u""" {{ down }} """
* Is Unicode aware, can work with UTF8 data, which includes foreign characters/symbols
* Works with just MySQL for now, but can easily be expandable to other DBs in the future
* Common tests should be included in a ‘test’ folder as the tool is being developed
* Must be developed in an open environment via a Git repository which we will provide the details to
------------------------------------------------------------------
Further Details
------------------------------------------------------------------
Further documentation will be provided as necessary to the chosen developer
Opportunity
------------------------------------------------------------------
This is a fantastic opportunity for a seasoned software developer to work on a project which will have an impact in the open source community. All the source code from this project will be released for free for other developers to use and adapt. There may also be ongoing paid work on this project to create updates and add features
------------------------------------------------------------------
Developer Requirements
------------------------------------------------------------------
We are looking for someone who:
* Uses Git regularly for version control
* Is familiar with command-line (CLI) tools
* Is familiar with MySQL/SQL syntax
* Knows a little bit about database migrations, or is willing to learn!
* Puts in a proper bid with solid examples of previous software work done. Generic proposals/messages will be rejected
------------------------------------------------------------------
The Problem
------------------------------------------------------------------
I've found a few solid migration tools like Flyway (https://github.com/flyway/flyway) and Simple DB Migrate (https://github.com/guilhermechapiewski/simple-db-migrate), the latter being my preference for it's simplicity but the former having a lot more commits and contributors. However, these are just migrators and do not help produce the actual diff/migration to be versioned, which I would not like to do manually if it can be automated.
I found a diff tool by MySQL called mysqldiff (http://dev.mysql.com/doc/mysql-utilities/1.3/en/mysqldiff.html), which outputs SQL for (some) schema and data changes but it shockingly doesn't produce valid SQL!!!
I must say though everything the mysqldiff tool offers is great, doing the diff by connecting directly to a source and target database (can be locally or on another server) and doing a series of tests/checks (all of which can be skipped if need be), then checking both the schema and the data - but we just need it to produce valid SQL and I think it also had trouble in validating the schema fully.
I actually looked through A LOT of different schema and data diff tools (took me a whole weekend) and they all either 1. only do schema diffs, 2. are just too complex to get working or 3. just don't produce valid SQL.
------------------------------------------------------------------
The Solution
------------------------------------------------------------------
I think a solid migration tool mixed with an automated schema and data diff tool would be a great contribution to the open source community and exactly what we need for our project now. Can just work for MySQL for now.
Let’s call this tool the dbdiff CLI.
------------------------------------------------------------------
Specification
------------------------------------------------------------------
In summary, we would like a tool which can:
* Work on the Linux & Mac command-line/Terminal (can be developed in your choice of programming language e.g. Node.js, Java, Python etc. and can include as many other open source plugins as you like, but cannot include any commercial plugins)
* Connect to a source and target database, or two SQL files made by mysqldump, to do the comparison diff
* Diffs can include changes to the schema, data, both in valid SQL to bring the target up-to-date with the source
* Since this diff tool is being used for migrations, it must be able to provide up and down SQL in the same file e.g. The up SQL may include adding a row of data, but the down SQL would involve deleting that same row
* Work with existing migration tools like Flyway and Simple DB Migrate by specifying output template files/formats, for example, Simple DB Migrate may work with simple-db-migrate.tmpl which includes: SQL_UP = u""" {{ up }} """ SQL_DOWN = u""" {{ down }} """
* Is Unicode aware, can work with UTF8 data, which includes foreign characters/symbols
* Works with just MySQL for now, but can easily be expandable to other DBs in the future
* Common tests should be included in a ‘test’ folder as the tool is being developed
* Must be developed in an open environment via a Git repository which we will provide the details to
------------------------------------------------------------------
Further Details
------------------------------------------------------------------
Further documentation will be provided as necessary to the chosen developer
Jasdeep K.
87% (3)Projects Completed
5
Freelancers worked with
4
Projects awarded
33%
Last project
22 May 2015
United Kingdom
New Proposal
Login to your account and send a proposal now to get this project.
Log inClarification Board Ask a Question
-
There are no clarification messages.
We collect cookies to enable the proper functioning and security of our website, and to enhance your experience. By clicking on 'Accept All Cookies', you consent to the use of these cookies. You can change your 'Cookies Settings' at any time. For more information, please read ourCookie Policy
Cookie Settings
Accept All Cookies