SmartMerge – Free Command Line Directory Merge Utility
Here is a great free utility from Jacques Bensimon.
A common operation when reorganizing or migrating a data folder in Windows is the merging of a source directory tree entirely into another. Examples include merging an old documents folder into a new one, such as when migrating an old user profile to a new one (possibly under a new profile technology), and reorganizing the contents of a remote directory, including a cloud repository (via the vendor’s “mapping” or “sync” tool). Ideally, such an operation would have both of the following features:
- Existing destination files not older than their exact counterparts under the source directory (if any) would not be overwritten.
- If both the source and destination directories are on the same local volume or remote share, the merge would only involve directory operations (“renames”) without any copying, i.e. no data movement, especially important in network or cloud scenarios.
Yet, surprisingly enough, there doesn’t seem to be any native tool in Windows that combines both features (am I missing something here?): CMD’s MOVE command and PowerShell’s Move-Item cmdlet do avoid data movement when possible, but they’ll unconditionally overwrite destination files regardless of timestamp, whereas RoboCopy, which can preserve newer files (/XO), always implements its /MOV and /MOVE switches as “copy then delete”, regardless of source and destination locations.
So, not finding a fully satisfactory third-party option (after an admittedly cursory search and a few tests), I whipped up the SmartMerge utility (with the very simple syntax highlighted in the screenshot below) that implements my two target features:
A few notes:
- Given the possibility of accidental misuse, especially if specifying the source or destination using relative references, it is highly recommended to first test your proposed syntax by adding the /? switch as a 3rd parameter. This will first test whether the effective source and destination exist as directories and whether the destination is at least minimally writeable with the current credentials, before displaying the full source and destination paths that would be merged. No files will be moved or deleted.
- SmartMerge works in two phases once source and destination are deemed valid: it first deletes files anywhere under the source directory that are not newer than a counterpart under the destination (based on their last modification timestamp), then performs an unconditional merge of what remains of the source directory tree into the target.
- If any file fails to be deleted during the first phase (because of permissions or lock status), SmartMerge will exit with a non-zero ErrorLevel and a message identifying the file in question. Some older files may at that point have already been deleted. Once the issue has been corrected, the command can be re-run to complete the process.
- If the second phase (the folder merge) doesn’t fully complete, again because of permissions or lock status in either source or destination, ErrorLevel will be non-zero and the source directory will still exist, still containing those files and subfolders that hadn’t yet been merged when the issue was encountered. All older (actually “not newer”) matching files under the source will have already been deleted by that point, and again the command can be re-run to complete (or at least further) the process once the permission or lock issue has been resolved.
- While I have tested SmartMerge in multiple environments and scenarios and confirmed that it behaves as described above, it is nonetheless a powerful tool capable of deleting or “misplacing” unintended files if used incorrectly or in unexpected scenarios, including hardware or network failures. You, therefore, assume all risks associated with any intended use of this tool, which should only be considered after thorough testing in your own environment.
Note also that both the 32-bit and 64-bit executables are compressed using MPRESS, which may trigger some anti-virus warnings or blocks. If you’re uncomfortable about whitelisting SmartMerge in that case, simply delete the executables from your system.
JB
Follow Jacques Bensimon on Twitter @JacqBens for more great Windows insights and tricks.