Saturday, December 15, 2007

McTool is a multithreaded Move and Copy Tool

The very first program I wrote in C (about 20 years ago now) was a program that appended one file to another. It evolved into a file copying program (DCOPY) that I shared with a few people. Since then, I have used a number of good utilities to move files around on various systems (XCopy, xXCopy, RoboCopy, TotalCopy, etc.). No one tool does everything everyone needs, but all together, perhaps they can come close. Anyway, when I couldn't find any file copy tools that supported multithreading and worked well with large numbers of small files, I wrote one.

Basically, I had tons of relatively small files (images, mp3 files, etc.) that I wanted to move around and I needed a program that could run multiple copies at once. Many systems nowadays are capable of handling multiple requests at a time (even single drives have NCQ). You no longer have to have an expensive fiber-attached SAN in order to benefit from running more than one thing at a time (a dual core processor is not even needed). Starting multiple instances of any of the tools I listed above works well to speed things up, but it was a hassle to keep things straight and I didn't want to have to bother with that all the time. Plus, at times when I have had really large sets (I've worked with millions of files, and multiple terabytes at once), the other tools just weren't able to handle it and I ran into problems.

More details and a free download link.

I rewrote McTool in C# to learn about handling threading and it uses the Microsoft .NET framework under current versions of Windows (including 2003 server, XP, Vista, 32 & 64 bit). I have not made any effort yet in putting together an installation package of any sort since it only requires two files and I think it is pretty easy to use. If I'm proven wrong by getting some feedback, that could change.

I'm pretty excited about it because it seems to work well enough in the limited testing that I've done (5-10 times faster in my tests), but I wanted to share it with others in order to see if there are any problems left that need to be fixed or other features that should be added. It seems to work for what I've been doing, but if you find this page and want to snag a copy to try out and let me know whether it works for you, that would be great! I would appreciate the feedback, and you might be able to use the tool...