creacominc / mtfilecopy Goto Github PK

for large collections of small files, this wastes a lot of time re-creating the file for every entry.

It can take 2 seconds to write 106k records on a slow Drobo array. What if the drive went offline during that write, the queue would be lost. We should write to a temporary file and move it into place only when the write is completed.

We should find a way to lock the files so that multiple threads or processes do not attempt to move or copy the same source file.

I see the following as a file it is trying to rsync: "/Volumes/CCMv/copyOfParalandra/VideoProjects/VideoProjects.fcpbundle/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/__AsyncCopying/2014-03-20 19_57_13 (id)-tTqWGs.mov" "/Volumes/BackupCCMv/CCMv/copyOfParalandra/VideoProjects/VideoProjects.fcpbundle/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/__AsyncCopying"

The process of reading the folder needs to avoid following links.