Copying Folders

Transfer the contents of a directory by specifying either the "-i" (streaming) or "-h" (one-at-a-time) option.

To Download:

    movedat -i user@example.com:remote_dir local_dir or  movedat -h user@example.com:remote_dir local_dir

To Upload:

    movedat -i local_dir user@example.com:remote_dir or  movedat -h local_dir user@example.com:remote_dir

When copying a folder, the contents of the source folder are placed inside the destination folder.  The destination folder must exist and must be writeable.  Hidden files, hidden folders, and the contents of hidden folders are not included unless you enable the ShowHidden (-H) option or include each explicitly on the command line.

Streaming versus One-At-A-Time

Streaming mode (-i) transfers the entire contents of a folder as a single real-time data stream.  One-at-a-time mode (-h) first scans the source folder, checks for local conflicts, then copies each file and subfolder individually.

Streaming mode is faster for many small files.  One-at-a-Time mode provides more efficient error recovery.

Streaming is faster when there are a very large number of files or the average file size is small, because it does not take the time to scan the source and does not wait for each file to be written to disk before sending the next one.  Any files at the destination with conflicting names may be silently overwritten.  If the network or server is disrupted during a Streaming session, auto-retry will attempt to restart from the beginning.  If any individual file cannot be read or written, the entire transfer may be aborted.  Unix symbolic links will be copied to unix systems and ignored by Windows systems.  Compression is not available with Streaming.  Streaming Folders requires only a limited amount of memory (see buffer size), allowing up to 2 billion files to be transferred per session.

For paths which are very fast or high latency, Streaming performance may be affected by the buffer size.

One-at-a-time will be almost as fast as Streaming when the average file size is large enough to take several seconds each, and the number of files is only a few hundred or less.  One-at-a-time can warn of conflicts and may prompt the user for resolution unless "-y" or "Yes" is set.  If the network or server is disrupted during a one-at-a-time session, auto-retry will attempt to resume from the point of disruption.  If an individual file cannot be transferred or retried, the remaining files may still be transferred.  Unix symbolic links to regular files will be followed and the target file copied.  Symbolic links to folders and broken links will be skipped.  Compression is available with one-at-a-time directory transfer.  The number of files which can be transferred with one-at-a-time is limited by available client memory.

For server-side object handlers which support folder transfer, one-at-a-time is used regardless of the Options setting.

Server Behavior

When uploading one-at-a-time, conflicts with remote files will only be logged when the NoOverwrite setting has been enabled.  Otherwise, uploads will silently replace conflicting files.

Streaming transfers are considered a single transaction and are logged as such by the server.  One-at-a-time transfers are marked as "bulk".  The server may filter out logging of individual bulk transfers, by setting LogFilter to a non-zero value.  Remote directory scans and directory creations will be logged as normal transactions during one-at-a-time.  If NoOverwrite is set, a streaming folder upload will fail if the top-level folder exists, regardless of its contents.

Feature Summary

Below is a summary of the behavioral differences between Streaming and One-at-a-Time directory transfer:

FeatureStreamingOne-at-a-Time
Transfer SpeedFastVariable
Pre-scan SourceNoYes
Conflict DetectionNoYes
Checkpoint RestartNoYes
Continue After InterruptNoYes
Continue After File ErrorNoYes
Compression AvailableNoYes
Object HandlersNoYes
Client Memory UseLowVariable
Unix Symlink to FileCopiedFollowed
Unix Symlink to FolderCopiedIgnored
Hidden/Invisible ItemsOptionalOptional
UTF-8 File NamesYesYes
Modified DateYesYes
Creation DateYesNo
Locked/Read-OnlyYesYes
Unix UID/GIDYesYes
Log Each FileNoOptional
Keep Existing FilesNoOptional

These behaviors may change in future versions.