Known Issues

The following issues and limitations are known to exist in SyncDat 1.2I or the systems which it supports.  Also refer to the Limitations section.  For further information about these issues, please contact DEI Technical Support.

Network Attached Storage Maximum File Sizes Memory Limitations
Special Characters Maximum Path Length Non-ASCII Characters
Windows Performance Small MTU Resuming Partial Downloads
File Access Conflicts System Clock Accuracy Windows Symbolic Links
Irregular Files Checkpoint Files "Lost" Transactions
Server Restarts Slow Devices

Network Attached Storage

Transferring files which are stored on NAS devices may be severely limited by the speed of the legacy protocols used by those devices.  If you must use NAS, see Tech Note 0029 for guidance on tuning NAS devices for high performance.  See the Performance chapter for general guidance on storage performance.

Maximum File Sizes

Individual files transferred by SyncDat must be less than 8 exabytes each, or the maximum file size supported by the host operating system.  The total size of all files transfered per client session must be less than 16 exabytes.

Memory Limitations

The maximum number of files which can be synchronized in a single syncdat session is limited by available client system memory (RAM).  This is typically between one and eight million for systems with at least 4 gigabytes of free RAM.

AIX systems may limit available memory, which in turn limits the number of files which may be synchronized.  To increase the available memory for syncdat under AIX, set the LDR_CNTRL environment variable as follows:
  export LDR_CNTRL=MAXDATA=0x8000000

Special Characters

Certain characters should not be used in file and folder names because they have special meaning on some platforms.  When transferring files between different platforms is is possible for a file name which is valid in one context to cause problems in another.  Avoid the following characters in file names:

/   \   :   *   ?     "   '   `   ;   (   )   [   ]   &   |   !   ~   {   }   @   $

For example, if you try to upload a file named "recipes:recent" to a Windows NTFS filesystem, the file or its contents may seem to disappear and servedat may log an error because Windows reserves the ':' character for Alternate Data Streams.

Maximum Path Length

SyncDat limits the total length of a file's pathname to 1024 bytes, including UTF-8 encoding.  Some operating systems, particularly Windows, may impose much shorter limits on pathnames for some operations.  This varies greatly depending on the operating system version, but it is generally safest to keep Windows paths below 256 characters.  Network attached storage devices (NAS/SAN) may also impose shorter limits.

Unicode / Non-ASCII Characters

On all platforms, non-ASCII filename characters are encoded as UTF-8 and will be preserved between platforms.

The ability of SyncDat software to display non-ASCII characters will depend upon the fonts and settings of the host system.  This is particularly true for the Windows console.  See the syncdat introduction for steps to improve the display of non-ASCII characters with the Windows console.

In some cases, the Windows version of SyncDat may not correctly display non-ASCII characters even when unicode fonts are available.  However, SyncDat will correctly preserve unicode file name characters even when they are not displayed correctly.

Some systems allow file names to be arbitrary binary strings, which may not be valid unicode.  SyncDat will attempt to preserve such file names, but they are likely to display incorrectly and may cause errors.

If either the source or destination filesystem is a network mount, support for unicode file name characters will depend on the type and configuration of the network mount.

Usernames and passwords should consist of only ASCII letters, numbers, and printable symbols.  The use of other characters, such as extended unicode characters, may work in some environments but is not assured and may compromise security.

Windows Performance Limits

Windows has a fixed limit on the amount of memory available for buffering file and network operations.  When the total amount of I/O occurring in the system exceeds this level, operations begin to fail and Windows may become unstable and require a reboot.  SyncDat typically reports this as:

Insufficient system resources exist to complete the requested service

The problem may be triggered by file transfers totaling more than a hundred megabits per second, by multiple applications performing high volume I/O operations, or by the presence of third-party anti-virus, firewall, or virtualization software.  Once Windows' paged pool memory has been exhausted, it is necessary to reboot Windows to prevent application crashes and other system-wide problems.  The following Microsoft article explains how to improve paged pool performance, which may allow faster throughput:

http://support.microsoft.com/kb/312362

If it is not possible to adjust the paged pool, or if doing so does not solve the problem, it may be necessary to disable other software, limit SyncDat's speed, or reconfigure a virtualized environment.

Windows XP and Server 2003 are particularly vulnerable to this problem, but it has also been reported with Windows Vista and Windows Server 2008.  At this time it is not known to what extent later versions of Windows may be susceptible to this problem.

Small MTU

Network links with a Maximum Transmit Unit (MTU) smaller than the common 1500 byte minimum will cause datagrams to be fragmented unless fragmentation has been disabled in the router.  IP fragmentation will magnify packet loss on congested or unreliable networks, resulting in loss rates that are two or three times greater than they would be without fragmentation.  This may cause poor performance with SyncDat.

If IP fragmentation is disabled, then the small MTU will be automatically detected.  If fragmentation cannot be disabled on a link with a small MTU, then use the MaxDatagram setting of servedat, or syncdat to limit the size of SyncDat's datagrams.  Choose a value which is about 100 bytes less than the MTU, or experiment to find the value which achieves the best performance.

Resuming Partial Downloads

If a download is interrupted, it will be restarted from the beginning upon retry.  Interrupted uploads may be resumed from the point of interruption.

File Access Conflicts

The SyncDat server takes steps to ensure that files being uploaded are not written to or read from by other transactions.  However, the mechanisms for this protection are highly dependent upon the operating environment and are not perfect.  In particular, files on network storage devices may be susceptible to modification by other programs while in use.  Users should take care not move or modify files which may be actively transferring.

System Clock Accuracy

Severe problems may occur if the system clock of a host running SyncDat software is not accurate or changes substantially.  See the "System Clocks" section for details.

Windows Symbolic Links

Symbolic Links under Windows Vista and Windows 7 are not replicated.  Instead, their targets will be replicated.

In a unix shell environment, the syntax "path1/symlink/../path2/" would access the directory "path2" in the parent directory of the symlink target.  Using such a path via servedat will instead access the directory "path2" in the parent directory of the symlink itself.  In other words, the path would be resolve to "path1/path2/".

Irregular Files

Irregular or "special" files cannot be synchronized.  If an irregular file (device, socket, pipe, etc.) already exists at a destination path, then the synchronization may fail.

Checkpoint Files

The server stores partial uploads using "-sv.tmp" and "-sv.met" files.  See the Upload Status chapter for details.

"Lost" Transactions

If a transaction fails while the server is under heavy load and the network path is experiencing packet loss, a client may receive a "Lost Transaction" error instead of the exact error message.  The exact error message will appear in the server logs.

Server Restarts

Restarting the server via SIGHUP, which is necessary to reload the configuration and private authentication files, will interrupt any active transactions.  Interrupted transactions will automatically retry.

Slow Devices (CD, DVD, tape, NAS over WAN, etc.)

Targeting files on very slow filesystems or devices such as optical disks, tape drives, network attached storage across a WAN, or any other filesystem with extreme latency may result in severely reduced performance or loss of connectivity.  Targeting slow filesystems on the server may affect the performance and connectivity of all clients accessing that server, even those targeting other filesystems.