EC2 Instances

As with all aspects of AWS, performance will vary depending on the resources you allocate.  Below are the maximum upload speeds and numbers of concurrent uploads to be expected for each instance type:

Instance Type	Max Upload	Max #
c5n.large	1 Gbps	4
c5n.xlarge	2 Gbps	8
c5n.2xlarge	3 Gbps	16
c5n.4xlarge	5 Gbps	32
c5n.9xlarge	5 Gbps	64
c5n.18xlarge	5 Gbps	128
Regions without support for c5n may use c5.

Speeds are based on encrypted uploads over the internet to S3 storage.  AWS throttles EC2 internet bandwidth to 5 gigabits per second per instance.  Use multiple instances for faster total bandwidth.

Maximum concurrency is based on each S3 upload consuming 1 gigabyte of instance memory for buffering.  Exceeding the available memory of an instance may result in reduced performance or system failures.  See S3 Upload Tuning to adjust memory usage.

Folders

Folders do not exist in S3 buckets.

For the purpose of listing remote files, pseudo-folders are displayed by grouping together objects sharing common name prefixes ending in "/".

While pseudo-folders allow you to browse the contents of a bucket as though it were organized into folders, some folder related functions will not work with S3.

• The "Rename" function cannot be applied to S3 pseudo-folders.
• The "New Folder" function is silently ignored.

To use ExpeDat Desktop to upload objects into a pseudo-path that does not yet exist, type the path in the "Remote Prefix" field prior to clicking "Send".

Listing a bucket containing many objects may take a long time, even if they are grouped into pseudo-folders.

Network

Data cannot move faster than your underlying network and storage hardware.

The EC2 instance hosting CloudDat must be in the same availability region as the S3 bucket.  Accessing a bucket in a different region than the gateway instance would severely limit performance.

Because each transaction must establish communication between the EC2 instance and S3, it may take several seconds for each action to start and finish.  If you are storing large numbers of files in S3, consider packaging related files as a ZIP or TAR archive, rather than storing each file as a separate object.  Remember that S3 objects are not files and you must plan your workflow around S3's limitations.

Reporting

Clients display the progress of uploading data to the CloudDat instance.  It may take several seconds more for the data to then finish being written into S3.  There may be times when S3 takes up to a minute to confirm receipt of the data.