Data Expedition, Inc.

Move Data Faster

SUPPORT

Support
Tech Notes
Configuration
Configuring Firewalls
License Binding
Anon. Win32 servedat
Windows 7 Firewall
Amazon Web Services
NAS Authentication
Diagnostic Logging
Load Balancing

Amazon Web Services

Page Index:
Instance Types
Firewall
Authentication
UDP Buffers
Network Speed
License Binding
Tech Note History
Apr172017CloudDat
Oct072014Updated Everything
Jul232014Updated UDP
Oct302013Marketplace
Elastic IP
Feb062013EBS-Optimized
Oct132011Utilization
May042011Updated Firewall
Apr222011Changing Type
Mar232011Instance ID
Dec072010Choosing
Nov172010First Post

Amazon Web Services

ExpeDat and SyncDat software can be installed on AWS EC2 instances just like any other Linux or Windows system.

This article is about installing stand-alone DEI software on EC2 instances.  For information about our AMI backed AWS Marketplace offerings, see our CloudDat site.

Choosing an Instance Type

AWS bandwidth is a shared resource.  Larger instance sizes with a higher "Network Performance" rating will more often receive a larger share of AWS bandwidth.  As with all AWS resources, performance will vary.  Below are some example instance sizes and their typical speeds, assuming your local bandwidth is sufficient.

Instance TypeEstimated Top Speed
c4.large100 to 200 megabits per second
c4.xlarge200 to 400 megabits per second
c4.2xlarge400 to 700 megabits per second
c4.4xlarge700 to 900 megabits per second

c4.2xlarge will provide good performance most of the time.  To achieve maximum performance more often, choose the c4.4xlarge instance type.  If you have a slower internet connection or do not require as much consistency, you may choose c4.large for reduced cost.

Memory (RAM) can become a factor when running a high capacity server (16 or more) with many active transfers.  For high capacity servers, consider using an m4 instance type instead of c4.

Filesystem performance will be highest for local instance (ephemeral) storage.  EBS (Elastic Block Storage) filesystems are normally limited to a few hundred megabits per second, but you can achieve high speeds by provisioning your storage as an EBS-Optimized volume.

An EC2 instance may experience severe performance limitations for several hours after changing an instance type, or mounting a new EBS volume.

For accelerated access to S3, you may subscribe to or purchase the CloudDat for AWS.  It can move data into and out of S3 objects at up to 900 megabits per second.

Performance will also be influenced by all of the factors which normally affect a computer system.  See the ExpeDat performance documentation for details.

Firewall

When installing servedat on EC2, you will need to open UDP port 8080 in the firewall.  This can be done by modifying the "Security Group" after the initial setup.

Security Group
  1. In the EC2 management console, click "Security Groups".
  2. Select the group assigned to your EC2 instance.
  3. In the frame below, select the "Inbound" tab.
  4. Click Edit.
  5. In the "Edit inbound rules" dialog, click "Add Rule".
  6. In the "Type" pop-up menu, select "Custom UDP rule".
  7. Enter a port range of "8080" (assuming you are using the default ExpeDat or SyncDat server port).
  8. For general client use, set Source to "Anywhere 0.0.0.0/0".
  9. Click "Save".

For general information about firewalls, see Tech Note 0002.

Linux System Authentication

The default Linux AMI does not assign a password to user accounts.  If you wish system users to authenticate to servedat, you must create a password for each.

While logged on as an administrative user, use the command "sudo passwd username".  For example, to create a password for the default user account:

sudo passwd ec2-user

For general information about system user authentication, see the servedat "System Authentication" manual page.

Linux UDP Buffers

The linux kernel limits UDP buffers to sizes which are not appropriate for modern high-speed networks.

If you install the servedat server, it will automatically correct the buffer sizes each time the service starts.

To permanently correct the UDP buffer limit for both clients and servers, add the following lines to /etc/sysctl.conf:

net.core.wmem_max=1048576 net.core.rmem_max=1048576

For those changes to take effect, you must either reboot or type the following commands:

sudo /sbin/sysctl -w net.core.wmem_max=1048576 sudo /sbin/sysctl -w net.core.rmem_max=1048576

See Tech Note 0024 for more about operating system UDP limitations.

Network Speeds

Amazon does not guarantee any particular level of network speed, but the following guidelines generally apply:

  • EC2 instances rated for "High" Network Performance will provide more consistent performance.
  • EBS volumes which are configured as "EBS-Optimized" will provide higher performance.
  • The fastest speeds are between instances within the same AWS region.
  • Second fastest speeds are between instances in different AWS regions.
  • Speeds to the Internet are the slowest and most variable.

Performance will also be influenced by all of the factors which normally affect a computer system.  See the ExpeDat performance documentation for details.

License Binding

Traditional host identifiers and IP addresses are not persistent in EC2 and cannot be used for MTP license binding.  Instead, the following options are available for EC2 hosted software:

  • Elastic IP  Each AWS account may hold one or more static, public IP addresses which may be assigned to individual instances.  Binding to an Elastic IP address will allow the software to run on the EC2 instance which has been assigned that address.
  • Instance ID  Each instance has an identifier in the form i-xxxxxxxx which is persistent across reboots, stop/start cycles, and type changes.  An instance ID remains constant so long as you do not terminate it.

See Tech Note 0007 for more details about host binding options.

Note that if you subscribe to CloudDat from AWS Marketplace, no host binding is required as the software is automatically billed by AWS.

More

AWS may change EC2 specifications at any time, and additional setup steps may be required when installing host-bound software on an EC2 instance.  Make sure to discuss EC2 details with your DEI representative prior to deployment.