Installation

Amazon Web Services

REQUIREMENTS

To use CloudDat, you will need the following:

  • An Amazon Web Services account
  • An S3 bucket
  • EC2 SSH Keys for the bucket's region
  • A high-speed Internet path to Amazon Web Services
  • A Secure Shell (SSH) client

SUBSCRIPTION VS PERPETUAL LICENSE

If you purchase a perpetual license for CloudDat, you will receive ExpeDat and CloudDat-AWS packages from DEI.  The CloudDat-AWS package contains installation instructions in the enclosed README file.

The steps below are for a CloudDat subscription obtained through AWS marketplace.  When launching a CloudDat instance in AWS Marketplace, you will be offered two choices of "Delivery Methods".  New users are strongly encouraged to choose the CloudFormation method labeled "CloudDat".  Instructions for both methods are provided below.

The method labeled "Single AMI" launches the instance without any setup.  You must manually create and attach an IAM Role and security group prior to launch, then use Secure Shell to complete instance setup.

This is not recommended for new users.

  1. Verify that you have all of the requirements listed above.
  2. Choose an S3 account and bucket to access.
    Note which region hosts the bucket.  Buckets in the "Standard" region are "US East (N. Virginia)".
  3. Create an IAM Role
    Use the AWS "Identity & Access Management" console to create or choose an IAM role.  Attach a policy which allows access to S3 in general, or the particular S3 bucket(s) you will use with the gateway.  This step is done automatically if you choose the Cloud Formation method.
  4. Go to the AWS Marketplace and choose the Hourly, Monthly, or Monthly High Capacity CloudDat product.  Choose the Hourly product for the 7-day free trial.
  5. Click the "Continue to Subscribe" button at the top of the page to proceed to the Launch page.  Note that the button may say this even when you are already subscribed. Continue Button
  6. If this is your first time subscribing to this product, or if you have previously cancelled your subscription, click the "Accept Software Terms" button. Accept Button
  7. The next screen will verify that you are subscribed and may display other offers, terms, and details.  Click "Continue to Configuration". Continue to Configuration
  8. Select a "Fulfillment Option" by clicking on the dropdown menu and choosing "Amazon Machine Image".  If drop-down menus interfere with your ability to make a selection, try making your browser window wider. AMI
  9. Verify that the latest "Software Version" is selected.
  10. In the "Region" box, select the AWS region where your S3 bucket is located. Region
  11. Click "Continue to Launch" at the top of the page. Continue to Launch
  12. In the "Choose Action" box, select "Launch through EC2". Launch AMI
  13. You should now be directed to the EC2 launch step "Choose an Instance Type".  You may need to scroll down to find the available instance choices.  Refer back to the AWS Marketplace page for available types and pricing.  Choose c4.4xlarge for the best performance (up 2 gigabits per second).  Choose smaller instances for lower costs. Instance Type
  14. Next, "Configure Instance Details".  In the middle of the page, select an "IAM Role" from the pop-up menu.  If you do not already have a role providing S3 access, click "Create a new IAM role", then return to this page and click the Refresh button to make that new role available. IAM Role
  15. Click past "Add Storage" to "Tag Instance".  Here you may enter a custom name for your gateway. Instance Type
  16. Click next to "Configure Security Group".  Refer back to the AWS Marketplace page to make sure that you have all three Inbound rules set. Security Group
  17. Click "Review and Launch" then "Launch".  If you receive a "Launch Failed" error, return to the "Choose an Instance Type" screen and make sure you have selected one of the instance types shown here. Security Group Security Group
  18. Return to the Instance list and wait for the Instance to report Ready.
  19. Zoom for Screenshot

    Once the instance is ready, ssh to the ec2-user account on that instance.  When the instance is first being launched, the setup script will run automatically.  Later, you can run it manually by executing /usr/local/expedat/s3setup.sh.  Click the screenshot on the right to see an example.
  20. The script will identify the IAM Role attached to the gateway.  If the role cannot be found or is incorrect, AWS requires that you terminate the instance and start over.
  21. Next, the script will ask for the name of the S3 bucket you wish to access.  Do not enter the endpoint URL, just the name.  The script will test whether the bucket exists, is accessible, and is in the same region as the gateway.  If a problem is found, follow the instructions or contact us for help.
  22. Finally, the script will ask you to enter a username and password for your initial gateway account.  These are the credentials end-users will enter into ExpeDat clients to access your S3 bucket.  You may create additional accounts later.
  23. Zoom for Screenshot

    Upon successfully completing the setup script, the URL of the gateway will be shown.  Direct a web browser to that instance public address.  An example page is shown on the right.
  24. Download a client.
    Choose Windows or Mac for the graphical client and extract the contents of the ZIP package after download.  A command line client is included in the "Full Client Package" links near the bottom of the page.
  25. Zoom for Screenshot

    Run the client.
    No installation is required.  You may move the client executable to any location on your system before running it.
  26. Back in the web browser, click "Access your CloudDat Gateway for Amazon S3".
    The ExpeDat Desktop client will prompt you to enter the password you created earlier.  For command line clients, see the Client Use page.

After clicking the Access link, the client will list the contents of your S3 bucket (if any) and you can begin transferring files.  You may wish to bookmark the link for easy access later.  The next page describes more about using clients.  See the Support page if you the client has trouble reaching the CloudDat instance.

Next: Client Use >

The method labeled "CloudDat" uses AWS Cloud Formation to perform the initial gateway setup on AWS' website and automatically create an IAM Role for S3 access and a security group for network access.  This is the recommended choice.

  1. Verify that you have all of the requirements listed above.
  2. Choose an S3 account and bucket to access.
    Note which region hosts the bucket.  Buckets in the "Standard" region are "US East (N. Virginia)".
  3. Go to the AWS Marketplace and choose the Hourly, Monthly, or Monthly High Capacity product.  Choose the Hourly product for the 7-day free trial.
  4. Click the Continue button at the top of the page to proceed to the Launch page. Continue Button
  5. If this is your first time subscribing to this product, or if you have previously cancelled your subscription, click the "Accept Software Terms" button. Accept Button
  6. The next screen will verify that you are subscribed and may display other offers, terms, and details.  Click "Continue to Configuration". Continue to Configuration
  7. Select a "Fulfillment Option" by clicking on the dropdown menu and choosing "CloudFormation".  If drop-down menus interfere with your ability to make a selection, try making your browser window wider. Fullfilment CloudFormation
  8. Verify that the latest "Software Version" is selected.
  9. In the "Region" box, select the AWS region where your S3 bucket is located. Region
  10. Click "Continue to Launch" at the top of the page. Continue to Launch
  11. In the "Choose Action" box, select "Launch CloudFormation". Launch Cloudformation
  12. Click the "Launch" button. Launch
  13. You will now be taken to the start of the launch workflow.  The CloudFormation template URL will already be specified, click "Next" to continue. Launch Next Button
  14. Click to Zoom

    Now you can configure your CloudDat instance.  Most settings may be left with their default values or the first option in the drop-down menu.  Click the screenshot on the right to see an example.
    • Stack name: Optionally name this collection of CloudDat resources.
    • Instance Type:   Use instance size c4.4xlarge for maximum performance, but if your internet connection is less than a gigabit per second, you may choose a smaller instance for reduced cost.
    • VpcId: If you do not already have a VPC, choose any of the default values.
    • Subnet: If you do not already have a VPC, choose any of the default values.
    • Key Pair: Choose the SSH key which you will use to access the CloudDat instance for administration and advanced configuration.
    • Restrict Client Access: Set a subnet mask to limit file transfers, or enter "0.0.0.0/0" to allow file transfers from anywhere.
    • Restrict Admin Access: Set a subnet mask to limit SSH access, or enter "0.0.0.0/0" to allow admin access from anywhere.
    • Existing S3 Bucket: Type the name of your S3 bucket.  Do not enter the endpoint URL, just the name.
    • ExpeDat Username: Choose a name for your initial file transfer user account.
    • ExpeDat Password: Select a password for your initial file transfer user account.
    Click "Next" once you have completed the configuration form.  If you receive errors, check that you have completed all form elements according to the instructions above and try again.  Remember that you can always contact us for help. Next Button
  15. There are no changes required on the Options page, click "Next" again.
    Do not enter an IAM Role here: one will be created for you, which you can modify later.  If you need to apply a pre-existing IAM role, go back and follow the Single AMI instructions instead. Next Button
  16. On the Review page, click the checkbox at the bottom to allow the creation of an IAM role which will allow the CloudDat instance to access the bucket you named.  Then click "Create". Create Button Create Button
  17. You will now be taken to the Cloud Formation page.  This page sometimes loads before the CloudDat stack has started and may say "Showing 0 stacks".  If that hapens, click the Reload button at the top right to find the new CloudDat stack. Reload Create In Progress
  18. It may take several minutes for AWS to deploy the CloudDat resources.  Once the Status shows "CREATE_COMPLETE", select the CloudDat stack line and open the "Resources" section.  Click on the instance id in the "Physical ID" column. Resources
  19. You should now see the CloudDat instance in the EC2 console.  Once the Instance State reaches "running", copy the Public IP address to a web browser. Reload
  20. Zoom for Screenshot

    Your web browser should now show the CloudDat instance landing page.  If the page does not load, check that the instance is running and that its security group allows TCP port 80.  If the page displays yellow warnings, or the left-side "Server Status" is red, contact us for help.  Click the screenshot on the right for an example of what the page should look like.
  21. Download a client.
    Choose Windows or Mac for the graphical client and extract the contents of the ZIP package after download.  A command line client is included in the "Full Client Package" links near the bottom of the page.
  22. Zoom for Screenshot

    Run the client.
    No installation is required.  You may move the client executable to any location on your system before running it.
  23. Back in the web browser, click "Access your CloudDat Gateway for Amazon S3".
    The ExpeDat Desktop client will prompt you to enter the password you created earlier.  For command line clients, see the Client Use page.

After clicking the Access link, the client will list the contents of your S3 bucket (if any) and you can begin transferring files.  You may wish to bookmark the link for easy access later.  The next page describes more about using clients.  See the Support page if the client has trouble reaching the CloudDat instance.

Next: Client Use >

Free Trial

Try before you buy

We're so confident in our technology that all of our end-user products are available to try - no strings attached.

Request Trial