Tech Note 0017
MTP/IP Integration Strategies
Guide to integrating MTP/IP technology into applications and systems
MTP/IP is designed for integration into the widest possible variety of system and environments. This flexibility is supported by multiple applications and interfaces which can be used alone or in combination. The following guidelines may assist developers in choosing which approaches are most suitable for integrating MTP/IP network acceleration technology into any given workflow.
The simplest way to integrate MTP/IP technology into an existing workflow is to embed an off-the-shelf DEI product alongside the target application. All DEI products are available in the form of small, single executables which can be called from scripts, batch files, or other executables.
ExpeDat, in particular, has many features specifically designed to enable tight integration through its command-line interfaces. These features include detailed exit codes, machine parseable logs and progress updates, standard I/O streaming, and more. The ExpeDat server can also call on third-party applications and scripts to process data or perform automation. See Tech Note 0015 for details about Embedding ExpeDat.
ExpeDat Client SDK
The ExpeDat Client Software Development Kit (EXP) is a C language library which can be used to build custom ExpeDat client software. DEI's own ExpeDat and SyncDat clients are built using EXP, giving your application access to the same functionality as our own.
EXP provides simple, high-level interfaces for communicating with an ExpeDat server. This includes sending and receiving files, authentication, encryption, directory listings, file maintenance, and even streaming file content to or from memory. Once the parent application starts a file transfer, it does not need to perform any further actions until it is notified that the transaction has completed.
The MTP Tunnel port forwarding utility captures connections from TCP clients to TCP servers, transporting the data across the network using MTP/IP as an intermediary. This allows you to accelerate off-the-shelf and legacy TCP applications without changing the existing software or workflows. Any client-server TCP session involving large amounts of continuous data flow can be captured by directing it to an outgoing tunnel port, similar to Secure Shell port-forwarding. Client applications can also be forwarded via stdio for embedded solutions.
Document Exchange SDK
The Document Exchange Software Development Kit (DOC) is a C language library which provides session management and data delivery for general purpose data transfer between networked applications. DEI's own end-user products are built on top of DOC, providing a consistent platform for inter-application communication.
The DOC SDK is focused on the idea of sending or retrieving blocks of data which may be located on disk, in memory, or in sequential streams (see Sequencing below). It provides a high-level model for both client and server networking, allowing the parent application to focus on data management. Once an application initiates or accepts a document transfer, it does not need to perform any further actions until it is notified that the transfer has completed.
The MTP Sequencing Software Development Kit (SEQ) is a C language library which allows data being delivered via MTP/IP to be buffered and accessed using a read/write data model. This allows program generated data to be sent, or incoming data to be processed sequentially and entirely within memory. DEI's own end-user products use SEQ for tasks such as compression and decompression, listing directory trees, and streaming standard I/O.
The SEQ SDK can be used as a data target for any of the other DEI Software Development Kits, including the ExpeDat Client SDK, Document Exchange, and Core MTP/IP. It can also be used for general purpose intra-process buffering, independent of the network. In addition to basic read/write functionality, it also provides asynchronous notification, random buffer access, flow-control, and the ability to examine buffer capacities. If you are used to programming against TCP/IP sockets, the SEQ interface will be at once familiar and liberating.
The Core MTP/IP Software Development Kit (Core MTP) is a C library providing access to the most basic building blocks of DEI technology. MTP/IP transactions are highly optimized to deliver maximum network performance with minimum overhead. For example, an entire network transaction can be reliably completed in a single round-trip time with no setup or tear-down overhead.
Implementing on top of the Core SDK delivers maximum efficiency in both network and CPU utilization, and maximum flexibility to achieve that efficiency for almost any network communication task. However, with that flexibility comes a great deal of design responsibility. Developers working directly with Core MTP/IP need to have a strong understanding of how packet-switched IP networks operate and experience designing network applications. MTP/IP will handle the details of flow-control and error-recovery, but it is up to the developer to decide issues such as session management, authentication, data processing, and so on.
Most MTP/IP integrators find it simplest to begin with Embedded ExpeDat. This provides the fastest integration path and, for most file transfer applications, it provides more than adequate functionality. For projects which require maximum client-side integration, the ExpeDat Client SDK provides source-code level control without requiring the programmer to delve into network level details.
Developers seeking to create or accelerate general purpose network communication paths will find that the Document Exchange and Sequencing SDKs provide that functionality with minimal network level consideration. The Core MTP/IP SDK is for advanced network programmers who wish to create their own highly-optimized communication software.
Integrators seeking to accelerate legacy applications, where source code is not available and ExpeDat's file-transfer model does not fit, may consider MTP Tunnel as a way to capture and accelerate existing TCP/IP sessions.
For more information about MTP/IP integration options, including a consultation with one of our engineers and a free trial of any SDKs, contact us.
Tech Note History
Reference ExpeDat object handlers