Professional Documents
Culture Documents
Data Transport Standard
Nathan Chitty PESC
Gary Sandler PESC
Data Transport Standard DTS
• DTS uses Internet technologies to
facilitate real time data exchange and
transaction processing
• DTS builds on stable technologies, not
specific products
• DTS, once implemented, reduces
programming and pertransaction costs
through standardization 2
DTS Defined
• Data Transport Standard is a
specification not a product
• Established by Postsecondary
Education Standards Council (PESC)
for exchanging data for:
– Inquiries
– Reports
– Transactions 3
DTS Defined
• A specification for an adjunct to or
a replacement for existing data
transport mechanisms
– PGP / GnuPG encryption
– SecretAgent w/ Email
– FTP and SecureFTP
4
DTS Benefits
• A Web Services implementation
– Delivery confirmation included – no
guessing
– All requests get a response
– All submissions get an answer of
some kind
• Facilitates real time data exchange 5
DTS Benefits
• Includes automatic data encryption
• Uses digital signature standards
• Platform independent
• Strong authentication with non
repudiation
6
Benefits To “System Providers”
• Add value to schools’ systems
– Schools want transport added to systems
and are generally not concerned with the
technologies
• Easier to build one transport protocol
for all recipients
– Just as CommonRecord created the drive
to build one XML format
7
Benefits to “Service Providers”
• As everyone implements DTS, the need
to support other transports will drop
• If any school implements DTS, service
providers will have to support it
• Also provides a single communication
infrastructure option for internal systems
8
DTS Specification
• Specification covers
– Technical interchange rules and
processes
– Recommended best practices
• Technical Specification is the pure
Simple Object Access Protocol
(SOAP) interface
• Implementation Guide is for both .Net 9
and Java reference implementations
DTS Specification
• Reference implementation
examples are available
• Specification does not cover
– Business rules for transaction
processing
– Operational oversight, monitoring or
escalation
10
Data Transport Issues in Higher
Ed
• Email is not reliable or flexible
enough
– No guarantee of delivery
– No guarantee of order of delivery for
sequence dependent data
– No automatic confirmation of receipt
or facility for retransmit
11
Data Transport Issues in Higher
Ed
• Email is not reliable or flexible
enough (continued)
– No synchronous response available
– Email size limitations
12
Data Transport Issues in Higher
Ed
• FTP data exchange has own
challenges
– Possible to overwrite earlier files
– No confirmation of receipt
– No synchronous response
13
Data Transport Issues in Higher
Ed
• Encryption today is always
separate and subject to its own
– Issues
– Maintenance
– Failures
14
DTS Addresses These Transport
Issues
• DTS addresses
– The confirmation issue with a send
receive protocol – confirmation is
built in
– The order of delivery problem by
actively delivering and receiving the
data – no unconfirmed handoffs
15
DTS Addresses These Transport
Issues
• DTS addresses
– The size problem through data
compression
– The FTP overwrite problem by not
using filenames
16
DTS Addresses These Transport
Issues
• DTS addresses
– The lack of a synchronous response by
building in a required synchronous
response, even if only for handling status
– The encryption issue by using standard
HTTPS for encryption – the same
technology as for online banking
17
DTS Technical Workgroup
• Task: Create a written specification for
realtime exchange of data between
organizations
– Meets business requirements
– Standards based
– Standard technologies (Java, .Net)
– Payload Insensitive
– Secure and reliable 18
DTS Technologies
• Global XML Web Service Architecture
(GXA), generally accepted as the
foundation for building Web Services
– WSDL (Web Service Definition
Language)
– SOAP (Simple Object Access Protocol)
– WSI (Web Service Interoperability)
– WSS (Web Service Security) 19
DTS Technologies
• WSSecurity (Digital Signatures)
– Strong authentication with non
repudiation
– X.509 encryption keys and
certificate authorities
• SSL encryption of HTTP streams
20
Anticipated Architectures
• Immediate processing
– Request and processed Result Response
• “Push/Push” deferred processing
– Request and Acknowledge Response
– Request with Result and Acknowledge Response
• “Push/Pull” deferred processing
– Request and Acknowledge Response (just send)
– Request for Result and Result Response
21
Immediate
Data Transport Standard: Immediate Processing DTS implementation
Central Repository
r 3.
t fo (CR) Se
oin Key/URL/Services r
dp se vice
En
n nd c
t ai e er o n
This configuration could ob ic t o t ac
have a server acting as a to serv a u ts
CR er th CR
DTS Client that received c ts nt p en t
tic o o
data for transport from the nta pie at b
co reci e a ta i
back end. ie nt nd n p
Cl au ub
1. th lic
or ke
ize y
o f
Central Repository 3.
This configuration could for (CR) Se
o int of rvi ce
have a server acting as a ndp Key/URL/Services
in E se
nde co n
DTS Client that received to obta ice r to tact
data for transport from the s CR er se rv au s CR
ntact tp the Validation could be
back end. t co ecipien nt ic to o
Clien r at e bta performed by the back-end
1. an in pu system in a synchronous
d a bli manner allowing errors to be
uth c k
oriz ey reported back immediately
e
Processing performed in the middle of accepting request and delivering processed result- Time frame of this step is immaterial
Server as
DTS Service
DTS Client
submitDTS()
Back-end Processing
2. submitDTS invoked against enpoint
Data passed to back
end for processing. 4. Acknowledgement- <DTSAcknowledge>Received</DTSAcknowledge> Response Data passed to
transport for delivery
r
3. t fo
Se oin to
rv dp g
sen ice c en ss in
de ont t ain oc e
r to ac ob pr
au ts C to of
R s ult
the R t C
ntic o o e t
a te b ta c ts R ien
an in p nt a he cip
co k t re
d a ub
uth lic k ient bac
oriz ey Cl ing
e of 1. end
s
Central Repository
(CR)
Key/URL/Services
Central Repository 3.
This configuration could for (CR) Se
o int of rvi ce
have a server acting as a ndp Key/URL/Services
in E se
nde co n
DTS Client that received to obta ice r to tact
data for transport from the s CR er se rv au s CR
ntact tp the Validation could be
back end. t co ecipien nt ic to o performed by the back-end
Clien r at e bta
1. an in pu system in a synchronous
d a bli manner allowing errors to be
uth c k
oriz ey reported back immediately
e
Processing performed in the middle of accepting request and delivering processed result- Time frame of this step is immaterial
DTS Responder
DTS Requestor
submitDTS()
2. submitDTS invoked against enpoint; Back-end Processing
<DTSPayloadType>DTSRetrieve</DTSPayloadType>
Data passed to back
end for processing. Response Data passed to
4. Response- processed result returned as payload
` transport for delivery
f
1. yo
Cli
en ke
t co u blic iz e
n ta p or
cts tain auth
C ob
p ro R to to and
ce R
C ate
sse obta cts ntic
d r in e
esu n dp nta he
lts oin e c o aut
. to
t to rvic er
If token was received in response to r et
rie Se end
original request and client wants that ve 3. s
specific response , the token must be Central Repository
passed as payload. Otherwise, first (CR)
response available for that client Key/URL/Services
would be returned. Subsequent
“DTSRetireve” would get next
available response . Exact same repository as the
CR at top of page
24
DTS Analogy
• DTS is the definition of the “Pipe” and the
structure of its contents
– The “Pipe” is the internet
– The content is SOAP
– The end points/junctions are Web Services
– The sources are Web Service enabled clients
25
DTS Analogy
• DTS defines how others can connect to
the “Pipe” already installed
– Any connections must have certain
“threads”
– Any connections must handle two way
traffic independent of how the traffic will
be used
26
DTS Analogy
• By knowing about the
pipe and the type of
connections, any
“plumber” can use
his/her own tools to
make connections;
just so long as the
threads match
27
Extending the Analogy
• We all have plumbing and fixtures
• Very unlikely we all have the same
type of fixtures
• Yet our water companies still deliver to
us all
• All our fixtures use (“process”) it
• All our drains return it
28
How Did We Do It?
• Created basic HelloWorld service and
client
– Worked interoperable
• Added simple Headers to HelloWorld
– Was not interoperable
• Added complex Header to HelloWorld
– Was not interoperable
29
Why SOAP Headers
• To answer routing and processing
expectations without opening the
payload
• Remain payload insensitive
• Allow extensibility for new
processes
30
DTS SOAP Headers
• DTSRequestRouting
• DTSRequestServiceExpectation
• DTSRequestPayloadType
• DTSRequestSignature
• DTSResponseRouting
• DTSResponseAcknowledge
• DTSResponsePayloadType
• DTSResponseSignature 31
Convoluted Filename vs Header
Elements
• A [B] <X.Y.Z:M>
A = File Type, B = Encrytption, X.Y.Z = key
identifier, M = Unique message ID
• Encryption unnecessary because using HTTPS
• DTSRequestPayloadType = A
• DTSRequestRouting
– SourceIDSubCode = X, SourceID =Y(.Z)
– UUID = M
32
Interop Problem with SOAP
Headers
• xsi:type attribute in Header
elements
– Java includes and requires this
attribute
– .Net does not
33
All about SOAP
<soap:Header>
<DTSRequestPayloadType
xsi:type="DTSRequestPayloadType"
xmlns="http://www.datatransportstandard.com">
<value>CRC01Request</value>
</DTSRequestPayloadCode>
34
SOAP is the Key
• The SOAP transmitted across the wire
is of primary importance
– Element names
– Type attribute
– Not Namespace moniker (Java uses one
by default, .Net does not)
• How you get the correct SOAP is not
important 35
SOAP Differences That Do Not Matter
Java:
<ns1:DTSRequestSignature
soapenv:mustUnderstand="0"
xsi:type="ns1:DTSRequestSignature"
xmlns:ns1="http://www.datatransportstandard.com">
<ns1:value>SignatureValue</ns1:value>
</ns1:DTSRequestSignature>
.Net:
<DTSRequestSignature
xsi:type="DTSRequestSignature"
xmlns="http://www.datatransportstandard.com">
<value>SignatureValue</value>
</DTSRequestSignature> 36
Reference Implementation
Architecture
• Client Application
• Client Core
• Service Core
• Service Application
37
Client Application
• Knows nothing of SOAP or Web
Services
• Implements Client Core Interface
– “Setters” and “Getters” of DTS
specific elements
• Houses specific business logic
38
Client Core
• Knows nothing of business logic
• Uses properties set to construct the
SOAP
• Interface for “setting send” and
“getting returned” elements
• Handles the communication to Service
Core DTS Specification
39
Service Core
• Accepts transmissions from Client Core
• Implements Service Application Interface
– “Setters” and “Getters” of DTS specific
elements
• Creates return SOAP
– Format return acknowledgement or data from
Service Application
– Construct SOAP faults
40
Service Core (continued)
• Isolated business logic
– Examples
• Invoke Service Application based on
payload
• Place payload in “queue”
41
Service Application
• Interface for “setting sent” and
“getting to be returned” elements
• Houses specific business logic
• Knows nothing of SOAP or Web
Services
42
Connecting the layers
Platform specific
communication
Entity A Entity B
Client Application
Service
Application
Platform specific
communication
DTS Spec Internet DTS Spec
43
Connecting the layers
Platform specific
communication
Entity A Entity B
Client Application
Service
Application
Client Application
Platform specific
communication
DTS Spec Internet DTS Spec
44
Connecting the layers
Platform specific
communication
Entity A Entity B
Client Application
Service
Application
Client Application
Platform specific
communication
DTS Spec Internet DTS Spec
45
Connecting the layers
Platform specific
communication
Entity A Entity B
Service
Client Application Application
Service
Application
Client Application
Platform specific
communication
DTS Spec Internet DTS Spec
46
Connecting the layers
Platform, but DTS interface ESB Platform, but DTS interface
Point A Point B
Service
Client Application Application
Service
Application
Client Application
47
Additional DTS Information
• Visit PESC at www.pesc.org
• Materials available include
– Executive summaries
– Specifications
– Reference (proof of concept)
implementations
48
Adding SOAP Headers
• Change WSDL – Still has
problems
• Create Container Classes
• Container Classes require
serialization/deserialization
directives
49
Adding SOAP Headers
• Augment Service Code
• Augment Client Code
50
Java Creating the Container Classes for the Service (IN)
package com.datatransportstandard.www.serializable;
public String getRecipientID() { return recipientID; }
import java.io.Serializable; public void setRecipientID(String newRecipientID)
{
public class DTSRequestRouting implements Serializable recipientID = newRecipientID;
}
{
private String sourceID = null; public String getRecipientIDCode() { return recipientIDCode; }
private String sourceIDCode = null;
public void setRecipientIDCode(String newRecipientIDCode)
private String recipientID = null; {
private String recipientIDCode = null; recipientIDCode = newRecipientIDCode;
private String uuid = null; }
public String getSourceID() { return sourceID; } public void setUUID(String newUUID)
{
public void setSourceID(String newSourceID) uuid = newUUID;
{ }
sourceID = newSourceID;
public String getTransmissionDateTime()
} {
return transmissionDateTime;
public String getSourceIDCode() { return sourceIDCode; } }
public void setTransmissionDateTime(String newTransDateTime)
public void setSourceIDCode(String newSourceIDCode) {
{ transmissionDateTime = newTransDateTime;
}
sourceIDCode = newSourceIDCode;
} } 51
Java Creating the Container Classes for the Service
(OUT) public String getRecipientID() { return recipientID; }
package com.datatransportstandard.www.serializable;
public void setRecipientID(String newRecipientID)
{
import java.io.Serializable;
recipientID = newRecipientID;
}
public class DTSResponseRouting implements Serializable
{ public String getRecipientIDCode() { return recipientIDCode; }
private String sourceID = null;
private String sourceIDCode = null; public void setRecipientIDCode(String newRecipientIDCode)
private String recipientID = null; {
private String recipientIDCode = null; recipientIDCode = newRecipientIDCode;
}
private String uuid = null;
public String getUUID() { return uuid; }
private String transmissionDateTime = null;
public void setUUID(String newUUID)
public String getSourceID() { return sourceID; } {
public void setSourceID(String newSourceID) uuid = newUUID;
{ }
sourceID = newSourceID;
} public String getTransmissionDateTime()
{
public String getSourceIDCode() { return sourceIDCode; } return transmissionDateTime;
}
public void setSourceIDCode(String newSourceIDCode)
public void setTransmissionDateTime(String newTransDateTime)
{ {
sourceIDCode = newSourceIDCode; transmissionDateTime = newTransDateTime;
} }
52
}
Java Examples (Service)
• Modify the WSDD
<beanMapping
languageSpecificType="java:com.datatransportstandard.www.serializable.DTSRequestRouting"
qname="ns2:DTSRequestRouting" xmlns:ns2="http://www.datatransport.com"/>
<beanMapping
languageSpecificType="java:com.datatransportstandard.www.serializable.DTSResponseRouting
" qname="ns3:DTSResponseRouting" xmlns:ns3="http://www.datatranportstandard.com"/>
• Augment the Code (inbound)
MessageContext ctx = MessageContext.getCurrentContext();
SOAPEnvelope requestEnv = ctx.getRequestMessage().getSOAPEnvelope();
SOAPHeaderElement requestHeader =
requestEnv.getHeaderByName("http://www.datatransportstandard.com",
“DTSRequestRouting");
DTSRequestRouting incontainer = (DTSRequestRouting) requestHeader.getObjectValue();
System.out.println(“SourceId = “ + incontainer.getSourceId());
53
Java Examples (Service)
• Augment the Code (outbound)
DTSResponseRouting outcontainer = new DTSResponseRouting();
outcontainer.setSourceId(“TEST SOURCE ID”);
outcontainer.setSourceIdType(“TEST SOURCE ID TYPE”);
…
SOAPHeaderElement responseHeader = new SOAPHeaderElement(“
http://www.datatransportstandard.com”
,“DTSResponseRouting”)
responseHeader.setObjectValue(outcontainer);
SOAPEnvelope responseEnv =
ctx.getResponseMessage().getSOAPEnvelope();
responseEnv.addHeader(responseHeader);
54
Java Examples (Client)
• Augment the Code (General)
QName qn = new
QName(“http://www.datatransportstandard.com”,“DTSRequestRouting”);
call.registerTypeMapping(DTSRequestRouting.class, qn,
new
BeanSerializerFactory(DTSRequestRouting.class, qn),
new
BeanDeserializerFactory(DTSRequestRouting.class, qn));
QName qn1 = new
QName(“http://www.datatranpsortstandard.com”,“DTSResponseRouting”);
call.registerTypeMapping(DTSResponseRouting.class, qn1,
new
BeanSerializerFactory(DTSResponseRouting.class, qn1),
new
BeanDeserializerFactory(DTSResponseRouting.class, qn1));
55
Java Examples (Client)
• Augment the Code (outbound)
DTSRequestRouting outcontainer = new DTSRequestRouting();
outcontainer.setSourceId(“TEST SOURCE ID”);
outcontainer.setSourceIdType(“TEST SOURCE ID TYPE”);
…
SOAPHeaderElement requestHeader = new
SOAPHeaderElement(“http://www.datatransportstandard.com”,
“DTSRequestRouting”);
requestHeader.setObjectValue(outcontainer);
call.addHeader(requestHeader);
56
Java Examples (Client)
• Augment the Code (inbound)
SOAPEnvelope responseEnv =
call.getMessageContext().getResponseMessage().getSOAPEnvelope()
;
SOAPHeaderElement responseHeader =
responseEnv.getHeaderByName(“http://www.datatransportstand
ard.com”,“DTSResposneHeader”);
DTSResponseHeader incontainer = (DTSResponseHeader)
responseHeader.getObjectValue();
System.out.println(“Source Id=“ + intcontainer.getSourceId());
57
.Net: Creating the Container classes for Service
[XmlTypeAttribute(Namespace=“http://www.datatransportstandard.com”)]
[XmlRootAttribute(ElementName=“DTSRequestRouting”, Namespace=“http://www.datatransportstandard.com”, IsNullable=false)]
[XmlInclude(typeof(DTSRequestRouting))]
public class DTSRequestRouting : System.Web.Services.Protocols.SoapHeader
{
public string UUID;
public string transmissionDateTime;
public string sourceID;
public string sourceIDCode;
public string recipientID;
}
[XmlTypeAttribute(Namespace=“http://www.datatransportstandard.com”)]
[XmlRootAttribute(ElementName=“DTSResponseRouting”, Namespace=“http://www.datatransportstandard.com”, IsNullable=false)]
[XmlInclude(typeof(DTSResponseRouting))]
public class DTSResponseRoutingElements : System.Web.Services.Protocols.SoapHeader
{
public string UUID;
public string transmissionDateTime;
public string sourceID;
public string sourceIDCode;
public string recipientID;
}
[XmlTypeAttribute(Namespace=“http://www.datatransportstandard.com")]
public class DTSResponseRouting : DTSResponseRoutingElements{};
58
.Net: Augment the Service
• Add declarations to service Class
public DTSRequestRouting DTSRequestRoutingVal;
public DTSResponseRoutingElements DTSResponseRoutingVal;
public SoapUnknownHeader[] unknownHeaders;
• Add serialization directives to
WebMethod()
[SoapHeaderAttribute("DTSRequestRoutingVal")]
[SoapHeaderAttribute("DTSResponseRoutingVal",
Direction=SoapHeaderDirection.Out)]
[SoapHeader("unknownHeaders")]
59
.Net: Creating the Container classes for Client
[XmlTypeAttribute(Namespace="http://www.datatransportstandard.com")]
[XmlRootAttribute(ElementName="DTSRequestRouting",Namespace="http://www.datatransportstandard.com",
IsNullable=false)]
[XmlInclude(typeof(DTSRequestRouting))]
public class DTSRequestRoutingElements : System.Web.Services.Protocols.SoapHeader
{
public string UUID;
public string transmissionDateTime;
public string sourceID;
public string sourceIDCode;
public string recipientID;
}
[XmlTypeAttribute(Namespace="http://www.datatransportstandard.com")]
public class DTSRequestRouting : DTSRequestRoutingElements{}
[XmlTypeAttribute(Namespace="http://www.datatransportstandard.com")]
[XmlIncludeAttribute(typeof(DTSResponseRouting))]
[XmlRootAttribute("DTSResponseRouting", Namespace="http://www.datatransportstandard.com", IsNullable=false)]
public class DTSResponseRouting : System.Web.Services.Protocols.SoapHeader
{
public string UUID;
public string transmissionDateTime;
public string sourceID; 60
public string sourceIDCode;
public string recipientID;
}
.Net: Augment the Client
• Add declarations to Client Web Reference/Proxy
Class
public DTSRequestRoutingElements DTSRequestRoutingVal;
public DTSResponseRouting DTSResponseRoutingVal;
• Add serialization directives to WebMethod()
[SoapHeaderAttribute("DTSRequestRoutingVal")]
[SoapHeaderAttribute("DTSResponseRoutingVal",
Direction=SoapHeaderDirection.Out)]
61
Contact Information
We appreciate your feedback and comments.
We can be reached at:
Nathan Chitty, Nelnet, Inc.
(904) 2817235
nathan.chitty@nelnet.net
Gary Sandler, ELM Resources
(510) 9037960
gsandler@elmresources.com
62