[dchuk]

I work in the trucking (telematics) industry. I've dealt with a decent amount of ftp file transfers, but most stuff I've seen has been some sort of API based (SOAP, REST, etc).

Can you give a few more specific examples? Are you in the industry now? Working at a carrier?

[nulagrithom]

I work in intermodal. Whenever one of our larger customers wants to give us a load to move, we use X12 EDI. This is standard throughout at least the intermodal industry, though from what I gather that may also be true of over-the-road as well.

An example 204 EDI (Load Tender) looks like this:

    ISA*01*0000000000*01*0000000000*ZZ*ABCDEFGHIJKLMNO*ZZ*123456789012345*101127*1719*U*00400*000003438*0*P*>
    GS*SM*4405197800*999999999*20111219*1747*2100*X*004010
    ST*204*0001
    B2**XXXX**9999955559**PP
    B2A*04
    L11*NONPRIMARY*OK
    MS3*XXXX*B**M
    NTE**FROZEN GOODS SET TO -10d F
    N1*PF*XYZ CORP*9*9995555500000
    N3*31875 SOLON RD
    N4*SOLON*OH*44139
    N7**NONE*********FF****5300
    S5*1*CL*27800*L*2444*CA*1016*E
    L11*9999001947*DO
    L11*9999670098*CR
    L11*9999001866*DO
    L11*9999669887*CR
    G62*69*20111218
    N1*SH*XYZ CORP*9*9991555550000
    N2*TERMINAL FREEZER
    N3*5555 TERMINAL RD
    N4*CLEVELAND*OH*44023
    S5*2*PU*3042*L*312*CA*146*E
    L11*9999001866*DO
    L11*9999595358*PO
    L11*9999669887*CR
    G62*70*20090728
    N1*ST*1 EDI SOURCE*93*9990055555
    N3*31875 SOLON RD
    N4*SOLON*OH*44139
    OID*9999669887*99999595358**PC*312*L*3042*E*146
    L5**FREIGHT
    G61*IC*FEEDBACK*EM*[e-mail]
    S5*3*CU*24758*L*2132*CA*870*E
    L11*9999001947*DO
    L11*9999008881*PO
    L11*9999670098*CR
    G62*70*20111218
    N1*ST*1 EDI SOURCE*93*9990055555
    N3*55555 5TH AVE
    N4*MAYFIELD*OH*44244
    OID*9999670098*999608881**PC*2132*L*24758*E*870
    L5**FREIGHT
    G61*IC*FEEDBACK*EM*[e-mail]
    L3*27800*G*******1016*E*2444*L
    SE*46*0001
    GE*1*2100
    IEA*1*000002104
  

I haven't proper parsed it, but I believe that's going from Cleveland to Mayfield. One of those L11 segments is probably a reference number. There's no MS1 segment so it's likely over the road? Anyway, it's not exactly descriptive or even human readable...

A reply accepting a load looks like this:

    ISA*01*0000000000*01*0000000000*ZZ*ABCDEFGHIJKLMNO*ZZ*123456789012345*101127*1719*U*00400*000003438*0*P*>
    GS*GF*4405197800*999999999*20111219*1742*000000003*X*004010
    ST*990*000000003
    B1*XXXX*9999919860*20111218*A
    N9*CN*9999919860
    SE*4*000000003
    GE*1*000000003
    IEA*1*000000003

These are commonly exchanged as text files over FTP sites.

Some of our more forward-thinking, larger customers are considering moving to AS2, which I believe is sent over HTTP vs FTP. A cursory Google search doesn't really turn up any clear examples on AS2, which doesn't exactly comfort me, but at least there's an RFC[0] for it, whereas for the X12 spec you have to pay[1] to see certain parts of it.

Not that anyone follows the "spec" anyway. We code special handling for every single one of our customer's EDI transmissions.

I wish everything was REST, or at least JSON. That would be 10x easier. Instead we spend weeks going back and forth on silly things like what a 07 means in the ATS segment, or what character to use for line endings (wish I was kidding -- we've been blocked for two months on the line ending character).

What's more is with the ELDs in all our trucks, customers are increasingly wanting GPS updates. I'd love to offer them a streaming socket with GPS data -- it's completely feasible considering our ELD backend. Instead everyone is wondering how we can send updates in 15 minutes increments over FTP, especially when these transactions are often batched in 5 minute loops on both ends in the first place.

It kills me a little. We could be doing so much more. I can't believe we aren't pushing for real time. I can't believe five to fifteen minute batching loops are acceptable.

[0]: http://www.ietf.org/rfc/rfc4130.txt [1]: http://www.x12.org/x12-work-products/x12-edi-standards.cfm

[hboon]

Good old EDI. I last worked with it more than 15 years ago. I used to have several thick, bound EDI specs for each message my team was working with on our desks. After a while you get really good at counting the number of separators...

For small messages and loads, it's entirely reasonable to work towards near real-time processing. A possible reason why some EDI folks you talk to might be reluctant to think in that direction is because EDI payloads can go upwards to 10s or 100s MB. Yes, text. Instead of trucks, think ship cargo manifests. That could blind-side them.

[dchuk]

Thanks for the reply! That certainly does look like a pain in the ass. Being on the telematics side specifically, we deal less with EDI stuff, but I'm sure down the road I will have to, and I agree that a structured API would be much better.

If you don't mind me asking, what ELD are you running? My company specifically builds one of those, amongst the rest of the productivity suite necessary for a driver to do their work.

[nulagrithom]

We started out using XRS. We run almost 100% independent contractors. Started out putting our own tablets in all the trucks. We discovered drivers can run up enormous data bills when they figure out how to circumvent the MDM... It apparently was also a pain to physically install XRS (I don't know the details of that). We also have nearly 50% turnover per year. The whole situation was really untenable.

We've since switched to GeoTab and a BYOD model. The GeoTab devices are a lot cheaper, so a contractor walking away with one isn't that big of an issue. Rollout was much smoother this time.

[dchuk]

I would really like to pick your brain a bit more if possible. We directly compete with both XRS and GeoTab, so getting insight into your fleet's decision making process would be super helpful if you're willing.

Also could potentially talk about our ELD offering (amongst a bunch of other stuff) if you're interested.

darrin [at] platformscience.com

[dhruvkar]

I work for a stone distributor and we have issues with trucking, but had never delved into it.

This was interesting to read.

Donyou have more sources for gaining more insight into how this all works? I'd love to take a crack at parts of this.

[nulagrithom]

If you Google '204 EDI specification' you should find a lot of random specs in PDF out there for various companies. 214 will show you some of the status updates too. Stuff like this: http://www.shipfsp.com/media/pdf/it/EDI_214_X12_V4010.pdf

I've found this EDI notepad useful for parsing flat files: https://www.liaison.com/products/integrate/edi-notepad/

And this npm library actually parses EDIs pretty well too: https://www.npmjs.com/package/x12

[dragonwriter]

> I wish everything was REST,

REST and X12 EDI are orthogonal features: REST can use any format for resource representation, and X12 EDI formats are a data representation.

X12 EDI can be used in a REST way (one of the two mechanisms in healthcare standard operating rules for certain HIPAA-mandated X12 transaction is RESTish—the HTTP+MIME method—though there is also a SOAP method.)

[querulous]

they're probably using edi via ftp because the entire logistics industry is still using edi via ftp. edi is still the dominant way retailers, suppliers and shippers transmit information