[pilif]

Likely "Cowboy" is a transparent proxy added by your mobile service provider. I had a similar thing happening a year ago when the mobile provider used by most of our barcode scanners decided to add a transparent proxy into the loop (without telling anybody).

The solution for this problem: Use SSL.

I mean: There are already many good reasons to use SSL, but whenever you need to send any kind of mission critical data over the mobile network, you practically must use SSL if you want any kind of guarantees that the data you send to the server is what actually reaches the server (and reverse).

Here's my war story from last year: http://pilif.github.io/2013/09/when-in-doubt-ssl/

[rubiquity]

Cowboy is the name of an Erlang web server and Heroku uses Erlang for their routing. I imagine the reason Cowboy is showing up is due to Heroku's routing layer.

[pilif]

I would agree, but the fact that it suddenly stopped working points to carrier interference rather than Heroku (because as a developer focussed company, they would have informed OPs company of the change)

[shawabawa3]

It seems it is on Heroku's end. https://news.ycombinator.com/item?id=8515164

[monokrome]

Heroku isn't in a frozen state. I'm sure that they change the code daily. The fact that it suddently stopped working doesn't really point to any specific endpoint unless you know which ones have changed. Then it might mean something.

[mentat]

Or perhaps they would have noticed increased error rates for a client and investigated. Monitoring right?

[goleksiak]

We would really like to use HTTPS but it's not supported by the Arduino chipset as I understand it. Though I'm not the hardware guy here at eatabit...

[linuxlizard]

Why not ROT13? Or a simple substitution cypher?

Not trying to be silly. But if the only goal is to prevent man-in-the-middle attacks such as someone mangling the data, why not "corrupt" the data such that the phone company in the middle can't read it?

You control both ends. You can make your own "security".

You're not explicitly worried about security. You're not worried about Evil Person reading your messages. You just want your carrier to stop f'ing with your data.

If the data is slightly corrupted so the carrier's crappy software can't recognize it as http headers then the carrier's software (hopefully) won't fck with it.

[MichaelGG]

They could try a different port - some systems won't bother.

They might also use TLS with null cipher. That should be not-so-intensive, even on a tiny processor. And it could be enough to defeat some packet-modifiers (they may notice it's TLS and not analyze), while maintaining HTTPS compatibility.

[lnanek2]

Last I checked, you have to use some sort of special sockets add on to use raw TCP instead of HTTP over TCP with heroku. So you are making your heroku setup more complicated and potentially more expensive since they sometimes charge for add ons. It may disable some of their routing and load balancing capability as well. It is kind of silly to start rewriting standard transport layers anyway since you are going to spend a long time doing that instead of working on your product.

[swordswinger12]

You guys build all the hardware - why not just use one of the pre-shared key TLS ciphersuites? No expensive public-key crypto required, just AES and SHA.

[lmb]

Last time I checked (about a year ago), support for PSK was pretty abysmal on the server side of things.

[angersock]

Why don't you guys just use a Beaglebone Black ($50), M2M cape (http://www.yantrr.com/products/m2m-cape-for-beaglebone), and a thermal printer (https://www.sparkfun.com/products/10438). BOM for that, plus project enclosure, is like $200. That's also just after a bit of quick googling...there's bound to be a much cheaper solution.

What's your price point for hardware?

[goleksiak]

We are at about $200 now for hardware. We have a custom PCB and 3d printed case. We also have an LCD and some control knobs in the mix. Checking out the Beaglebone stuff now...very interesting since there is basically no 3G/4G modems for Arduino right now...

[angersock]

I'm just curious why the default response is still to reach for an Arduino--much more powerful SoC chips are cheap these days.

[goleksiak]

For us, the reason is that my co-founder and I (both are not hardware guys) were able to build a proof of concept in my garage and Arduino seemed like the best (easiest) choice. Since then, we hired a hardware guy who designed a custom PCB etc. HTTPS would be great but we don't transmit any personal data so it's not a high priority right now.

[Alupis]

> HTTPS would be great but we don't transmit any personal data so it's not a high priority right now.

You are sending people's orders around the web. I'd consider that "personal".

None-the-less, use SSL, there is little reason not to use it these days. And as others have pointed out, it's the only good and easy way to guarantee what you send to one of these printers is what it actually received (no carrier tampering of your packets, etc).

Just use SSL.

[angersock]

Hey, no worries. Real artists ship. :)

[curveship]

One reason to still use an Arduino is realtime. The arduino's minimalist OS is realtime by default, but the BBB runs non-realtime Debian. They might need RT timings for the printer interface? You can add realtime linux extensions to BBB, but it's not a beginner task -- basically rolling your own Linux installation and writing your app as a kernel module. You can also use the PRU on the BBB to get insane RT performance, but you'll have to code it in assembly. I love the BBB, but Arduino is still an easier package for simple realtime.

[sitkack]

Yeah, like any number of Olimex parts. Or a Raspberry Pi Model A.

[serf]

larger pool of cheap developers, and a hook onto an ongoing popular trend (arduino gizmodery), cheap prototyping.

[Alupis]

if you buy a batch of beaglebones from adafruit.com or another, perhaps larger mid-tier distributor, you can likely get a discount per unit.

[tedchs]

SPDY is an alternative to HTTP that avoids at least Verizon's carrier shenanigans.

[scottlinux]

Sounds right. I also found this:

http://stackoverflow.com/questions/15830333/arduino-due-http...

[sitkack]

Your network stack is being handled by a GSM/GPRS module, it might support SSL. Using Arduino was easy in this case, but I would have gone with something a little more substantial like an STM32F4

[Filligree]

Then, don't use SSL if it's too heavyweight.

I know everyone will tell you not to roll your own cryptosystem, but rolling your own is superior to having no encryption or authentication, and so long as you're sane about it the result should be no worse than passing plaintext.

Your messages are small. Encrypt (or maybe just sign) them with RSA and call it a day. You don't really need to use port 80 and a HTTP preface at all, do you?

[lmb]

That is pretty bad advice. RSA is slow, needs a lot of memory and is difficult to get right. Just go with AES in CTR mode if you absolutely have to. And remember that encryption != authentication.

[pavel_lishin]

Who was replacing numbers with asterisks, and for what purpose?

[pilif]

That was a "privacy" mode of some personal firewall that was protecting the user's phone number from leaking out.

Incidentally, this has leaked the users phone number because only that specific numer was being replaced with asterisks.

Welcome to the world of very crappy "security" (-theater) end-user products.

[goleksiak]

The asterisks hide this request's BasicAuth credentials between the cellular printer and our app servers...we don't transmit customer phone numbers between the printer and our app servers. Twilio transmits them to our app servers but that is HTTPS

[pilif]

I think pavel_lishin was referring to a side-note in my war-store I listed as the parent comment.

[pavel_lishin]

Yup.

[goleksiak]

ah - my bad

[goleksiak]

I didn't want to hang our BasicAuth creds out to dry

[lclarkmichalek]

You were sending them over http, so aren't they already kinda fucked?

[goleksiak]

y, true

[pavel_lishin]

Oh, I was asking pilif to comment about their experience.

[ldng]

What's wrong with transparent proxy ? Isn't how HTTP caching is supposed to work ? I would think the cache headers are the solution rather than SSL.

It feels like you are kind of throwing the baby with the bath water. IMHO, badly configured transparent proxy does not mean the concept is bad, does it ?

[fulafel]

No, "transparent proxying" is a clear violation of HTTP specs (as well as TCP protocol, and IP's "thou shall not mess with packets in transit" principle/specs). It's essentially a MITM attack and all bets are off wrt correctness.

[ldng]

We'll disagree then :)

From RFC 2616 "The HTTP/1.1 protocol allows origin servers, caches, and clients to explicitly reduce transparency when necessary."

As I said, bad configurations dos not mean the principle is unsound.

[fulafel]

No, RFC 2616 uses transparency in a different meaning that the common usage of "transparent proxy" is.

Common meaning (from https://en.wikipedia.org/wiki/Proxy_server#Transparent_proxy): "Also known as an intercepting proxy, inline proxy, or forced proxy, a transparent proxy intercepts normal communication at the network layer"

RFC 2616 uses the term to describe a property of a normal, opt-in HTTP proxy: "A 'transparent proxy' is a proxy that does not modify the request or response"

In preceding discussion we were using the term in its common usage meaning.

Also, you misrepresent what RFC 2616 says about the its concept of transparency. The part you quoted continues:

  "the protocol requires that transparency be relaxed

      - only by an explicit protocol-level request when
        relaxed by client or origin server

      - only with an explicit warning to the end user when relaxed by
        cache or client "

[abhaga]

Problem is often the competence of people handling these proxies. Yesterday I discovered that my ISP is blocking all DELETE http calls (ACT fibernet in India)!

And before that they decided to enforce a reverse DNS lookup so all the name based virtual servers with local DNS entries stopped working - I had my staging instances setup that way.

And not only all this happens without any prior information, it is next to impossible to climb through the support layers to finally find someone who even understands what are you talking about. They just want you to restart your modem to "resolve the problem".

[sitkack]

Sounds like you need to VPN out of your ISP, ;)

[abhaga]

Yeah but the DELETE blocking affects any customers on this ISP as well. I am now working on moving to SSL asap.

[pilif]

The problem is that you have no recourse. When the carrier decides to f up your connections somehow, you can try to work around the issue, until they break it some more to the point where the one thing you really needed also stopped working.

Then you can hope that you are big enough to have priority with the carrier or you know somebody who knows somebody who can fix it.

Or you don't deal with any of this and just go SSL. A certificate will cost you $100 per year in the worst case. Thats about one hour of your time spent fixing proxy issues (not including customers and/or end users breathing down your neck because their software just stopped working for some as yet unknown reason)

[ldng]

Except the carrier, or the that case the hoster, did not break anything. It's their software that don't comply to the RFC. HTTP is supposed to be cacheable when it makes sens. With SSL to work that around you break the Internet somehow.

To me, there are valid usecase for SSL, using it to work around proxies is not one. That said, I get your point, you prefer the possibly easier and safer way. But you still might run into another set of problems (https://news.ycombinator.com/item?id=8471877).

[ctz]

This would require a vast, vast upgrade of client power to achieve the same communications performance. If you could achieve it all, SSL would also likely decrease reliability over a spotty GSM link.

[pjc50]

Is client CPU actually a limiting factor? How does this affect reliability?

[ctz]

You cannot physically fit a whole SSL datagram (max size 16KB) into 8KB of RAM. SSL requires multiple passes over the data to (eg) decrypt and verify a datagram. At this point, you cannot use standard SSL at either the server side or client side.

On the subject of reliablity: a 8 bit uC running at 16MHz needs a long time to do the public key crypto required to set up the connection. This means you need a GSM data link to be continuously available for a longer period.

[pjc50]

This is true, but if you're using GSM you have a much beefier processor handling the GSM side and there's no need to use a tiny microcontroller. Some of the GSM modules will offload the whole HTTP(S) request for you.

Edit: or you could get a Cortex-M0 with 32K RAM for $2.

[MichaelGG]

SSL supports a null cipher, so why not use that? The handshake alone may be enough to prevent packet inspection.

[serf]

as someone unfamiliar with their specific workload, but who has used such boards to do vpn/ssl stuff, no -- it's not a limiting factor unless trying to skim power requirements.