[hellofunk]

I had a senior dev interview with a major company in SV last year where the lead and hiring manager claimed that the description for this behavior was "side-effect free".

I didn't want to ruin my chances by correcting him that side-effects and idempotence refer to different qualities.

I passed the interview, but I've never forgotten that. The guy also seemed to have washed into the interview off his surf board.

Oh well. It was a very major company, so I let it all go.

[dylanpyle]

In the context of software it seems like the word has pretty broadly adopted the meaning of "side-effect free". It's even made it to Wikipedia: https://en.wikipedia.org/wiki/Idempotence#Computer_science_m...

One notable example: Stripe's "idempotency keys": https://stripe.com/blog/idempotency

[twic]

I understand 'idempotent' not as 'having no side effects' but as 'having the same side effects whether performed one or many times'. This is side-effect free:

  echo hello

This is side-effectful, but idempotent:

  echo hello >world

This is side-effectful and not idempotent:

  echo hello >>world

[xelxebar]

Mathematical idempotency depends heavily on the function's domain. It's not too challenging to construct a non-idempotent function that has an idempotent restriction. So depending on what your domain of interest some operations may be idempotent or not.

In the hello world example, I guess we are considering 'echo' as the function. If then if we consider stdout and echo's args to be the same, echo will be idempotent.

However, stdout and command args are pretty different things on any system I know. If we care about things like command line history, then clearly 'echo hello world' is not idempotent.

[ComputerGuru]

I disagree that your ‘ echo hello’ example is free of side effects. It writes to stdout. It overwrites $?. It changes the value of the last executed history command. Etc.

About the only side effect free operation at a blank line terminal prompt is _probably_ hitting enter or ctrl c. Or ‘#something<Enter>’

But your idempotency examples are spot on.

Basically, idempotency is a very useful concept. Side effect free isn’t.

[twic]

> I disagree that your ‘ echo hello’ example is free of side effects.

I posit that you understand perfectly well what i was getting at! If it helps, imagine that you are spawning a fresh shell to execute each command line, through system(3) or some such.

[ComputerGuru]

Fair enough!

[Gaelan]

Interestingly, in this case the non-idempotent option is less safe.

[dragonwriter]

Non-idempotent options are always unsafe; safe operations are a subset of idempotent operations.

(I suspect you meant “safe” in a different sense, but I couldn't resist.)

[twic]

> safe operations are a subset of idempotent operations

Definitely not. 'Safe' means too many things in different contexts to make sweeping statements like that.

Here's one i came across recently. You are interacting with a stock exchange. You have sent an order to buy 100 shares of Twitter stock at some price, and it is now resting on the order book, waiting for orders to sell at that price. You see that the character limit on tweets is being increased, and decide that you don't want to buy 100 shares, but only 90, so you send a message to modify your order. At the same time, two other people, who have also seen the news, decide to dump their holdings of 90 shares each by sending sell orders at the price you are bidding. Their messages arrive just before and after yours. What happens?

If the message to reduce your order is a nasty non-idempotent one which says "reduce my order by 10", then:

1. First seller's order to sell 90 matches your order to buy 100; you buy 90, and your order is reduced to one to buy 10

2. Your modify message reduces your order of 10 to 0

3. Second seller's order to sell 90 arrives, but has nothing to match with

You have successfully bought 90 shares, well done.

If the message to reduce your order is a nice safe idempotent one which says "change my order to 90", then:

1. First seller's order to sell 90 matches your order to buy 100; you buy 90, and your order is reduced to one to buy 10

2. Your modify message increases your order of 10 to 90

3. Second seller's order to sell 90 matches your order to buy 90; you buy 90, and your order is reduced to 0

You have bought 180 shares. Oh dear.

This is not hypothetical. More than one new exchange has launched with a protocol where orders are modified to a target amount, rather than by a target delta, and changed it some time later, after their users got themselves in a pickle.

You can do a bit better by making the modify message a compare-and-set, so it only changes the quantity to 90 if it is currently 100. In that situation, the modify will fail, and the second order will sell you 10 shares. That's still not as good as the non-idempotent message.

> (I suspect you meant “safe” in a different sense, but I couldn't resist.)

The moral of this story is that computer scientists should not be allowed to use the word "safe"!

[dragonwriter]

> > safe operations are a subset of idempotent operations

> Definitely not. 'Safe' means too many things in different contexts to make sweeping statements like that.

The intended reference (which I hoped would be recognized generally in this audience) was to he definition of “safe” and “idempotent” in HTTP (see section 4.2 of RFC 7231.)

(In your example, neither operation is “safe” in the HTTP sense; all state-modifying operations are unsafe.)

[Sacho]

I agree with you generally that talking about 'safe' may be misleading. However, I don't think your example shows that idempotent operations can be 'unsafe' - I think it shows that the translation of user's intent to a system operation was lossy. The user's intent was to "buy a total of 90". Translating that to "Set my current order to 90", regardless of the idempotency of the operation, is not accurate, for precisely the reasons you stated. Neither is "reduce my order by 10", because the user may attempt to send the message "set my current order to 90/reduce my order by 10" multiple times(just to make sure it went through), and then they would get to buy less than 90 shares.

It would take different data structures and a different API to translate the user's intent correctly, and then the idempotence of the operation may be important. For example, if the order had a "total" and "current balance", you could be sending a message to set the total to 90(which would internally reduce the current balance), and since that message is idempotent, it's safe to send it multiple times.

[taejo]

> That's still not as good as the non-idempotent message.

Only if you can be sure the non-idempotent message arrived. It's not safe to re-send the delta message.

[jholman]

I disagree that this is common the world of software, and I disagree that the WP article that you linked makes that claim. I humbly suggest that you might benefit from reading that article more carefully.

[dylanpyle]

You're right, I meant to summarize more along the lines that calling something multiple times doesn't cause more than one set of side effects. Unfortunately too late to edit by the time I re-read it!

[ComputerGuru]

It’s funny because if you read what they wrote, they’re fully aware of the difference between the two (since they reference the HTTP RFC definition of the idempotency of PUT and DELETE).

[js2]

That difference is somewhat subtle and I had to read the description in the sibling comment to understand it. I would have welcomed a correction from somone I was interviewing.

On a side note, the founders of Google once tried to explain recursion and idempotence to Terry Gross on NPR. It's pretty amusing. It starts at 13:45 into the interview here:

http://www.npr.org/2003/10/14/167643282/google-founders-larr...

[Jach]

You may have just been victim to the "let's see if they take a risk and correct me on my BS" test some interviewers give. Or they just didn't get it, that happens too. Some won't admit they're wrong even in the face of a mathematical proof..

[lawn]

I wonder what would have happened if you gave a short counter example.