[nerdponx]

What would the elegant Ruby version look like, and what was inelegant about the Python version?

In Python I imagine you could just do something like this:

    from time import sleep

    sleep_increment = 1
    sleep_time = 0
    success = False
    while not success:
        sleep(sleep_time)
        success = try_thing()
        sleep_time += sleep_increment

[stouset]

    def backoff(wait = 5, exponent: 1.5)
      yield
    rescue BackoffError
      sleep(wait)
      wait = wait ** exponent
      retry
    end

    backoff { connect_to_a_thing }

This is trivially composable with any other control flow you might want to write in Ruby.

Personally I find the Python approach inelegant. Can it be composed? Why am I having to work with low-level looping constructs instead of higher level control flow constructs that map more closely to the task I’m actually trying to accomplish?

Same thing goes for any non-functional looping at this point. Why am I having to care about loop indices, incrementing counters, creating result arrays and inserting items into them, etc.? It’s (almost) 2018 and people are still writing low-level looping logic for the n-billionth time. Worse, people have to read it and parse it for the n-trillionth time to figure out which looping idiom is being used, instead of being able to see at a glance that something is being mapped, selected from, reduced, etc.

[masklinn]

Nothing precludes a similar approach in Python:

    def backoff(wait=5, exponent=1.5):
        while True:
            yield
            sleep(wait)
            wait = wait ** exponent

    for backoff():
        connect_to_a_thing()

Here's the question: do you need exponential backoff in more than one place? Because if you don't, bundling all of that doesn't buy you anything and the original works just fine.

> This is trivially composable with any other control flow you might want to write in Ruby.

And it is actually trivially composable with other Python control flow, it's just a backoff iterator, you can drive it however you want, or compose it with other iterators (e.g. enumerate() to know which instance you're on, islice to stop after a certain number of tries, …)

[stouset]

Your solution is much cleaner to me, except (not really knowing Python) it looks like it might have a bug, in that the `while True` will never terminate.

[masklinn]

> the `while True` will never terminate

The iterator is infinite on purpose (it has no reason to be finite).

You'd terminate it from outside, either when the operation succeeds, so the example loop should more properly be something along the lines of:

    for _ in backoff():
        try:
            c = connect_to_a_thing()
            break
        except ConnectionError:
            pass # retry

or you'd use composition to e.g. stop after a set number of tries:

    for _ in itertools.islice(backoff(), 5):
        # do stuff

most likely a combination of both.

[creatornator]

I think you can break the for loop after the function call and it'll end after the wait is successful. Bit of a hack though

[nerdponx]

It looks like the Ruby-specific elements here are (1) the 'retry' statement, and (2) the ability to yield a block twice (Python's equivalent `with` syntax explicitly prohibits re-yielding a block).

(1) might just be a matter of taste -- that 'retry', to me, makes me now have to step back and think "oh, I'm retrying something now? what exactly is being retried here?" Having it all wrapped up in a 'while' loop makes the intent explicit right from the get-go.

(2) I agree is nice. It would be great if you could do this:

    with backoff():
        connect_to_a_thing()

but you're not allowed to 'yield' twice inside a context manager. However you can get damn close (and fully composable) by wrapping the connection logic and the retry logic in functions:

    def connect_to_a_thing(url, access_token):
        ...

    def backoff(connector, wait=5, exponent=1.5, *args, **kwargs):
        success = False
        while not success:
            try:
                connector(*args, **kwargs)
            except BackoffError:
                sleep(wait)
                wait = wait ** exponent
            else:
                success = True

    backoff(connect_to_a_thing, url, access_token)

Or you could be even more explicit with recursion:

    def backoff(connector, wait=5, exponent=1.5, *args, **kwargs):
        try:
            connector(*args, **kwargs)
        except BackoffError:
            sleep(wait)
            backoff(connector, wait ** exponent, exponent, *args, **kwargs)

As for your point about loop indices, Python generally has wonderful support for not worrying about loop indices and counters. I'm also not sure how that applies in this case; you still need to increment the value of 'wait'. Then again...

    def exponentiate_forever(value, exponent):
        while True:
            yield value
            value = value ** exponent

    def backoff(connector, wait=5, exponent=1.5, *args, **kwargs):
        for wait in exponentiate_forever(wait, exponent):
            try:
                connector(*args, **kwargs)
            except BackoffError:
                sleep(wait)
            else:
                break

But please don't do that.