[gmantom]

While I think F# is a great programming language. The article missed some good uses of it.

Specifically the article missed one of largest F# deployment, in production, in the world at this point. We use F# at Jet.com and it powers every part of our core business from our dynamic pricing algorithm to search and analytics.

Over 4 million customers already on jet and over 2200 cores on azure all running F# code.

[louthy]

> Over 4 million customers already on jet and over 2200 cores

That sounds like a lot of cores for such a small data-set. Are you able to expand a little more on why you need so much computing resource?

[gmantom]

Have you ever used Jet? Try it, add a few things to your cart then add a few more and see what happens to the prices of the previous items. There are millions of permutations being computed behind the scenes.

The system is computing prices all of the time based on many factors. Plus we have built everything in house from our warehouse management system and supply chain tools to order management and everything else.

That amount of compute encompasses QA, Dev environments and any experimental and R&D work we are doing.

Plus jet.com is trying to compete with Amazon so that means the system has to be ready for many million more users than there are currently shopping with us.

[louthy]

I haven't used Jet, no. I'm not trying to call you out, just wondered why it was so large. I have to deal with a similar amount of pricing complexity (actually, probably more complex than Jet) in my line of work, and we don't need anywhere near that amount of resource, but granted we don't have your numbers either, and our setup is much easier to silo groups of users - so probably not directly comparable.

[joneholland]

I work at Expedia. The hotel dynamic pricing engine alone is more than 2200 cores.

And this is c++. It does handle a average of 200k requests per second though.

Don't underestimate true cpu intensive work.

[partisan]

Do tell... what tools/languages/frameworks/architectures do you use?

[louthy]

My comment wasn't so much about languages and frameworks (we use both C# and F#), but pricing complexity. I'm in healthcare software, and the complexity of pricing is crazy.

We have a system that joins hospitals, primary and secondary care providers together. Each hospital might have it's own prices, as will their consultants, etc. Insurers have standard price lists, but some consultants and practices have direct deals with them, which can affect the price (and the creditor for the invoices), we also support rule-sets for modifying prices based on location, clinician, day of the week, availability, benefit maxima, credits, etc. We support the notion of inherited prices, where a standard template can be setup and then used as a basis for more bespoke plans etc. All of which can be manually overridden. If additional services are added then that can affect appointment lengths and costs. If the patient self-pays they may get different prices to what they would on their insurance.

Some of our customers do occupational health where they go out and get contracts with the big city firms to provide schemes for their employees. Each deal they get is different and negotiated to the nth degree, which can include various plans/tarrifs/credits, etc. Some services the patients can book themselves online, some can only be booked by their HR manager, some as follow-ups by clinicians, etc. Also rules around what can go on invoices, how billing items are grouped, who can see what (i.e. should an HR person see what services you've had etc.)

The additional complexity comes with matching that all up with efficient scheduling. We have some customers where they have appointments that involve up to 8 separate clinicians or resources; so they need to be scheduled in and priced without leaving massive gaps in the schedules. We also support call-centres that could see 1000s of practices and can query all of them. The end result looks very much like an airline booking model, but behind it all is a mass of complexity.

The combinatorial issues are large, and have definitely caused us some headaches over the years. So I fully understand how these systems can get very resource hungry. We're not in the same league as Expedia or Jet in terms of scale (although we hold more patient records than Jet has customers, if that means anything) - and as I mentioned it's relatively easy for us to silo groups of users, so I suspect some complexities just go away for us because we can filter down the potential results quite quickly, but still 2200 sounded like a lot - it was just a gut feeling. I don't understand how a shop selling a product can have difficulty calculating a price, but just as my customers have no idea of the complexity involved in calculating prices, I almost certainly don't understand the complexities that the team at Jet have to deal with.

The way we've dealt with it is to build massive data-sets of various static combinations, so if patient X on chargeband Y wants service Z then they'll be changed N. That does mean some large data updates when someone changes their pricing and rules; but it means doing less work live - although it can still be significant. This all runs on two 32 core servers.

[eggy]

What languages and platform are you using?

I know healthcare insurance pricing has become the most insane invention of bureaucracy on the planet. That is why I can pay cash out of pocket and sometimes get an 85% discount. The providers don't have to chase their money from the insurance companies - private and government included.

[baldfat]

I used jet.com and I got my box from Amazon :)

[brianwawok]

Yay dropshipping

[eterm]

Yeah that stood out to me too, since when do people boast about how much resources they consume?

That's 2k customers (not concurrents) per core, which is a terribly low rate. Even if they've only been running 1 month that's 1200 seconds (20 minutes!) of CPU time per customer, and it gets rapidly worse the longer you assume they've existed...

[markfsharp]

There are planning to take on Amazon, for one, but not sure what they are doing internally so couldn't possibly comment on that front.

[cm3]

What concurrency models do you employ to saturate all cores, and how's GC behavior?

[gmantom]

F# has fantastic concurrency support. Great support for Asynchronous operations and Multi Threading. Being a functional first language and immutable by default means most of the micro services are stateless and this means we can take advantage of concurrency like crazy and not worry too much about race conditions. I think functional languages in general make concurrency much easier not just F#.

GC on the other hand is very aggressive with all of the immutable data structures F# creates. GC in F# is very good, though, I think without good garbage collection you have a tough time in a functional world. Microsoft is especially interested in GC performance for .NET and they have explored memory dumps to improve GC so it performs well even under load and when used with F#.

[cm3]

I should have been more specific in the question. I'm used to Erlang and Haskell (GHC) concurrency primitives and frameworks. To reformulate: what concurrency models can you employ in F# that don't involve manually managing threads/processes?

[strmpnk]

I work with F# and have a lot of experience with Erlang (less so with GHC and it's lightweight green threading but I've heard good things about it).

F# inherits lots of .Net's primitives which at their core are thread pool based task scheduling with the usual optimized data structures you see from this world. While this might sound primitive, F# takes it further by providing very good functional libraries which abstract this away allowing lightweight asynchronous computations to be passed around as values (the type being Async<'a>).

Alone, this is pretty nice but it wouldn't feel as natural without computation expressions which are similar to Haskell's do-notation but with some generalizations and flexibility added in. This allows asynchronous code to look and feel first class. If you're curious I'd check out https://fsharpforfunandprofit.com/posts/concurrency-async-an... or check similar links on your search engine of choice.

Now, the story isn't complete. If you're comparing things to Erlang, there are also things like MailboxProcessor<'a> and such which allow other common patterns to be easily implemented. The type checking here is a nice bonus over some other approaches to message passing languages. The big minus of course is that it's still a traditional runtime. If you really want Erlang's isolated processes, it's hard to do that well outside of a runtime like BEAM. I also think F# could use better fault-tolerance primitives but I'm actively working on this with heavy influence from my work with Erlang.

[Paradigma11]

For actor model: http://getakka.net/ https://msdn.microsoft.com/visualfsharpdocs/conceptual/contr... For channel/~process calculus: https://github.com/Hopac/Hopac/blob/master/Docs/Programming.... https://github.com/fsprojects/FSharp.Control.Reactive

There is also the Async monad: https://msdn.microsoft.com/en-us/visualfsharpdocs/conceptual...

Ofcourse it is also possible to use standard .net stuff like the TPL https://msdn.microsoft.com/en-us/library/dd537609(v=vs.110)....

[ZenoArrow]

If you want to use an Erlang-style actors in F#, you can use Akka.NET:

http://getakka.net

[eulerfx234]

I'm an engineer at Jet and hopefully I'll be able to answer your question. The concurrency model within F# is based on continuations. The type Async<'a> = (('a -> unit) -> unit) - its a function which accepts a callback to be notified when the async operation completes. The existing .NET ThreadPool is used to schedule these continuations across OS threads. It uses a trampoline to tame the stack. The ThreadPool itself is a fairly sophisticated piece of work, with scaling heuristics, work-stealing queues, etc. The ThreadPool interacts with the Windows IO completion port multiplexer for IO. We extend this primitive in a variety of ways, notably into an AsyncSeq<'a> which in Haskell terms is ListT Async - a linked list interleaved with Async. We use this for stream processing, sockets, fault tolerance, etc. Async is very similar to Haskell's IO monad, although the representations are a bit different.

However, both Async and IO are insufficient to represent disjunctions. To that end, another concurrency library that we use is Hopac. Hopac is an F# implementation of CML, with some differences. Hopac provides a notion of an alternative (called event in CML; think Haskel's Alternative typeclass if you relax the laws a bit) and synchronous channels (note that async channels are special cases of sync channels). Hopac's has experimental support for lawful MonadPlus as well (see transactional events in Haskell = IO + STM + CML).

Some things that we're heading towards next are generalizing STM to be a bit to be more like RCU (see relativistic Haskell). Additionally, we are experimenting with extending this to session types, but nothing in production yet.

F# also provides a MailboxProcessor, which is similar to an Erlang actor, however without explicit distribution support, so perhaps more of an "agent". We typically use this as a low-level concurrency primitive, rather than a full-blown programming model. Most of our services are compositions of various request/reply interactions, and the Async model above is a great fit for this. In fact, we've primitives centered around the notion of an arrow 'a -> Async<'b> (specialized to Async). These primitives provide support for fault tolerance, logging, tracing, etc.

All of this works well with the GC. Async does cause allocations of course, but this is a price we're more than willing to pay. We've shared some GC dumps with the designer of the .NET GC and she believes they are sensible for a functional language. Hopac took optimization to a greater extreme, reducing allocations where possible.

Another F# library of interest is MBrace. This takes the notion of Async and fits it with a scheduler that schedules across a cluster rather than an individual instance.

Hopefully this helps!

[eulerfx234]

Also, to compare with other languages:

Async is similar to a Future (such as in Java), with the difference that Future produces a result once and caches it, whereas Async re-evaluates each time it is executed. It can be made to cache of course. A Future is more like a TPL Task, though IMO, Async provides a more predictable programming model.

Go has go-routines and channels. A go-routine is similar to Async. In essence, they are a notion of light-weigh thread. Note however that Hopac support for channels is far richer than that of Go.

[bad_user]

> async channels are special cases of sync channels

You got it the other way around - synchronous is a special case of asynchronous, because any synchronous result or stream can be processed asynchronously, but for having guaranteed synchronous results you're adding restrictions. And going the other way, from async to sync is not possible without blocking threads, which is an error prone, platform specific hack. Take the possibility of blocking threads away and you'll notice the true nature of these models.

[eulerfx234]

In CML/Hopac, async channels (buffers) are implemented in terms of sync channels - there is not async channel primitive built in. Synchronization is the essence of this model. When an operation is waiting on a matching communication through a channel, it is suspended, but no OS thread is blocked.

But yes, going from async to sync requires blocking, which is why CML/Hopac takes the approach of making sync the core primitive.

[cm3]

Thank you, that satisfies my curiosity.

[klibertp]

Futures, for one (called Tasks I think). There are probably implementations of other models as well, but I never looked for them personally.

[nickpeterson]

I would assume you guys are making heavy use of the sustained low latency GC modes that got introduced in .net recently, or do you use the default?