[JSR_FDED]

I wrote a trading system where the strategies are written in Lua. It has been a delight, fast and simple. The traders have the expressivity of a full programming language and I can add any function to that language and provide any data as well so that their programming (and execution) sandbox is extremely ergonomic and suited to their specific task. In other words, a trading DSL.

Other code outside the sandbox pulls in up to date price data and if certain safety rules are violated will automatically close out positions. So even if the traders code their way into an infinite loop or make other mistakes, the supervisor can step in and automatically prevent disaster.

Using Lua to make a language for others has been a wonderful experience. FYI, it was approx 11K lines of Lua).

[rhelz]

Man, I tried at Bloomberg to get my managers to let me incorporate lua into Tradebook's trading system. They just simply couldn't get it. I'd try to explain and they would look at me like I was from mars or something.

Eventually they got tired of me pitching for it and fired me. How did you ever get managerial buy-in for something like this?

[catlifeonmars]

Did you try to justify in terms of reduced costs or increased sales?

[rhelz]

Certainly. The problem is, not all cost savings are welcome. What if your manager is halfway through a 2-year, $5 million project----which you made obsolete in 2 days using a swig + lua + a 1-page long script?

Is your manager going to go to his manager, and tell him he just wasted $2.5 million? When his manager has promised him a big bonus if he completes the project by the end of the year?

---///---

I've seen--very understandably--comments like "there's no way that could ever happen" or "this guy must be very hard to work with in other ways." Not so.

The problem, was a culture clash between how west coast/silicon valley software companies work, and how east-coast companies work. Not that one is necessarily better or worse than other, but they are very different and if you are slow to pick up on that (as I unfortunately was) its very easy to get burnt.

[catlifeonmars]

I’m so sorry this happened to you. I’ve been extremely fortunate so far, and if this had happened to me early on in my career I would have been absolutely crushed.

[rhelz]

Thanks. It does help with the crushing, though, that apparently somebody got it to work--thereby proving, at the very least, I wasn't crazy.

[chipdart]

> Eventually they got tired of me pitching for it and fired me.

I'm sorry but your anectode is simply not believable. No one gets fired for suggesting a harmless feature.

[cess11]

If you repeatedly contact management with the same proposal you risk coming across as dissatisfied in the organisation and sometimes there's very little tolerance for this, might be because it's just a tyrannical place, but could also reflect the type of business and customers.

[theplatman]

If you’re the type of person who won’t just move on from an idea when you’ve been repeatedly told no that indicates you’re probably a difficult person to work with in other ways.

That kind of behavior is less welcome in enterprise environments where they want you to get in line with the top level goals.

From my experience working at Bloomberg, there was a limit to how much you could rock the boat as a dev and in many ways the technology/language choices were limited to the approved stack.

[jnordwick]

bloomberg has some great teams though. i had lunch with the bond pool team about 5 years ago, and oh my gd they were good. plus the snack floor is awesome in the main office.

[rhelz]

I loved most of my time working at Bloomberg. Super smart people, a very engineering-centric culture. The politics, however, were very different from the west-coast companies I worked at before, though. You might not think that there are much politics where you work, but that just means you are so familiar with the politics that you conform to them without even thinking about it.

But every organization has its sacred cows, and no matter where you work, if you happen to get yourself into a situation where in order for you to be right, your bosses have to be wrong, it's game over.

[theplatman]

Yeah I definitely agree that they’re doing some great engineering work in the company.

It’s a strange place because it’s fully owned by Mike Bloomberg without any board or other group of shareholders that oversees his decisions. There is a “board” equivalent but they report to him. It’s great in someways like how they’re able to spend so much of their profits on philanthropy but leads to a lot of quirks.

[karmajunkie]

i have a colleague like this whose main avenue of logic seems to be wearing teammates down. i wouldn’t fire him over this, but gawd are there days where i wish he’d move on. we’d be losing out on a few skills but i’d dread meetings a little less.

[chipdart]

> If you repeatedly contact management with the same proposal (...)

My point is that at most presenting a proposal is only tangentially related with a grievance that's important enough to warrant firing someone. No one gets fired for reaching out to their boss and say "hey, I think I can improve this". It's a scenario that's unbelievable.

[rhelz]

// It's a scenario that's unbelievable //

I understand your skepticism. Here's the thing: pretty much every time you go to your boss and say "hey, I think I can improve this" you are going initially get blown off. Every organization has inertia, and--lets face it--most clever ideas don't really work out anyways.

If you really want to effect change in an organization, you have to champion it. Your bosses are already busy with other things, you have to get their attention and get on their agenda. If they raise objections, you have to find answers to their objections, and ask them again. And again.

How do you know whether you are the heroic technical visionary, or whether you are just "that guy" who never will just shut up and go away?

More to the point---how do you know how to raise the issue to your bosses, and how many times you can raise the issue to your bosses, before they class you as "that annoying guy."

The answer depends upon the politics and culture at your company. Some things are considered rude in some cultures but not in others. If you are politically acute enough, you can sense when you ar going too far, or you are kicking some sacred cow, or interfering with some powerful person's agenda.

In my case, I had always worked at West coast companies, and I didn't pick up on these cues. Am I hard to work with? Am I "that guy"? Yeah, I guess I was, but I have worked at other companies which would have really appreciated my approach.

BTW, I'm not making value judgements here--when you are dealing with somebody else's money, and in areas where there are a lot of government regulations, etc, a more conservative culture is entirely appropriate. Alas, I'm a pretty good programmer, but slow to pick upon these sorts of things. Live and learn.

[nattaylor]

A friend built a system where the ad campaigns with all their targeting rules were Lua scripts and it was also fast and simple in a glorious way!

[ing33k]

I tried a similar approach with a team I was working with. We were building it on top of redis and after some basic benchmarks gave up the idea and figured out from the docs that that eval script is blocking.

[bastawhiz]

Lua in Redis is not useful for serving as a runtime for running Lua application code; it's valuable for allowing you to perform a series of steps atomically. You can read and write data without worrying about something else changing: only one piece of code can write at a time. We use it for rate limiting, which requires reading and writing atomically, for instance.

[btilly]

When I did that on a project I solved that problem by replicating redis to the same container that the ads were being served from. Replication was very fast, and all that was blocked was the local redis. I worked to make the matching rules in the script efficient, and the blockage was truly not a problem.

[catlifeonmars]

Doesn’t local redis kind of miss the point of using redis in the first place? On the surface, that’s a big chunk of additional complexity for something that could be done internal to an application. Was this meant to be an incremental step in a larger refactor?

[ing33k]

well, redis manages the replication without fuss. Unless the framework / language one uses supports it out of the box, there is no point in re implementing replication.

https://www.erlang.org/docs/17/reference_manual/distributed

[catlifeonmars]

Oh so this would effectively serve the same purpose as a centralized cluster.

[knighthack]

Why wasn't Python used? I imagine for such a use case it wouldn't be that much heavier to use Python vs Lua.

[catlifeonmars]

Integration complexity could be a factor. I’m making an assumption about the architecture, but embedding Lua in a program is dead simple and you can do so without introducing external dependencies. Python IIRC requires you to ship and package the standard library or have it already installed, the Lua interpreter can be statically embedded in a program.

[qwertox]

Speed, probably. Lua is much faster.

[nattaylor]

Yes, speed. 1,000,000+ QPS and 1,000s of ads to evaluate is common. At that scale it's very distributed, so replication is another challenge.

In a way, its a search problem where bid≈relevance and targeting_match≈recall, so I've seen Solr used here too.

[theturtletalks]

Would love to check it out if you’re accepting new users.

[dist1ll]

What latency range are your strategies aimed at? Are you executing strategies on CPU, or lowering the DSL to some kind of HDL or ASIC config?

[edem]

where can i try this system? I've been working on my own but if there is already one out there i might use that instead

[jasfi]

Which language did you write the engine in?