[pixelpoet]

(Aside about Zig, sorry. Although this applies to Go as well, I think?) Urgh I am so keen to switch to Zig but their attitude towards having vector operators just completely kills the viability for me as a graphics programmer.

I've asked in their Discord, Andrew Kelley himself passed on commenting (I know his stance, every C++ dev wants their fav feature), but the reality remains that it's just infeasible to do with a DSL so it's just the wrong language for writing graphics code.

[zoogeny]

I was watching a recent talk about the new Mojo language [1]. There is a section on SIMD and how they treat scalar values as a special case of vector operations (around the 33 min time).

It does seem that tensors are one of the core abstractions for modern ML systems. I've heard people joke that AI is just matrix multiplication. Python is such a flexible language that creating abstractions around its syntax was super easy. I believe that was part of the reason Python has come to dominate this space (not the only reason obviously).

I too felt the same as you, but as a distant admirer of Zig. I totally understand that operator overloading can be misused. But I have an intuition that at least providing operators for linear algebra (and probably complex/quaternion) is a really important feature for any languages from this point in history going forward.

1. https://www.youtube.com/watch?v=SEwTjZvy8vw&ab_channel=LLVM

[xh-dude]

Julia has good ideas, interesting trade-offs here as well. ISTM the deep problem is that there’s so much room for optimization under composition of operations or with invariants not captured in types that the typical LinAlg library will never capture. But I agree most languages should be anticipating that users need something helpful here.

[andy99]

  providing operators for linear algebra (and probably complex/quaternion) is a really important feature for any languages from this point in history going forward.

This is why I have started using Fortran for writing AI model inference code. It natively handles array manipulation like python and has a `matmul` intrinsic and compiles into fast code. There are some rough edges, but it's great as a low level matrix programming language which was its original point.

[sevagh]

>I've heard people joke that AI is just matrix multiplication

Not sure where the joke is, today's deep learning based AI models are literally matrix multiplications with learned weights.

[ForkMeOnTinder]

That's one of the downsides of a BDFL. Zig is like 98% amazing, and 2% strange decisions that I think could have been avoided if the creator had more experience with languages other than C and Javascript.

I'm still a happy Zig user though, and hey, there's still time before 1.0.

[pixelpoet]

He's actually fluent in a lot of languages[0]. Just somehow apparently not much sympathy for numerical computing, or UI code (I'm fairly sure you want 2d vectors there too), or plotting, or ...

[0] https://andrewkelley.me/post/not-a-js-developer.html

[dilyevsky]

Not supporting operator overloading is not a “strange decision” it’s widely hated feature outside of very specific domains (like maths and graphics) and every shop i worked at which bothered with a c++ guideline banned it

[tester756]

You guys are "interesting" in this c++ world

C# has operator overloading and during my whole career I have never seen it abused so hard that people needed to ban it, let alone write guidelines and a lot of shops adopting it.

I barely see anyone use it not for really good cases like graphics.

The only interesting case was using "/" operator for Path Combines so "home" / "folder1" performs Path.Combine("home", "folder1")

but still, that was PoC or lib, not even prod.

So, is it about community, some culture or actually what?

[dilyevsky]

I think it’s mainly c++ devs are very performance sensitive (why else would you subject yourself to this torture?) so they hate it when your ‘+’ is suddenly O(n^2) and with side-effects

[tialaramex]

C++ operator overloading gets banned because C++ programmers invariably abuse it and cause problems, even the standards committee can't help themselves, which is how I/O Streams are a standard library feature.

C++ also allows you to overload short-circuiting operators, but of course your overload can't short-circuit so you just silently destroyed an important feature. Why ?

As others have pointed out, several languages have been able to provide this feature without causing half the mess and disappointment. Ten years ago if you said move assignment semantics are a bad idea you might persuade people because the C++ move semantics are messy, but hey, turns out a fresh language is able to just provide the destructive move developers actually wanted (but couldn't pull off for C++) and that's really nice.

[throwawaymaths]

> even the standards committee can't help themselves

cout << "foo" << endl;

[snovv_crash]

Banned for in-house use, sure. Require an extra review before including a library, sure. But don't tell me you refuse to use Eigen and force everyone to write their own matrix multiplication routines that don't have operator overloading...

[dilyevsky]

That is why I specifically mentioned math and graphics as exceptions. Honestly I still like explicit DSL model better for these than trying to build dsl on top of your existing language c++/lisp style

[pcwalton]

But you have to use operator overloading to use smart pointers.

This isn't a pedantic nerd snipe; the point is that operator overloading really is indispensable in certain contexts.

[truckerbill]

It is strange that we stop at scalars for algebra in most languages. Odin’s approach is almost more frustrating, allowing for quaternion types, but not clifford algebra which can elegantly describe them and much more(though i know it’s a comically niche request at this point in time)

[pixelpoet]

The main thing that stands out about this comment to me (agreeing 100% about algebraic types) is the "at this point in time" bit, such boundless optimism :D

[reactordev]

agreed, further with today's parallelism it should be trivial but like you said, most languages stop at scalars and force the onus on you. I get it, higher order math is an implementation detail but to not provide the proper tools for those implementations is frustrating.

[el_oni]

Odinlang is a language in a similar space that seems to have first class support for matrix and vector operations. As well as having built in support for various graphics apps [0].

Seems like there is a bunch of interesting low level languages gaining steam at the moment.

[0] https://odin-lang.org/news/major-graphics-apis/

[lenkite]

Does Odin have support for async/concurrency ?

[lll-o-lll]

Doesn’t this open github issue imply that vector operators are still being considered? https://github.com/ziglang/zig/issues/7295

Or is it that the three years it’s been around indicate it will never progress?

[pixelpoet]

This is far better than my attempts at initiating the discussion, thanks for linking!

Also it seems like even the discussion of Geometric Algebra has been covered before 3 years ago: https://github.com/ziglang/zig/issues/7295#issuecomment-7389...

Unless it's actually possible after all this time (even in some random beta) to do vector code in infix notation, it doesn't feel like it's going to happen :(

[crest]

Didn't Zig "just" decide to permanently stay irrelevant for performance critical code by replacing LLVM by a yet to be written home grown optimiser and code generator? Don't get me wrong LLVM has lots of warts, but the a good multi architecture optimiser and code generator is a larger project than the frontend and standard library for any reasonable language.

[throwawaymaths]

No. They are turning llvm into an "optional dependency", or plugin, or call it what you want. Day to day/debug compilation will be done with their own compiler and if you want you can cut a release with llvm to get all the optimizations when you're done, or at any time really.

Zig won't ship with llvm as part of the standard download, but i imagine it will be easy to get zig+llvm working

[renox]

What's really sad is that not having overloading "ON" by default is a good idea, but it doesn't mean that language has to stop there, it's really simple to have a small language feature which would say that

a #everything_until_the_next_space b is rewritten in #everything_until_the_next_space(a,b)

So you'd have:

-a #foo b #foo c is foo(foo(a,b),c)

-a #foo b #bar c is forbidden, use parenthesis.

-and maybe #: for the inverse composition where a #:foo b #:foo c is foo(a, foo(b,c))

It's explicit, no hidden overloading, yet it's "short" enough to be usable: (a #* b) #+ c isn't that much worse than a*b+c and is much more readable than madd(mmul(a,b),c)..

If there is no implicit conversion name clashes shouldn't be too bad (both #+ could be usable for matrix and for graphic libraries).

[__loam]

Am I crazy or is there nothing stopping us from writing functions that work with vectors? I don't really understand why this is a big headache for graphics programming other than it not being your preferred syntax.

[Night_Thastus]

I'm a C++ dev, could you explain a bit of what is missing in Zig for vector operations? Does Zig not have operator overloading?

[klyrs]

Nope, no operator overloading. As I understand it, the philosophy is to not hide O(n) behavior behind notation that looks O(1).

[ForkMeOnTinder]

And what's funny about that stance is mathematical operators aren't actually O(1).

https://en.wikipedia.org/wiki/Computational_complexity_of_ma...

You don't want to know what the innocent-looking `*` in this function compiles to:

  fn square(num: u10000) u10000 {
    return num * num;
  }

[throwawaymaths]

Is the underlying algo not O(1)? Under what pairs of two u10000 numbers would you get different execution time?

[ForkMeOnTinder]

Sure, if you ignore the n in n=10000, multiplication is O(1). But the same is true for e.g. heapsort--all lists of 10000 items take asymptotically the same amount of time to sort.

But that's a strained interpretation of complexity, and not very useful.

[throwawaymaths]

A heapsort is designed to take a variable number of items at runtime for given code. The n is fixed at compile time in the multiplication examples and is invariant at comptime (in zig).

[pixelpoet]

It's mildly offensive that u10000 is a builtin type but vec2f is not (also note that I am not asking for general operator overloading!). Which has dedicated CPU instructions across all modern architectures, what's the relative usage frequency, what's the cost/benefit of each, etc etc... :( And we all know why u10000 is in there: because you have to have arbitrary-width integers when writing a compiler, so they figured eh, we'll just expose that because we have it already. Absolutely transparent compiler-programming (cf. rendering-programming) bias, almost nobody else needs that.

And yeah, I get that Add(Add(Add(a, b), Mul(c, 3)), d) is possible, but come on... imagine if you had to write your normal "a + b + c * 3 + d" with ints/floats like that! What's that, suddenly people care, but nobody cares if it's not their field...

Whatever, I will continue to look longingly at Zig for all the bits of C/C++ (and apparently Go, to try bring it back to the original topic) it solves, but missing the trivial and absolutely critical single feature to enable an entire class of performance-critical programming.

[bsder]

> It's mildly offensive that u10000 is a builtin type but vec2f is not

u10000 exists only because we want u3, u7, and u13 as builtin types.

u3, u7, and u13 are useful for embedded and systems programming.

[defen]

> And we all know why u10000 is in there: because you have to have arbitrary-width integers when writing a compiler

You don't need them as a built-in type to write a compiler. They're there because LLVM was the original backend and you essentially get them for free (in the sense that the backend code generation is already handled for you, so why not include them).

[tuhats]

> It's mildly offensive that u10000 is a builtin type but vec2f

Zig does have builtin vec2f, it is spelt @Vector(2, f32).

[ForkMeOnTinder]

@Vector is counterproductive for cases like vec2f, since it tries to force data into SIMD registers even when it would be more efficient to leave it in normal registers. In fact I've wondered in the past if Zig might end up slower than C for graphics code, if people misuse @Vector to get normal math operators back. For that reason I never use @Vector unless I already know what SIMD intrinsics I'd be writing in C.

[Night_Thastus]

Strange.

How is that different from say, a function call? A function may look like a single O(1) operation from the input/output/name, but actually do something much more complex. That seems like the same thing to me, and very common. (and frankly I'm not sure that could even be avoided)

[klyrs]

I didn't mean to volunteer to defend this choice, but without investigating a function you can't really support an opinion about its runtime. A language can make such promises about its basic syntax however.

Perhaps I'll rephrase how I understand the philosophy: if it's a function call, it should look like a function call. Operator overloading breaks that.

That said, this isn't my hill to die on.

Edit to clarify my final sentence there: I have zero interest in debating this any further. Pixelpoet, if you're going to be so fussy, go read Harvey and van der Hoeven and stop trying to win language fights, they're tedious.

[pixelpoet]

I appreciate your being such a good devil's advocate, however as ForkMeOnTinder points out, there is already the super complicated[0] overloaded O(N^1.58+) operator for arbitrary precision integer multiplication, easily argued to be vanishingly less useful than simple vector operators, particularly if they were well-expressed at the IR level and well-mapped to modern hardware / instructions.

The ask isn't for general operator overloading (I'm also in favour of not having that), rather just not stopping native algebraic type support at scalars; the C function atan2(y, x) basically just wants to give you the complex argument, for example. Really it would just do so much to unify and simplify everything, besides being able to write vector stuff in a sane way; if every rando has to write their own vector and complex number classes, I'm much less likely to vouch for its correctness.

[0] I've recently been looking into Karatsuba multiplication to reduce it from O(N^2) to O(N^1.58): https://en.wikipedia.org/wiki/Karatsuba_algorithm

[Night_Thastus]

To me it does seem a weird hill to die on. If I'm using an operator on a non-primitive/non-trivial type - I'm going to consider the possibility that it does something more complex. It would be strange not to.