[colejohnson66]

As someone who doesn't write C++, why does almost everyone seem to insist on ignoring the STL and writing everything themselves? Both C++ and C# take a "bags included" approach to the standard library, but no one is writing their own `List<T>` for C#.[a] Yet, everyone seems to have their own opinion about `std::vector<T>`.

[a]: Obviously, people still write their own collections/containers in C#, but they tend to only do so for very specific/performance-sensitive circumstances.

[lenkite]

Because C#, at several points in its history, sensibly took the decision to explicitly break backward compatibility to satisfy legitimate requirements, adapting and evolving to the tenets of modern software practices. (Generics and new collections)

Whereas due to C++'s "never break compatibility" decision, the standard library has progressively decayed over time. It has become a bloated, rotting dinosaur where even the slowest of interpreted languages can comfortably beat several of its aspects. (Ex: std::regex is pathetic and pitiful, vector<bool> triggers laughter, substandard maps, etc). Considering that C++ thumps its chest and loudly proclaims its superb performance, this has now become a sad joke.

In the natural world, a species that cannot adapt to new circumstances and never discards undesirable characteristics simply perishes.

The C++ Standard Committee has firmly and unequivocally decided that the C++ language should mirror the same approach and limp down the road, carrying the full-weight of its sins for all its journey, until it falls into oblivion.

[pjmlp]

Note that as documented in the history of F# HOPL paper, .NET already had a generics prototype before 1.0 release, Microsoft decided to go ahead without generics to avoid delaying it further until they were stable.

Also breaking everything a couple of times is why most corporations are nowadays stuck in a Python 2 / 3 parallel world in .NET Framework / .NET Core , or in Xamarin.Forms / MAUI, UWP / WinUI,...

[sfpotter]

There are so many reasons. I'm sure others will provide their own favorites.

One that I find particularly annoying lately: if you work on a project that makes heavy use of the STL (or, really, any heavily templated library written in a similar style, with a focus on "ergonomics" :-( ), you'll quickly find that backtraces for debug builds can easily be 50-100 levels deep with most stack frames just consisting of incomprehensible layers of abstraction which get optimized away. So, debug builds are totally useless, and build times are typically long. Contrast that with something written in "C style" or simply making far fewer use of C++ features, written in a more straightline style, with fewer levels of abstraction: builds will be fast, stack traces will be small and can easily map directly onto the concepts relevant to the program or library, and debug builds are useful once more. Night and day difference.

[SleepyMyroslav]

It does not have to be night and day. In gamedev I saw over years that giving up debug builds for a set of your own abstractions has benefits. You do not have to use STL or exceptions so you can have a balance in your build times. Having abstractions in the code has a lot of value. People overestimate how hard is to do deoptimize on checkout or on demand which solves most of debugging needs.

[sfpotter]

I appreciate the sentiment of what you're saying, but I guess I'm not totally sure what you mean by "abstraction". Maybe "abstraction" is the wrong wording for my original post... For example, in boring code you can have "abstractions" in the form of data types that model high-level concepts along with high-level functionality on them. I'm not at all opposed to that. The "Sane C++ Libraries" linked seem to be a reasonable example of this style. There are no features in C++ that you absolutely need beyond what's offered in C to do this.

Maybe as an example just compare a library like Eigen to a similar imaginary library written in C.

Eigen leans heavily on C++ features to reduce line count and make something look visually more mathematical or maybe more MATLAB-style in the name of ergonomics; and obviously on the backend it relies on the behavior of the C++ compiler to simplify the elaborate template expressions and make the code efficient. If you ever try to step through code that uses Eigen in a debug build or examine a stack trace from inside Eigen where an exception has originated, the situation is not pretty! Contrast this with what will happen in the imagined C-style library. If all the heavy lifting happens in BLAS or LAPACK and the C library is basically a thin library to make things a little more automatic and easier to manage, the stack traces will be short and each stack frame will be easy to digest at a glance because of names like "mat_mat_mul" or similar, instead of "mat<double, double, 4, allocator<...>> const & operator*(mat<double, double... 159 more characters of template gobbledygook that makes the eyes glaze over)". The former will also be significantly faster to compile.

Anyway, I guess I just don't see how the latter STL/Eigen/whatever-style approach to things is an improvement over the far simpler idiot style of doing things.

I'm not trying to be argumentative here, I just want to clarify my point as much as possible.

[SleepyMyroslav]

yeah, you have it completely consistent.

My point is that one can have a lot of C++ working at scale. Reading code is very important part of large scale projects. Dropping debug builds is a reasonable tradeoff.

[sfpotter]

OK, got it... I suppose that's reasonable, but depending on what you're working on, I'm not sure seriously reducing the debuggability of your project is acceptable. For the kind of work I do, I need a debugger.

I would also argue that keeping it so that you can always easily run your project "REPL style" from a debugger exerts a very favorable downward force on all the complexities that make large projects hard to maintain.

[jeffbee]

In decades of professional C++ experience I have never noted depth of backtrace as a usability issue. Give us an example of how a 100-deep backtrace arises as a consequence of the STL.

[summerlight]

Agreed, at least for STL containers I didn't really have to dig into the implementation for 99.9% of the cases. One or two specific old cases were due to memory corruption, but nowadays address sanitizer should do the job. Of course, a lengthy symbol for heavily templatized functions/classes can sometimes be a problem, but it's a separate issue from stack trace depth.

[sfpotter]

"Functional" C++ with deeply nested loops implemented using stuff from <algorithm> and <functional> and lambda.

[o11c]

Before concepts it was certainly possible, but when it happened you just had to add some extra type assertions to make it clear where the error is.

[jsheard]

Take your pick: undercooked functionality, lack of behavior/performance consistency between different toolchains, bloated implementations causing long compile times, inscrutable template vomit error messages and poor performance in debug builds (a dealbreaker for realtime apps like games), and design mistakes which can't easily be fixed, such as std::vector<bool> being a footgun or the way that the standard map types are defined making them impossible to implement efficiently.

[jandrewrogers]

The containers in the STL have issues due to the limitations of C++ when they were designed, catering to the lowest common denominator user, and simplifying things for implementors. It is not uncommon to find cases where the STL is a poor fit or annoying to use due to its implementation and design details. In the case of modern C++, you also have the issue that the STL containers aren't designed to be used in an advanced metaprogramming context, because they pre-date that being an intentional part of C++. Some of this is unfixable because backward compatibility. Also, C++ is commonly used in contexts where performance is critical, so the optimality of the STL for purpose matters more.

Some of the common issues: static allocation or lack thereof, requiring default constructible classes, initializing memory you are going to overwrite anyway, inability to be used in some metaprogramming contexts, suboptimal allocation behavior, etc. The STL is opinionated but unfortunately that opinion dates to a time when C++ was primarily used for ordinary app development, not high-performance code.

Like many, I maintain my own C++ "standard library" that is much better designed for the kinds of software I tend to work on (database kernels and data infrastructure, mostly).

[blovescoffee]

Because people tend to write c++ for performance reasons and the perf profiles of std::vector are not always sufficient

[jeffbee]

The only thing you can really quibble about with std::vector is whether your library has made an optimal choice of growth strategy, which you can often hack around by reserving. Aside from that, access via `operator[]` and growth via `emplace_back` will compile down to optimal code that is going to be close to impossible to beat. After the compiler gets done with it, it looks the same as if you had hand-coded it with arrays in a C-style, but without the plethora of bugs that often results from that approach.

[dataflow]

> The only thing you can really quibble about with std::vector is [...]

That's actually not true, though I certainly don't fault you for believing it :-) but there are definitely more things to quibble about around vector if you're serious about performance. As an example, try writing a can_fit(n) function, which tells you whether the vector can fit n elements without reallocating. Observe the performance difference between (a) a smart manual version, (b) a naive manual version, and (c) the only STL version: https://godbolt.org/z/88sfM1sxW

  #include <vector>

  template<class T>
  struct Vec { T *b, *e, *f; };

  template<class T>
  bool can_fit_fast(Vec<T> const &v, size_t n) {
    return reinterpret_cast<char*>(v.f) - reinterpret_cast<char*>(v.e) >= n * sizeof(T);
  }

  template<class T>
  bool can_fit(Vec<T> const &v, size_t n) {
    return v.f - v.e >= n;
  }

  template<class T>
  bool can_fit(std::vector<T> const &v, size_t n) {
    return v.capacity() - v.size() >= n;
  }

  struct S { size_t a[3]; };

  template bool can_fit_fast(Vec<S> const &, size_t);
  template bool can_fit(Vec<S> const &, size_t);
  template bool can_fit(std::vector<S> const &, size_t);

[aw1621107]

I was curious why the std::vector version produced so much more assembly, so I tried to dig a little. At least in libc++, the relevant code is (cleaned up):

    template<typename T, 
    class vector {
    private:
        pointer __begin_;
        pointer __end_;
        __compressed_pair<pointer, allocator_type> __end_cap_;
    public:
        constexpr const pointer& __end_cap() const noexcept {
            return this->__end_cap_.first();
        }
        constexpr size_type size() const noexcept {
            return static_cast<size_type>(this->__end_ - this->__begin_);
        }
        constexpr size_type capacity() const noexcept {
            return static_cast<size_type>(__end_cap() - this->__begin_);
        }
    };

So if the functions are fully inlined we should end up with

    template<class T>
    bool can_fit(std::vector<T> const &v, size_t n) {
        return static_cast<size_t>(v.__end_cap_.first() - v.__begin_) - static_cast<size_t>(v.__end_ - v.__begin_);
    }

At least algebraically (and ignoring casts) it should be equivalent to v.__end_cap_.first() - v.__end_, which is more or less what the manual implementations do. Maybe the optimizer can't make that transformation for some reason or another (overflow and/or not knowing the relationship between the pointers involved, maybe)?

If you change can_fit(Vec<S>) to:

    return (v.f - v.b) - (v.e - v.b) >= n;

You end up with code that looks pretty similar to the can_fit(std::vector<S>) overload (the same for clang, a bit different for GCC), so it does seem it might be something about the extra pointer math that can't be safely reduced, and the casts aren't really relevant.

(I'm also a bit surprised that can_fit_fast produces different assembly than the Vec can_fit overload)

[dataflow]

Kind of, yeah. The main thing to realize here is that pointer subtraction is not just arithmetic subtraction of the addresses; it's also followed by a division by sizeof(T). (This is not obvious in this context unless you've seen it before.) Thus for the compiler to prove that (f - b) - (e - b) == f - e, it has to prove the remainders of each address subtraction (mod sizeof(T)) don't affect the result. This is certainly possible, but it requires compilers to actually do some extra legwork (e.g.: assume these come from the same array, then prove that implies the remainders don't affect the result, etc.) prior to reducing these to arithmetic operations. For whatever reason they don't do that. My guess is the reason is something between "we believe the general case would break too much existing code" and "we believe this special case isn't common enough to be worth it". But who knows; maybe if someone files this as a missed optimization, they would try to implement it. I haven't tried.

(There's also a secondary issue here, which is that sizeof(S) isn't a power of 2. That's what's introducing multiplications, instead of bit shifts. You might not see as drastic of a difference if sizeof(S) == alignof(S).)

[aw1621107]

Yeah, the extra instructions from the division helped clue me into what might be going on, since the individual (f - b) and (e - b) calculations are visible in Clang's output.

I feel the division by sizeof(T) shouldn't matter that much, since the compiler knows it has pointers to T so I don't think the divisions would have remainders. I want to say pointer overflow and arithmetic on pointers to different objects (allocations?) should also be UB, so I suppose that might clear up most obstacles? I think I'm still missing something...

Does make me wonder how frequently this pattern might pop up elsewhere if it does turn out to be optimizable.

[summerlight]

I wonder if you actually measured it with proper compiler options? I saw some folks insist that STL is somehow slow blaming unnecessary layers but many of them are pretty much groundless. There are some genuine spots where STL has a real performance problems, but I don't think your example illustrates it well.

[dataflow]

Click the link in my post, I added it so you can see everything in action for yourself.

(Although, note I didn't claim "the STL is slow"... that's painting with a much broader stroke than I made.)

[CyberDildonics]

Why would that be faster? Those function calls are going to be inlined.

[jeffbee]

It's not the function calls they are alluding to, it's the way the compiler generates a bunch of shifts and multiplies except in the can_fit_2 case.

[dataflow]

Turns out GCC is more clever than Clang here, and MSVC is just horrendous. (See update, I posted a link.)

[jsheard]

There's also the issue that std::vector<bool> is required by the standard to be specialized as a bitset, which is a footgun in generic code since you can normally take the address of a vector element but not if it's a vector of bool. Having a bitset in the standard library is fine but it should have been a seperate type.

Admittedly that's not a performance issue, but it's annoying.

[jeffbee]

That's true but I think everyone knows about vector<bool> being quirky. By the way, the standard does not require vector<bool> to be implemented as a bitset. Instead, it relaxes some of the details of vector, in a way that allows the implementation to do it that way. But these choices are implementation details.

Vector<bool> is a little weird if you are just starting with C++, but it does have major performance benefits in its niche, and it came from the 1990s so we can be generous in overlooking its rough edges.

[HelloNurse]

It came from the 1990s, but it overstayed its welcome. I remember seeing proposals to remove the vector<bool> special case, what is the situation?

[CyberDildonics]

What specifically is wrong with vector? There have been a lot of hash maps done with flat memory to minimize allocations and pointer hopping over the STL but vector doesn't have those problems.

[summerlight]

> why does almost everyone seem to insist on ignoring the STL and writing everything themselves?

Not really. Unless you have a very specific set of performance requirements, STL containers are usually more than enough. And we have third party libraries such as Abseil/Boost to cover the major gaps in the rest. I do see some legitimate cases to write own container libraries. But for many cases people don't really measure their primary workload before writing such libraries, instead they just write it because they can write (and it's fun).

[MichaelMoser123]

Doesn't it contradicts the 'bloat free' requirement, if every library needs to have its own version of std::list std::vector, std::unordered_map etc ?

I mean: i remember the days, when each and every project had its own string class.

[pagghiu]

The bloat free is more referring to executable size, build complexity, compile time and in general to hidden complexity.

Every library having its own version of common data structure is unfortunately something that C++ programmers can't really seem to agree on :)

[MathMonkeyMan]

The implementations of vector, list, and unordered_map in every standard library I've used have been sufficient for every C++ library I've worked on.

There are special cases, and there are engineering politics, but it's all basically fine.

[neonsunset]

Because unlike CoreLib, the standard library in C++ tends to have a lot of insanity backed in. My knowledge on it is somewhat limited but looking into its APIs for working with strings (in large inherited from C) was so traumatizing that I only ever wonder why there are still self-respecting projects that don't reimplement parts of STL.

Example: want to measure the length of a UTF-8 code point in a string and did not synchronize the call? Well, too bad, now you might have corrupted the global mutable state it relies on! (the fact that you can make such a trivial piece of code have two!!! points of thread-safety failure one is C locale and another is transcoder still refuses to leave my mind)

[neonsunset]

Can't believe someone actually went out of their way to downvote this haha

[Chaosvex]

Sounds like you weren't working with C++'s string API. std::string knows nothing about the encoding of the data it encapsulates and can be used to store anything. There are the codecvt functions but it doesn't sound like you're talking about those, either.