In the same breath he talked about how he wanted to build this “pristine” system with safety and fault tolerance as priority and how he wanted to use raw pointers to shared memory to communicate between processes which both use multiple threads to read/write to this block of shared memory because he didn’t like how chatty message queues are.
He also didn’t want to use a ring buffer since he saw it as a kind of lock
I forgot to mention that we were building all this in C#, as mandated by Mr CTO.
He also couldn’t decide between windows server and some .RHEL or Debian flavor
I doubt this guy even knew what a kernel driver was.
He very transparently just discounted anything he didn’t already understand.
After poorly explaining why he didn’t like ring buffers, he said we should take inspiration from some system his friend made.
We started reading over the system and it all hinged on a “CircularBuffer” class which was a ring buffer implementation.
Okay, that would be a normal amount of bonkers thing to suggest in C or another language with real pointers.
But in C#, that is a batshit insane thing to suggest. I'm not even sure if it's even legal in C# to take a pointer to an arbitrary address outside of your memory. That's.. That's just not how this works. That's not how any of this works!
It is legal to do so. C# pointers == C pointers, C# generics with struct arguments == Rust generics with struct (i.e. not Box<dyn Trait>) arguments and are monomorphized in the same way.
All of the following works:
byte* stack = stackalloc byte[128];
byte* malloc = (byte*)NativeMemory.Alloc(128);
byte[] array = new byte[128];
fixed (byte* gcheap = array)
{
// work with pinned object memory
}
Additionally, all of the above can be unified with (ReadOnly)Span<byte>:
var stack = (stackalloc byte[128]); // Span<byte>
var literal = "Hello, World"u8; // ReadOnlySpan<byte>
var malloc = NativeMemory.Alloc(128); // void*
var wrapped = new Span<byte>(malloc, 128);
var gcheap = new byte[128].AsSpan(); // Span<byte>
Subsequently such span of bytes (or any other T) can be passed to pretty much anything e.g. int.Parse, Encoding.UTF8.GetString, socket.Send, RandomAccess.Write(fileHandle, buffer, offset), etc. It can also be sliced in a zero-cost way. Effectively, it is C#'s rendition of Rust's &[T], C++ has pretty much the same and names it std::span<T> as well.
Note that (ReadOnly)Span<T> internally is `ref T _reference` and `int _length`. `ref T` is a so-called "byref", a special type of pointer GC is aware of, so that if it happens to point to object memory, it will be updated should that object be relocated by GC. At the same time, a byref can also point to any non-GC owned memory like stack or any unmanaged source (malloc, mmap, pinvoke regular or reverse - think function pointers or C exports with AOT). This allows to write code that uses byref arithmetics, same as with pointers, but without having to pin the object retaining the ability to implement algorithms that match hand-tuned C++ (e.g. with SIMD) while serving all sources of sequential data.
C# is a language with strong low-level capabilities :)
In the same breath he talked about how he wanted to build this “pristine” system with safety and fault tolerance as priority and how he wanted to use raw pointers to shared memory to communicate between processes which both use multiple threads to read/write to this block of shared memory because he didn’t like how chatty message queues are.
He also didn’t want to use a ring buffer since he saw it as a kind of lock