[brabel]

I didn't investigate this more deeply , but I think if you used an arena allocator on each iteration of the loop, and simply freed the whole allocator after each iteration, you might get much better performance.

EDIT: the MNIST link requires authentication: http://yann.lecun.com/exdb/mnist/ How can we get access to the test run at the end?

[camel-cdr]

Why would any DNN ever need to deallocate other than at program termination? (And allocate other than at startup)

[brabel]

That's the point. With an arena allocator, you don't "really" de-allocate the OS memory, you just free the bytes to be re-used again in the next iteration (by using a FixedBufferAllocator as the backing allocator, if you know how much memory will be needed in advance). This is much better than trying to re-use the structs manually as the OP mentioned at the end.

EDIT: FixedBufferAllocator has a reset method , so maybe you just need to use that to "free" all memory on each iteration? Perhaps using that directly is enough if you know the max memory usage at compile-time... I was just thinking that it should be possible to use an arena allocator to basically tell the delegate allocator to make all memory available again without freeing it back to the OS?! If anyone knows more about allocators I would be happy to know this. https://ziglang.org/documentation/master/std/#A;std:heap.Fix...

[dundarious]

FixedBufferAllocator is just a "bump allocator", i.e., just a size and a pointer that moves forward. You allocate the memory it uses from somewhere else (stack, a big alloc from a general purpose allocator (think malloc), some pages from the OS via mmap or VirtualAlloc, etc. -- zig has simple cross platform functions for all of these).

Individual "allocations" just bump the pointer forward, and it succeeds unless the result is past the end of the fixed size given to it initially. You don't free any individual allocations (except maybe the most recent one by just moving the pointer back again), you just reset the pointer to the very start when you're done with all the allocations.

[kuroguro]

> the MNIST link requires authentication

doesn't ask for auth for me?

[brabel]

It's working now! I guess someone opened access after I checked.

[version_five]

I just tried and they downloaded for me...