Hacker News new | ask | show | jobs
by gingerlime 5053 days ago
Great talk, and without having any experience with FP, it really makes sense on many levels. I love data, and how transparent it is, and how objects seem to get in the way a lot of the time. I like queues, and shipping data from one process to another rather than sharing objects. RESTful interfaces, etc. Those concepts and tools are powerful.

The only thing I'm not too comfortable with is that space isn't really infinite. Yes, it's much cheaper, but still not infinite. If we stored all our logs in an ever growing database, and expect to be able to access it all the time, this is really very expensive. This is why we rotate logs, archive them and trash them eventually. Sure, we can afford this expense for source control, because this data (source code) is amazingly small in comparison. I'm not sure how it translates to real data on our systems, which is immensely bigger.

Also thinking about it in context of technologies like redis. redis manifets a lot of the advancement in technology in how memory is used. It's so vastly bigger and cheaper than before that we can afford to store our whole database in it, instead of on much slower disks. But then this super-fast in-memory database definitely faces storage size constraints that needs to be considered...

Just a few random thoughts. Wish I could have a chat to Rich Hickey one day. Even if I could, I have a lot more to learn until then, so I'd make the most of this chat.

2 comments

I think the notion is like garbage collection (calling "new", as he mentioned) — the illusion of infinite space. (http://mitpress.mit.edu/sicp/full-text/sicp/book/node119.htm...)
> This is why we rotate logs, archive them and trash them eventually.

i think an organization trashes old logs for out-of-band reasons - acquiring more disk space requires following an organization's procurement process which imposes tons of friction, or because compliance with applicable regulations requires saving, e.g., emails, for three months and saving them for longer is a legal risk.

Or... because I don't really care what what time Postgres started up 2 years ago.
Old logs may be a privacy liability; and depending on the type of log and the load on the system, the fully loaded cost of keeping logs indefinitely may be too high to justify based on the revenue for a customer (I'm thinking of ISP logs in particular, for example).