[corytheboyd]

I’ve seen it mentioned before, but I haven’t given it a demo yet. To be honest I wasn’t sure if TUI was the right UI for something like this, as it can’t pull off all of the things a GUI can. It’s not that I have a TUI allergy either (I can’t live without lazygit).

[tstack]

There are certainly limits on what can be displayed and done interactively due to the low resolution. Especially, when it comes to charting data. Some visualizations are available within lnav, mostly bar charts and the like. What type of things are you trying to do that you think are not possible within a TUI?

[scottlamb]

lnav is neat but doesn't really do structured logging AFAICT, just predetermined fields for the format.

[tstack]

lnav has support for JSON-lines, logfmt, as well as the Bro and W3C Extended Log File formats that are XSV and self-describing. The contents are also accessible through SQLite tables. Is there some gap here that you're thinking of?

[scottlamb]

The idea of structured logs is that every place in the code where you define a log message, you can throw in extra attributes (key/value) pairs.

As far as I can tell, the lnav feature you describing allows you to define a JSON log format with predefined fields. But if some log message uses an attribute you haven't anticipated in the format definition, there's no way to see it in the pretty-printed output or filter on it in the UI and no ability to see it in the SQLite virtual table. That's why I say lnav doesn't appear to support structured logs.

edit: oh, I missed `"hide-extra": false`! That significantly improves things! Still, I don't see a way to access it from the SQLite virtual table. I also don't see something else I might want: a way to have a different "view" for certain log messages; maybe to switch between filtering/viewing particular ones, maybe to just have line-format be conditional based on the detected format. (I guess I can sort of do this based on `module-field`? but I might want it lighter-weight/finer-grained than that.)

[tstack]

> But if some log message uses an attribute you haven't anticipated in the format definition, there's no way to see it in the pretty-printed output

Properties in the log message that are not in the "line-format" are displayed below the message as key-value pairs. As an example, the bunyan[1] log format[2] has a few standard properties that are used in the "line-format" to form the main message. Then, as shown in this test[3] for this log[4], the remaining properties (even ones not mentioned in the format file) are shown underneath.

> or filter on it in the UI

Since they are part of the message as mentioned above, they can be filtered on.

> and no ability to see it in the SQLite virtual table.

The "log_raw_text" column in the table can be used to access the original log message from the file. So, you can use the JSON functions in SQLite to retrieve the value:

    ;SELECT log_raw_text ->> '$.repository' from bunyan

[1] - https://github.com/trentm/node-bunyan

[2] - https://github.com/tstack/lnav/blob/master/src/formats/bunya...

[3] - https://github.com/tstack/lnav/blob/master/test/expected/tes...

[4] - https://github.com/tstack/lnav/blob/master/test/logfile_buny...

[scottlamb]

Thanks for this. When I switched my project to structured logging, I deleted its lnav log format file and was sad about it. I'll add one back now!

[tstack]

> I also don't see something else I might want: a way to have a different "view" for certain log messages; maybe to switch between filtering/viewing particular ones, maybe to just have line-format be conditional based on the detected format.

Have a look at the following comment on an issue that might be similar to what you're thinking of:

https://github.com/tstack/lnav/issues/1065#issuecomment-1602...

> I guess I can sort of do this based on `module-field`? but I might want it lighter-weight/finer-grained than that.

Unfortunately, the "module-field" does not work for JSON logs at the moment. It's something I should really fix.

Ultimately, lnav has existed for almost two decades now and I use it every day. So, it's always seeing improvements. If you're having a problem with it, file an issue on github. I don't always get around quickly to fixing other folks feature requests / issues, but it tends to happen eventually.

Thanks.

[physicles]

Yeah it's essential to have a viewer that deals natively with structured logs.

I'm iterating on a log pretty printer that accepts structured logs in a pipe and does things like color coding, adding terminal-recognized vscode:// hyperlinks for call stacks, smart wrapping based on the terminal width, and special formatting for panics and stuff.

NCurses is probably coming in a couple months.

Does anything like this already exist?