Loved section 1.5.1, "Bytes in color". The author wrote code to help them understand a custom binary format by coloring the bytes based on their value, making patterns in the format visually obvious. The structure of the binary becomes much more clear when visualized this way. That's a handy approach I've mentally bookmarked for the future.
> hexyl is a simple hex viewer for the terminal. It uses a colored output to distinguish different categories of bytes (NULL bytes, printable ASCII characters, ASCII whitespace characters, other ASCII characters and non-ASCII).
I was in a team that, among other things, was working on integrating 3d visualisation of binary files in a hex editor with the plan of making a reverse engineering tool (mainly for security).
Does that have the 3D visualizations shown in Christopher's 2013-2014 talks? Both the github readme and the associated blogpost from July make it look like it only includes 2D visualizations. There are fewer "view" buttons on the right toolbar than the demo had. I should answer my own question by simply trying it out, but I'm not quite sure when I'll have a moment to get around to it.
Some people have taken this concept very far. Batelle's CantorDust (was?) a pretty fully developed 3D visualization software for binary packages utilizing many different views and filters with convenient UX for selecting blocks of code graphically and zooming in on the corresponding hex, or vice-versa.
Edit: I looked very hard for this software about 6-12 months ago and couldn't find anything...it appeared to be vaporware. However, 'LargoLasskhyfv posted in this thread tree at the exact same time I did and noted that it was just released about 3 months ago as a plugin for Ghidra. I'm very excited to play around with it.
Wow. This was a fun read. I would have been lost - but loved for somebody to explain it in a way I was able to grasp with my level of technically understanding.
Thanks to the author - I would love for this data to be in an open format, as I would love to do some kind of personal analytics on that (same with Audible data actually).
On a different tangent, I really love the ease in the Kindle/Audible ecosystem to switch between the same content in different media. Reading on the kindle app, switching to Audible in the car and back to kindle in the evening in bed. That really makes reading/book consumption great.
I'm very interested in this. I read a lot of books that aren't from Amazon on my Kindle. I do a lot of highlighting, which I like to export to readwise.
In order to do this I have to plug in my Kindle. Mount it. Find the clippings file. Upload the file to readwise.
The best I came up with was a script that detects when I plug-in my Kindle and opens the finder window and opens the readwise page. Pretty lame.
Unfoturnately, Amazon does not sync highlights on books that have not been purchased on Amazon. This is also why Readwise cannot automatically sync those highlights.
would this work if Amazon was using cert pinning? cert pinning basically seems like a way to make apps able to resist user's having any freedom or power. the author here installs a custom CA to man-in-the-middle. now more & more apps use cert pinning, which would block that, I believe, & keep the user from being able to reverse engineer & probe.
imo the forces in favor of securing things have done the users great great damage. who are we securing things for? imo the security world needs to re-orient, & begin to harken to RFC 8890 The Internet is For End Users[1], allowing users some means to see how their systems are behaving. "secure" standards like cert-pinning favor giant companies & totalistic visions of security, taking from computing the person & personal. cert-pinning is a good layer of defense, but it takes away user affordances that must be added back.
Or just open up the apk in e.g. apk studio and replace the bundled cert with the one from your mitm proxy. Effectively, that'll repin the app to your proxy specifically. There are more exotic pinning methods that this won't solve (the forever cat-and-mouse game) but it's good for many apps.
It's a problem that, alas, must be tackled somewhat uniquely on every platform, where-as previously users were free to manage their own Certificate Authorities as they wished. Now users are safe & secure from themselves. :/ Admittedly there were a lot of problems with poorly managed CA stores, users being abused, but cert-pinning feels like such a drastic overreach in preventing any form of user control.
Techniques like this "rebuild your apk" are interesting & good to have, but every OS needs it's own bag of tricks. It's probably not a total show-stopper, but news like today's that Windows 10 will only install signed driver software (an admittedly niche-ish case), the closed Apple store,... there's a lot of places side-loading is not an option. Do those apps get a pass, get to be complete black-box software that we the users have zero ability to look at or understand?
You're making this sound so much harder than it is, though. None of the techniques for bypassing the security enhancements is difficult, nor are they undocumented. Who are these hobbyists who want to reverse engineer the whispersync protocol but can't follow a simple tutorial to swap out a CA?
you've listed one platform (of many) where it's possible to (in some cases where there is a .cert or .pem file sitting in an apk), updated that apk pkg, & then use side-loading (not available on os'es) to install that hacked package.
there's a number of caveats to your steps, it will definitely not be this easy in all cases. and i while this wasn't all that difficult, it's still a situation where the OS is actively working to prevent the user from being able to understand their system. and most platforms don't provide even this much of an affordance, of opening, rebuilding, & side-loading packages.
Amazing tool, neat trick. It requires xposed framework though, which is an interception framework for Android that requires root access to install.
Less and less devices seem to have root, and almost no devices these days seem to allow unlocking the bootloader, which is even better.
It feels like users are getting kicked further & further & further out. We are less & less able to have any chance to understand what computing is, & this deeply deeply hurts us, I tend to believe.
Hi, thanks I did not know Readwise. I signed up for an account to check how it works. It uses a browser extension to sync the highlights instead of using the API. It can sync automatically but your browser must stay open (which is OK IMHO). Readwise can sync much more sources also.
Thanks for introducing me to Readwise, I might use it :)
That said, this suggestion and it’s method of avoiding direct interaction, but instead using rendered content in browser is another reminder of how powerful browser extensions are.
They can also make additional requests using the user’s ip address.
Would pulling data this way also seems less likely to be a TOS issue?
Reason being that you are reading content you are manipulating or displaying in a browser “after” amazon does their own highlighting.