Hacker News new | ask | show | jobs
Higher Order Derivatives of Transforms (nosferalatu.com)
87 points by nosferalatu123 879 days ago
5 comments

Don't think I'll ever get used to stars in LaTeX when someone means simple multiplication.

The preceding blog post[1] seems to contain the more interesting parts though. This is just (d/dt)^n e^At x = A^n e^At x, which is kind of obvious from the definition of e^At.

[1]: https://nosferalatu.com/DerivativesLogarithmsTransforms.html

The article (even more the previous article) really misses a reference to matrix exponentials [1]. Everything about X(t) = X^t becomes a bit clearer. Also I think using the term transform is a bit loose here. IIRC Matrix exponentials only work with linear transforms that are represented by a square matrix.

[1] https://en.m.wikipedia.org/wiki/Matrix_exponential

The previous post in the series[1], helpfully linked to in the introduction, has a lot more details of the underlying concepts.

It also set my mind wandering to the not-technically-related functional derivatives[2], where you vary the function slightly rather than the argument value.

I'm not great at math, but I do love this what-if exploration you can do in math. Due to the various proofs underlying it all it seems sometimes more fruitful than similar exploration in programming, where one might quickly stumble upon obscure compiler errors or similar obstacles.

[1]: https://nosferalatu.com/DerivativesLogarithmsTransforms.html

[2]: https://en.wikipedia.org/wiki/Functional_derivative

If you're looking for interesting derivative-adjacent ideas, I would also recommend Clarke derivatives [1]. They occasionally show up in ML papers, e.g., [2], [3]. Unrelated bu tangential, another place where you need derivatives but don't have access to them (standard or otherwise) is in the area of black-box optimization. Within this area, Bayesian Optimization (BayesOpt) has picked up quite a bit, which I've successfully used quite a bit in my work - I've an introduction here [4]. There is also a good book available online for free on the topic [5].

[1] https://en.wikipedia.org/wiki/Clarke_generalized_derivative

[2] https://proceedings.neurips.cc/paper/2021/file/70afbf2259b44...

[3] http://proceedings.mlr.press/v202/lee23p/lee23p.pdf

[4] https://blog.quipu-strands.com/bayesopt_1_key_ideas_GPs

[5] https://bayesoptbook.com/

Functional derivative : Frechet derivative :: gradient : directional derivative
What is meant by a Transform in this context? The author doesn't seem to give a definition.
I meant a "transform" as a linear map. I'm using the word as it is used in computer graphics (my background), so it's something that translates, rotates, scales, etc. other things (such as points). That is often a 3x3 or 4x4 matrix, although it can also be a vec3 translation and a quaternion, or just a quaternion. I think "Transform" is clear in the context of computer graphics, but I see what you mean about it being vaguely defined in my blog post.
In robotics, it's pretty widely used to refer to a 6-degree of freedom pose or relative pose in space, for example it's widely used in ROS, a de facto standard https://docs.ros.org/en/melodic/api/geometry_msgs/html/msg/T.... Not sure if there they're just using it as an example in the graphics.
They give a brief statement in their previous post (see link at start of post), essentially it's a linear transformation also known as linear map[1].

[1]: https://en.wikipedia.org/wiki/Linear_map

I read their previous post, and nowhere do they explicitly say "a transform is a...". One might assume that it is indeed a linear transformation, as you suggest, but it shouldn't be up to the reader to do detective work just to understand the objects the author is talking about.
What I meant was that based on what they said in their previous post ("[g]iven a transform T and a point x, we can find the transformed point with T∗x") and the interactive graphics, I felt certain they meant a linear map.

I agree it's sloppy, at least a reference or something should be given if one doesn't want to spend time on the full definition.

Maybe the author could have used a more general notion, then, if omission and brevity were going to be present? Like, instead of a linear map or transform, he could have said an operator or something.

I don't know what is the general form of a transform or linear map. I think it's something like operator, though.

True, however I don't know how well versed the author is. Back in my late teens when I was deep into 3D graphics and ray tracing, I knew a lot about that specific math but not much beyond it. To me, "transform" was crystal clear to mean some kind of linear transform, and I hadn't yet learned of the more general operator notion[1].

So I can see myself writing something similar thinking it was clear.

[1]: https://en.wikipedia.org/wiki/Operator_(mathematics)

operators are usually defined as linear maps from spaces to themselves

    V -> V
so they are less general than a linear transformation
Terence Tao has a book “Higher order Fourier analysis”, can anyone chip in on whether the two are related?

Here is a video, it seems to focus on aaplications in number theory, but I cant tell as a layman https://www.youtube.com/watch?v=nr2Xv9v9CZc

For example, this is the tradFourier version of what hes trying to do

https://en.wikipedia.org/wiki/Hardy–Ramanujan–Littlewood_cir...

The techniques in this article would be considered common knowledge for any of the prerequisite classes you'd need to take to be able to follow a course on Higher order Fourier analysis.

I wouldn't say they're unrelated, but if you want to know whether both uses of the phrase "higher order" have any relation then no.

Thank you!

I remember seeing in the Tao book a Fourier operator that looks something like the “integral” of exp(ik^2 x).. (or exp(ik x^2), haha)

As a followup, is it possible to ELI5 roughly what Tao meant by “higher-order”?

The first paragraph of the preface on page 7 in the book gives an explicit example of “higher order”: https://terrytao.files.wordpress.com/2011/03/higher-book.pdf
Lollol thats what i half remembered.. i guess then what Terence Tao meant by higher order is lower order in the featured artiicle!
I like this summary of Lie theory in the context of robotics: https://arxiv.org/pdf/1812.01537.pdf
I like this tutorial because it doesn’t get too bogged down in abstractions and has numerous examples. When I’ve tried to learn differential geometry in the past, standard texts get very abstract very quickly and it’s hard for me to envision what the generalization is doing for me.