It can help to understand why some people think inaction is an action by abstracting the problem such that you can model it in a computer using standard decision problem algorithms. When you do that you have an initial state and you have two edges and they lead to terminal outcome nodes. The choice of which edge to take is an action. This isn't (usually) controversial. People generally call the do nothing action an action in these circumstances: game theory extensive form will call it that, as will Q learning, and often you'll see the same framing even in things like the expecti minimax algorithm though sometimes they'll prefer to call it a move.
In my estimation, inaction only seems to not be an action, because people are afraid of the idea that they are causally connected with outcomes in which things are killed - they want to avoid tit for tat game theoretic follow on concerns that aren't part of the game definition but might be implied by it. I think for myself, I discount this concern, because I assume policies of mercy are preferred to policies of judgement - the world is so complex and hard and some err isn't just reasonable, but inevitable. Not allowing for mercy in response to harm from decisions seems more likely to trap in tit for tat. So intent ends up mattering to me much more than the outcome when I consider the follow-on games that are implied.
I can either pull the lever or I can not pull the lever. Choosing to not pulling the lever when I know I can save others, would be me making the choice to not saving others.
In my estimation, inaction only seems to not be an action, because people are afraid of the idea that they are causally connected with outcomes in which things are killed - they want to avoid tit for tat game theoretic follow on concerns that aren't part of the game definition but might be implied by it. I think for myself, I discount this concern, because I assume policies of mercy are preferred to policies of judgement - the world is so complex and hard and some err isn't just reasonable, but inevitable. Not allowing for mercy in response to harm from decisions seems more likely to trap in tit for tat. So intent ends up mattering to me much more than the outcome when I consider the follow-on games that are implied.