Yes, it will show up on your bill as a cost drastically smaller than purchasing the equivalent amount of gas. If you put solar panels on your house you can get that cost even lower.
I'm sure it shows up on my bill, but there are so many other variable costs that I can't find it. The weather (how much I use the heat pump) is a much larger factor