The airtag case doesn't have a gap large enough to allow a regular wire to pass through. I guess you'd need to drill the cover, or just discard the cover and use tape. Then you could use one of these dummy batteries:
I have a dozen of those style temperature gauges all over my house. Mine are actually a slightly smaller, square version. You can use Home Assistant to read the Bluetooth temperature and humidity readings from them. They ended up costing maybe $3 each when buying several at a time. Battery tends to last a bit over a year.
https://www.aliexpress.us/w/wholesale-cr2032-dummy.html