you can make a tiny battery with ancient-style materials. It’s essentially a vinegar (or lemon juice) galvanic cell using copper and iron inside a porous clay jar. People often call this the “Baghdad battery"
the problem with these hypotheses is the lack of wiring. a very simple battery is not difficult to make (but pay no attention to the energy density), but making a useful circuit that does anything is pretty hard. the simplest possible useful circuit is a lightbulb which requires ability to create tungston wire, a vacuum, and very thin, precise glass.