You can use SDFs (signed distance functions) to express your objects in a mathematical form. It has its pros and cons. One of the bigger cons is that most tools use the polygonal approach, so you'd have to build a lot from scratch.
Lots of SDF examples in ShaderToy[2], and many introductory videos on YouTube (I specially recommend Inigo Quilez's channel[2] --co-creator of [1]--, and The Art of Code [3])