OpenCV has wheels built for all common platforms so it should be fine now. If your platform or python version is too new/old/unsupported then yeah, you can forget about building it and use another platform/python version.
To be honest, it's always this dice roll with most things on pypi. If there are wheels for you, golden. If not, you better hope it's a pure python package or it doesn't pull any outside libs. Things are getting better though with solutions like cibuildwheel.