Strictly speaking, Olber's Paradox is about an infinite, eternal/static, and homogeneous universe with an infinite number of stars; in such a case, the light the dust absorbs would cause the dust itself to start emitting light, which would make it visible [0]. I'm not sure to what extent this paradox applies outside that scenario (e.g., an infinite-but-changing universe)
[0]: https://www.scientificamerican.com/article/why-is-the-night-...