When we moved into our new place, we put our fence in a few feet from the actual back of our house, and planted a strip of native, pollinator-friendly plants in a little micromeadow.
I did a similar thing, planted rose bushes near the side of the house and fence. Then I was really happy with the bumble bees until I found-out they were wood boring bees and they had made their nests in the wood under my roof and fence posts. I had to replace the fence and the wood under the roof. I also uprooted the rose bushes :( Just chalk it up to bad luck, you should be fine.
My wife planted a pollinator garden last year and I has a ton of fun photographing bees and butterflies. I had no idea it would attract so many insects. (Sadly, the hummingbirds were too skittish to come close when I was outside.)
This was surprising to us too - like, we knew there would be bugs, but there were so many. We had a particularly successful Amaranth that was basically a free standing ecosystem.