That's a good point, it is possibly best to not mention anything. I don't hope for anything in particular but find it "easier" for myself to just be honest with people when they ask what I'm doing.
If people ask and you feel like answering, tell them.
Otherwise it's not dishonest. There's nothing unethical about earning money while also delivering on the promises made as a contractor for someone else.
In case that's not obvious enough, consider this scenario:
You're an actual employee at the day job, not a contractor. On the weekends you like hitting up thrift shops looking for antiques which you then sell on eBay.
Is it dishonest not to talk about that to your employer?
Not talking about something doesn't mean automatically mean it's dishonest. Just like I'm not talking about what I just ate for dinner, not talking about additional ways you make money doesn't mean you're being unethical, immoral, unkind, or dishonest.