Communication between open and closed source processes runs via IPC. Since all the IPC functions are declared in the open source part, you can exactly check what data is exchanged between the processes -> You can see that keys are not transmitted.
You can make it so that it will only store the key in RAM, but then you'd have to enter the key every time it launches, making automatic backups impossible.
Although, I agree, if you only allow the open source network access, AND can ensure that the key won't somehow be smuggled in the data the closed source process sends, you're probably fine.