They could run a rootkit/keylogger/download contraband to your computer in the same time. A lot more to worry about than just learning your wifi password.
Where exactly do you think the insecurity is coming from here? Your guests could do anything at all with a few seconds on your computer. Learning the network password is the least of your worries.