Jupylet is a library that enables creating and running pyglet games inside Jupyter notebooks both for kids learning to program and for researchers of reinforcement learning. RL researchers and developers can use it to easily create new environments, and to programmatically control and render thousands of frames per second.