As other poster mentioned, Dokku and Elastic Beanstalk are also good options--the latter would seem to be a natural migration, given you are already on AWS.
Yep love the ease of Heroku, just not a big fan of the $7/mo per container model since I have a lot of apps to host. Dokku looks like a great option for getting the Heroku experience on cheaper compute though, will check it out!