Every team I've known that adopted Lambda + DynamoDB (or equivalents) gave up on running their app locally, adding a lot of friction to the development process.
I highly recommend using AWS's own Chalice library as it makes local dev _and_ deployment very easy.
If you need more complex cases like deploying docker containers to a Lambda function, take a look at AWS's SAM library. Also supports local dev _and_ makes deployment easy (its essentially a wrapper around Cloudformation so its very powerful).
This is one of my concerns as well - apparently azure functions allows you to debug your function from within vscode. I see lots of issues with this in that 1. you are limited to vscode as your editor and 2. you can only interact with Azure resources in a "development environment" within Azure itself, i.e. no local copy of the database etc
If you need more complex cases like deploying docker containers to a Lambda function, take a look at AWS's SAM library. Also supports local dev _and_ makes deployment easy (its essentially a wrapper around Cloudformation so its very powerful).