|
|
|
|
|
|
by detroitcoder
1466 days ago
|
|
|
The jump of going from model -> webserver by placing the webserver in the same process as the model is enticing because you can get it to work in under an hour by adding flask/django/fastapi to the env and decorating a function. The problem is that that your model and webserver do NOT scale in the same way, and if you don't realize this fast, you are going to be trying to fit a square peg through a round hole once you have adoption trying to make it work. All models at scale eventually need to be executed by an async queue processor which is fundamentally different from a request response REST API. For simplicity managing this outside of the process making the web request will help you debug issues when people start asking why they are getting 502 responses. If you are forced to use python for this, I would always suggest of going to celery/huey/dramatiq as an immediate next step after the REST API MVP. I hear Celery is getting better but I have ran into issues over the year so it pains me to recommend it. |
|
|
Prime example of the difference is whether you accept disruption of inference when you deploy a new version of your webapp.
Very high chances you don't, thus you will start implementing a queueing mechanism without realizing it.