From the wage slaving, I created a web app with rest api & data pipelines and I am working on getting it deployed to azure.
I am using azure function for the data pipelines because they only need to execute every 15 minutes. And because the rest API is not very big I just put the rest API on the azure functions.
Using caching is perfect here cause when the pipelines are done processing, the data can be cached so the rest API can deliver them super fast.
In the ideal case I can use a cache like redis so if the functions scale to multiple instances they pull from the same cache.
Issue is the redis cache in azure is expensive. The budget is not big enough for this. So the best thing I can do is an in memory cache.
Downside of this is the data can be cached on one instance of the azure functions, but if another instance is created it has its OWN cache so the first load will be slow.
It’s annoying cause using something like redis is perfect to improve the performance in this use case.
I am using azure function for the data pipelines because they only need to execute every 15 minutes. And because the rest API is not very big I just put the rest API on the azure functions.
Using caching is perfect here cause when the pipelines are done processing, the data can be cached so the rest API can deliver them super fast.
In the ideal case I can use a cache like redis so if the functions scale to multiple instances they pull from the same cache.
Issue is the redis cache in azure is expensive. The budget is not big enough for this. So the best thing I can do is an in memory cache.
Downside of this is the data can be cached on one instance of the azure functions, but if another instance is created it has its OWN cache so the first load will be slow.
It’s annoying cause using something like redis is perfect to improve the performance in this use case.
