AWS Lambda, SQS and Event-driven development
Popular production relational databases (Oracle, PostgreSQL, MSSQL, etc.) are still bound by “single-server” infrastructure, they were designed that way because they were conceived in a different computing era.
There are promising products out there, such as Aurora Severless and YugabyteDB, but these are not purely “serverless” products (yet) where one can simply instantiate a virtual database and tables and not ever think about connection limits, available storage, CPU limits, etc.
In a recent project, we came across this common problem where we had to find creative ways to decouple the serverless stack from an RDS instance. The problem was that the serverless API was directly connected to the database and maxing out connections and taking up all the resources, dropping connections and important transactions ending in some data loss.
After a careful assessment, we found the most suitable way to solve the data loss problem was to decouple the serverless API using AWS SQS. Using several SQS pipelines we were able to throttle the number of transactions that would eventually end up in the same database. We allocated a maximum number of connections for SQS.
For reads, we simply used pgPool load-balancer to allocate for more connections than the maximum allowed in the RDS instance. While not a perfect solution, it definitely worked like a charm. We were able to see generally more health in the production database. Our partners are very happy, we accomplished another success story in no time.
The next steps are to separate reads from writes and migrate the RDS instances into an Aurora cluster that can be scaled up/down more easily.
