One day pretty early in the morning I got a message with an screenshot from the CTO saying:

Any thoughts on how we keep these cleaned up?

Feature Branchs deployments en ECS

A few months earlier we were discussing the need of deploying feature branches environments as a solution to avoid our delivery process getting stuck as we were deploying more than the QA team could handle. We created a feature branches deployments pipeline and it was working great as every important feature was tested isolated from the QA envrionment (No merges until approved by QA), essencialy we stopped having complains about our queue to production getting stuck by defective code or incompleted acceptance criteria.

This is devs responsibility, we could set a reminder in the channel, I said in response. But soon I understand that devs were not aware of how the feature environments work or the costs associated with it. Even though the pipelines for feature environments had one additional step for teardown, devs were not executing it after QA validation.

After a few minutes I came back with a proposal to execute a clean up every day and during our daily meeting I let everyone know that daily at 8PM CST all the feature environments will be flushed automatically. This process guarantee that only the required feature envs are running while maintaining the costs at minimum, supposing an environment is deployed at 9:00AM CST and it run 11 hours until 8PM CST, this set up cost less that $0.5.

Having the feature environments in the way I set them before the clean up was fairly cheap (1$ daily) to maintain but as we were not cleaning the environments properly this was increasing our monthly billing by 300$ every 10 environments. With the clean up set up we would need to deploy over 600 feature branches monthly (We were not even close deploy a tenth of it ) in order to reach the 300$.

So, by implementing this automated daily cleanup at 8 PM CST, we’ve not only ensured that our feature branch environments remain lean and cost-effective, costing less than $0.5 per typical 11-hour deployment, but we’ve also proactively addressed a potential $300 monthly overspend for every ten neglected environments. This approach strikes a balance between providing developers with the isolated testing environments they need and maintaining responsible resource management, allowing us significant headroom for future feature development without unnecessary expense.