Automating Multi-Region Scalable CI/CD Framework for Managing AWS CloudWatch Alerts
DOI:
https://doi.org/10.15662/c9d1wv10Keywords:
JSON, CI/CD Automation, Multi-Region Deployment, AWS Cloud formationAbstract
With the increase in applications, managing a large number of monitoring alerts in different regions in AWS is becoming more complicated. This work introduces us to an automated multi-region, scalable CI/CD system to manage Amazon CloudWatch alerts effectively. First, alerts were developed manually, resulting in more than 100 alerts in two regions, and the number steadily increased with the addition of new features. This paper based system was ineffective, prone to error and hard to keep up.We have come up with an automated solution to these challenges that incorporates CloudWatch alert management into a CI/CD pipeline. This is done by first retrieving the existing CloudWatch alarms in JSON format with a custom script. These are transformed into YAML format with a converter tool. A skeleton CloudFormation template is meant to determine the key parameters necessary to define CloudWatch alarms. The translated YAML files are streamlined by eliminating the redundant fields and conforming them to the CloudFormation template format.Lastly, the AWS Boto3 library that uses Signature Version 4 (SigV4) authentication is used to complete the automated deployment, which allows safe and programmable access to AWS services. The CI/CD pipeline, which runs on Screwdriver, is set to keep on deploying and managing CloudWatch alarms in multiple regions. This method provides consistency, scalability, and easy maintenance and eliminates much manual work and configuration errors. The proposed framework increases observability, operational efficiency, and reliable application performance within dynamic cloud environments.
References
[1] E. Soares, G. Sizilio, J. Santos, D. Alencar, and U. Kulesza,“The Effects of Continuous Integration on Software Development: A Systematic Literature Review,”arXiv preprint arXiv:2103.05451, pp. 1–20, Mar. 2021.
[2] B. Burns, B. Grant, D. Oppenheimer, E. Brewer, and J. Wilkes,“Borg, Omega, and Kubernetes,”Communications of the ACM, vol. 59, no. 5, pp. 50–57, 2020.
[3] R. N. Calheiros, R. Ranjan, and R. Buyya,“Virtual Machine Provisioning Based on Analytical Performance and QoS in Cloud Computing Environments,”Future Generation Computer Systems, vol. 110, pp. 1–12, 2020.
[4] R. Ghimire,“Deploying Software in the Cloud with CI/CD Pipelines,”International Journal of Computer Applications, pp. 1–8, 2020.
[5] E. Collins, J. Smith, and R. Kumar,“Challenges and Best Practices in AI-Driven CI/CD Automation,”Journal of Software: Evolution and Process, vol. 32, no. 4, pp. 1–15, 2020.
[6] J. Black and M. Roberts,“Managing Data for AI-Driven Automation in DevOps,” IEEE Cloud Computing, vol. 7, no. 2, pp. 56–65, Mar.–Apr. 2020.
[7] H. Klein and B. McKeen,“Interpretability in Machine Learning Models for CI/CD,” Journal of Computing and Information Technology, vol. 28, no. 1, pp. 19–35, Mar. 2020.
[8] J. P. Black, M. A. Roberts, and L. Green,“Automated Testing Frameworks Enhanced by AI,”IEEE Software, vol. 37, no. 1, pp. 42–50, Jan.–Feb. 2020.
[9] D. Robinson and A. Adams,“Future Directions in AI for Cloud-Native CI/CD Pipelines,”IEEE Future Directions in Computing, vol. 7, no. 1, pp. 99–110, Jan. 2021.
[10] P. Sivathapandi, D. Paul, and S. Ramasundaram,“Enhancing Cloud-Native CI/CD Pipelines with AI-Driven Automation and Predictive Analytics,”Australian Journal of Machine Learning Research & Applications,vol. 1, no. 1, pp. 46–59, 2021.
[11] E. Soares, G. Sizilio, J. Santos, D. Alencar, and U. Kulesza,“The Effects of Continuous Integration on Software Development: A Systematic Literature Review,”arXiv preprint arXiv:2103.05451, pp. 1–20, Mar. 2021
[12] J. Bhuvana,“Implementing Continuous Integration and Deployment Pipelines in Agile Software Development,”International Research Journal of Modern Engineering and Technology, pp. 2086–2091, 2021.
[13] J. Hellings and M. Sadoghi,“The Fault-Tolerant Cluster-Sending Problem,”arXiv preprint arXiv:1908.01455, pp. 1–18, 2019.
[14] S. Liu,“A Survey on Fault-Tolerance in Distributed Optimization and Machine Learning,” arXiv preprint arXiv:2106.08545, pp. 1–20, 2021.
[15] L. Chen,“Continuous Delivery: Huge Benefits, but Challenges Too,”IEEE Software, vol. 32, no. 2, pp. 50–54, 2020.
[16] D. Taibi, V. Lenarduzzi, and C. Pahl,“Continuous Architecting in Microservices and DevOps: A Systematic Mapping Study,”Journal of Systems and Software, vol. 182, pp. 111063, 2021.
[17] M. Shahin, M. A. Babar, and L. Zhu,“Continuous Integration, Delivery and Deployment: A Systematic Review on Approaches, Tools, Challenges and Practices,”IEEE Access, vol. 8, pp. 11390–11425, 2020.
[18] A. Rahman,“Monitoring Microservices: Challenges and Solutions in Cloud-Native Systems,”IEEE Cloud Computing, vol. 7, no. 5, pp. 70–77, 2020.
[19] P. Leitner and J. Cito,“Patterns in the Chaos—A Study of Performance Variation and Predictability in Public IaaS Clouds,”ACM Transactions on Internet Technology, vol. 20, no. 3, pp. 1–23, 2020.
[20] S. Newman,“Building Microservices: Designing Fine-Grained Systems,”O’Reilly Media, 2020.
[21] G. Kim, J. Humble, P. Debois, and J. Willis,“The DevOps Handbook: How to Create World-Class Agility, Reliability, and Security in Technology Organizations,”IT Revolution Press, 2021.
