May 2022 – Present · 4 yrs 1 mo · India

• Developed a logging solution for a microservices-based application deployed on Kubernetes, utilizing parsing techniques to identify the module in which exceptions were occurring.
• Employed Latent Dirichlet Allocation (LDA) to cluster the types of errors, enabling more effective monitoring and troubleshooting of the system.
• Utilized time series modeling techniques, including AutoARIMA and Prophet, to detect anomalies in the number of errors occurring. If the number of errors exceeds a predetermined threshold, an HTML page is generated and sent using Amazon SES.
• Designed and implemented an event-driven architecture using AWS Lambda and S3, enabling real-time processing of log files and triggering the program whenever a new event occurs.
• Configured Amazon Kinesis Firehose to stream the processed data into Amazon Redshift, enabling real-time analytics and monitoring of the system.
• Containerized the application using Docker, and deployed it on Amazon Elastic Container Registry (ECR) for image storage, and then utilized Amazon Elastic Kubernetes Service (EKS) to run the microservices, ensuring scalability, high availability and fault-tolerance of the system.
• Leveraged Airflow to schedule and manage batch jobs, ensuring reliable execution and efficient use of computing resources.
• Created a CI/CD pipeline using GitHub Actions to automate the deployment and testing of the microservices, ensuring continuous integration and delivery of new features and enhancements to the system.