Apr 2024 – Jun 2025 · 1 yr 2 mos · Hybrid

Feb 2022 – Mar 2024 · 2 yrs 1 mo · Hybrid

• Developed API in nodejs for Data Console Workflows Tool along with unit test cases. This tool greatly simplifies self serve access to data by combining individual building blocks as Workflow.
• Carried out design and implementation with test cases for adding functionality for consuming from SSL/TLS enabled Apache Kafka clusters in ELT pipeline.
• Implemented features in Bigquery Slot scheduler developed in python for managing creation of reservations, assignments of slots to different projects ensuring proper logging and monitoring for tool. Deployed tool in Google App Engine and used cloud scheduler, cloud tasks, cloud Pub Sub, Cloud Logging, Cloud Functions for different aspects of tool.
• Infrastructure Planning and end-end data infrastructure automation in GCP, involving networks, VPC peering, GKE, and Bigquery, keeping all essentials like scalability, security, reliability, cost under consideration.
• Deployed and automated management of Apache Flink, Apache Airflow, Apache Mirror Maker, Apache Spark, TimescaleDB clusters in kubernetes GKE.
• Improved security posture of kubernetes cluster by implementing workload identity, RBAC and private access.
• Improved observability of services by centralizing logs, metrics of all services running in kubernetes and VM's using tools like Telegraf, Cortex, Promtail, Grafana Loki, Grafana. Created Grafana Dashboards for frequently used logging queries to simplify usage of logging platform, thus effectively onboarding several users to platform.
• Self Serve alerting through GitOps approach to setting alert rules and thresholds.
• Developed Helm charts and Ansible Roles for tools deployed in kubernetes and configuring compute instances.
• Implemented Bigquery usage dashboard for identifying usage in near real time for analysts. This helped to have regular usage estimations and purchasing of committed slots to save 30%+ costs.
• Adaptable and proficient in learning new concepts quickly and efficiently.

Apr 2021 – Feb 2022 · 10 mos

• Contributed to OKR planning and worked along with the team to drive initiatives.
• Improved process for tracking issues using jira by building custom dashboards and boards.

Apr 2020 – Mar 2021 · 11 mos

• Built data pipelines for SES events to lake using Amazon Kinesis and s3, analysis of AWS cost billing reports and analyzing historical old logs for compliance using hive.
• Implemented Ververica Platform to run flink workloads in kubernetes using helm charts.
• Worked on Apache Druid for supporting warehouse capabilities.
• Have worked on hosting internal docker registry using Harbor open source project comprising of different proxy projects and internal projects, hosting 100's of repos and 350+ engineers onboarded and using actively, millions of image pulls every month.
• Worked with team in implementation of Big Data Architecture comprising of Hadoop, Hive, Spark, Trino, Airflow, Jupyterhub running in kubernetes using docker containers as highly scalable, available and cloud agnostic, elastic platforms. This being used by all Analysts and for serving external clients.
• Participated in all phases of system development life cycle from requirements analysis through system implementation with SaaS platform Qubole and Databricks for running Data Engineering workload involving spark, sql Analytics, etc in AWS Cloud.
• Built custom prometheus target discovery tool for scraping metrics off different EC2 service endpoints using boto library in python, ec2 discovery based on tags assigned to instances.

Aug 2018 – Mar 2020 · 1 yr 7 mos

• Automated end-end infrastructure setup in bare AWS account using terraform, packer, ansible, kubernetes, etc
• Worked on highly virtualized infrastructure comprising of Kubernetes to orchestrate several microservices running in docker containers.
• Experience of running kubernetes in bare EC2 instances and in AWS EKS.
• AWS Cloud comprising of AWS networking, EC2 virtual machines, RDS, S3, DynamoDB, IAM, Kinesis, SES, SNS etc
• Worked on Apache Ambari for running Hadoop, YARN, zookeeper and kafka configured running all components in HA mode.
• Worked on Apache Kafka to run in kubernetes as important component in Data Engineering Pipelines ingesting billions of events per day.

Jun 2016 – Aug 2018 · 2 yrs 2 mos · Mumbai Area, India

• Managed OVH Cloud running bare metal servers.
• Used LXC containers for running workload in virtualized environment.
• Contributed to building several modules from scratch using ansible for configuring web servers, Elasticsearch, email server, etc.
• Automated DNS management for hundreds of domains.
• Identified issues, analyzed information and provided solutions to problems.
• Carried out day-day-day duties accurately and efficiently.
• Monitored automated build and continuous software integration process to drive build/release failure resolution.

May 2015 – Jul 2015 · 2 mos · Hyderabad Area, India

• Worked in the performance team of Qualcomm, Hyderabad.
• Responsibities
• 1. Implement a system to analyse and compare various chipsets.
• Technologies Used
• 1. C,C++ for the development of the system.
• 2. ADB for linking mobile with the computer.
• 3. Worked on unix tools like top, htop, sed, awk and /proc filesystem.