Jul 2023 – Oct 2023 · 3 mos · On-site

• • Changed the data format of the data stored in kafka from json to avro that helped in direct conversion of raw data to parquet format that is stored in S3 using S3 sink connector which helped in real time availability of processed data and also helped in reducing S3 cost by 1700 USD per month.

Jan 2023 – Jun 2023 · 5 mos · On-site

• Redesigned data processing pipeline to form optimal size parquet files while storing raw data in S3 that helped in reducing S3 read/write cost by 120 USD a month and speeded up the spark processing time.
• Designed alerting system for AWS cost and built several scripts in order to optimize resource utilization and hence reduce the cost.

Jun 2022 – Sep 2022 · 3 mos · Dublin, County Dublin, Ireland

• Worked on developing an API for performing drain/undrain operations on the device using an on-box agent.
• Integrated the API with the drainer controller.
• Added end to end test cases for verifying and monitoring the operations on the box.

Jun 2021 – Nov 2021 · 5 mos · India

• Created and implemented scalable testing methods for different functions and recorded the test results to test various functions.
• Worked on handling the expressions for the language and added support for various operators for the language.