Aug 2020 – Mar 2022 · 1 yr 7 mos

• Performing Data Analysis and Data preprocessing with python, spark, and SQL
• Developing Machine Learning and Time-Series forecasting algorithms with python on AWS Sagemaker
• Building ML and ETL pipelines involving AWS redshift and S3
• Developing BI web dashboards for historical analysis and future forecasts using python (plotly and dash libraries)

May 2019 – Jul 2020 · 1 yr 2 mos

• Performing ETL using Spark Scala and Hive including transformations and data validations. Building and optimizing data pipelines with the help of AWS services like DMS, EC2, S3, lambda, e.t.c.
• Writing automation scripts using Python to avoid manual fixes during an active pipeline.
• Loading incremental and full load data into hive tables and then copying to redshift tables which are used for reporting through Microstrategy.
• Improving data quality and fixing bugs reported by the users.
• Migrated long-running hive jobs to spark. ( in scala)
• Worked on Real-time monitoring of pipelines. (mail notifications, logging tables' sizes. e.t.c)
• Developed file transfer scripts (servers to S3 & vice-versa) that eliminate 3rd party tools.
• Worked on a pipeline using AWS DMS with source as MySQL, destination as S3/Hive tables.

May 2018 – Jul 2018 · 2 mos · Chennai, Tamil Nadu, India

• Converted the latest version of vanilla Linux kernel to minimalistic object oriented Linux (MOOL) kernel for both desktop and android. Debugged the errors occurred during kernel compilation and booting. Successfully loaded a C++ module into the MOOL kernel.