Jul 2023 – Nov 2024 · 1 yr 4 mos · Indiana, United States · Hybrid

• Worked at Indiana farm bureau insurance through beacon hill between Jul 2023 and Nov 2024

May 2023 – Aug 2023 · 3 mos · Hybrid

Aug 2022 – Dec 2023 · 1 yr 4 mos · Hybrid

• Designed and built an end-to-end FRST Challenge judging platform using AWS and Python, evaluating data and generating reports from 25+ IoT prototypes simultaneously in near real time for $6M in awards
• Implemented serverless, near real-time data pipelines using AWS IoT Core, Lambda, Glue (PySpark), and S3, enabling scalable IoT data ingestion with monthly AWS cost under $100
• Utilize Python libraries such as Pandas, NumPy, and Matplotlib for thorough data analysis, data processing, and visualization.
• Leveraged AWS services including IoT, AWS Lambda, and Glue (PySpark) to ensure data quality, documentation, metadata management and transformation.
• Optimized data storage and visualization by harnessing the capabilities of Amazon S3 for efficient storage, PowerBI for insightful data visualization, and CloudWatch for robust monitoring and security, ensuring compliance with data governance standards.
• Got featured in an Indiana DHS Government article for contributions. https://www.in.gov/dhs/files/hoosier-responder-july-2024.pdf

Jan 2023 – May 2023 · 4 mos · Indianapolis, Indiana, United States · Hybrid

• Leading lab sessions on the Database Design course for undergrad students. Covering topics like ERD, Normalization process, SQL, and many more.
• Increased student participation to 100% and helped them improve their grades to more than 90%.

Jul 2020 – Jul 2022 · 2 yrs · Bengaluru, Karnataka, India · Hybrid

• Spearheaded end-to-end migration of 50+ legacy data pipelines and modernized data models on AWS, orchestrating 5PB+ of data with Step Functions, Redshift stored procedures, and Airflow, achieving 80% reduction in operating costs and 99.9% reliability while ensuring regulatory compliance.
• Led a campus-to-corporate program, trained 30 + new hires on AWS, Python and SQL.
• Led project planning, estimation, and end-to-end migrations of master data sources. Modernized the data model in AWS making it easier to extract the data and orchestrated jobs using Airflow.
• Led the automation of creating data quality jobs and batches using API Gateways and AWS Lambda. Saved 200 hours of manual effort.
• Built and optimized reports using Tableau to compare the performance of competitor products which helped accurate marketing and increased sales. Onboarded prescription and claims data from IQVIA and Symphony, used formulary and lives data from MMIT for payer reporting.
• Worked on data migration and Business Intelligence in the Pharmaceutical domain. Designed, built, and deployed fault-tolerant and cost-effective big data pipelines using Continuous integration and continuous deployment strategy on AWS cloud with a test-driven setup.
• Contributed to a 300k annual DBA cost reduction by developing the "Database Bootstrap Process" to automate database object creation.
• Developed a serverless process for onboarding data in S3 Data Lake using the recursive AWS lambda function. Saved 90% of the data extraction time.
• Received 4 awards from the stakeholders and supervisor.
• Received the RISE award and Lilly points for Payer reporting and for leading campus to corporate program respectively.

Jan 2020 – Jun 2020 · 5 mos · On-site

• Worked on a Business Intelligence project in the manufacturing domain and helped the customer migrate data assets in AWS. Aided employer to increase revenue by getting more business on the AWS tech stack. Migrated ETL pipelines and SQL databases using Informatica PowerCentre, Python, and AWS service
• Transferred data from various data stores in S3 using Informatica. Did content replication and converted the files in S3 to parquet format, joined the dimension and fact tables using PySpark and python in AWS Glue, Merged the data of different sources into one table using Union & Merge in Glue Jobs and populated it into Redshift. Used SNS, SQS, Lambda, and Glue Scheduler for automation.
• Used SQL for Data Analysis and to get insights into the data, used Informatica Power Center, AWS Glue & Lambda for data migration, S3&Redshift for data storage, and used Tableau and Athena for Data Visualization.
• Used AWS SDK Boto3, PySpark, Spark SQL, RDD, and DataFrames in AWS Glue & Lambda Functions for automation and fetching the files from different AWS services.
• Received several rewards for excellent work on technologies like AWS Glue for handling Big Data and delivering it within strict time constraints. Saved 70% on the operating costs after migration.

Jan 2018 – Jan 2020 · 2 yrs · On-site

• Worked on Data Analysis, integration & migration with the help of SQL & Informatica in a BI project.
• Expertise in SQL and RDBMS like Oracle, MS SQL Server, MySQL, SAP HANA, Redshift, Snowflake, and PostgreSQL. Worked Informatica PowerCenter and performed transformations like Vlookup, Joiner, Filter, lookup, etc. Used subqueries, joins, aggregations, rank over, joins, window functions, and other DML commands for data extraction. Also worked on session parameters and variables.
• Build reports using MS Excel for OneSource ERP application business users. Worked on the operation team for supporting the workflow and jobs using tools and technologies like COBOL, JCL, Oracle DB, shell scripting, and ServiceNow.
• Got trained in Business Intelligence and Performance Management(BIPM) Domain.
• Received on-the-spot awards from supervisor and appreciation from the customer for working round the clock and with strict deadlines.

Aug 2019 – Mar 2020 · 7 mos · Mumbai Area, India · Remote

• Content writer for the data science field. Wrote articles on data visualization using tableau, managing Big data in AWS, and many more.

Mar 2019 – Feb 2022 · 2 yrs 11 mos · Remote

• As a professional freelancer, I have worked on the below roles and projects
• Software Engineer:
• Application development for a hotel business company: Worked on API development and backend database design for a hotel search web application. Built Restful APIs using AWS API gateway and AWS Lambda. For design DynamoDB for key value-based data and Amazon Document DB(Mongo DB for more complex and nested JSON data. Helped the company retire the legacy system and reduced the operating cost by 60%.
• Snowflake Data Engineer:
• Analytics project for an event management company. Create a data pipeline to migrate data from the SQL server to the AWS Data lake and then ingest it in the Snowflake. Used Snowpipe and snowflake Tasks to orchestrate the pipeline, and stored procedures to transform the data. Also, worked on Hive and Hue for data warehousing and analytics. Data migration helped to reduce the operating cost by 50%.
• Azure Data Engineer:
• Covid-19 Reporting with Azure: Created Pipelines in ADF using Linked Services/Datasets/Pipeline/ to Extract, Transform, and load data from different sources like Azure SQL, Blob storage, and Azure SQL Data warehouse. Ingested data to one or more Azure Services - (Azure Data Lake, Azure Storage, Azure SQL, Azure DW) and processed the data in Azure Databricks.
• Used Power BI for reporting.
• DevOps Engineer:
• Cloud development project for an event management company. Create the infrastructure for data pipelines in AWS using AWS Cloud Development Kit for python and AWS Serverless Application Model. Used Azure DevOps for version control. Developed git flow strategy which helps save 100 of hours per week spent by developers on resolving the merge conflicts.
• Technical Trainer:
• Provided technical training on Data engineering concepts and AWS to IT professionals.
• Trained 70+ IT professionals and more than 90% of them successfully found relevant job.