Apr 2024 – May 2025 · 1 yr 1 mo · Remote · Remote

• Enterprise Reporting Platform
• Scalable Microservices: Architected and implemented reporting microservices using Spring Boot, enabling efficient querying from OLAP databases such as Postgres, BigQuery, and Elasticsearch.
• Flexible Reporting Engine: Designed a dynamic reporting engine supporting 20+ configurable metrics and dimensions with automated SQL generation.
• Optimized Performance: Improved report generation times through advanced querying, in-memory aggregation, multi-threaded execution, and connection pooling.
• Automated Scheduling: Developed an automated report scheduling system that processes scheduled reports and delivers them via email.
• Enhanced Security & Collaboration: Implemented role-based access control and team sharing features, supporting 500+ internal users.
• Objectivity Database Migration
• Seamless Data Transition: Led the migration of a legacy Objectivity database to PostgreSQL, successfully transitioning over 10 years of historical data.
• Modern API Development: Architected a modern REST API service to replace legacy RMI calls, eliminating database dependencies.
• Real-Time Monitoring: Integrated monitoring systems (Graphite, Elasticsearch) to ensure real-time visibility into service health.
• Efficient Onboarding: Authored comprehensive technical documentation that significantly reduced new team member onboarding time.

Mar 2023 – Apr 2024 · 1 yr 1 mo · Remote · Remote

• Migrated the CDH Hive data pipeline to Google Cloud BigQuery, streamlining data processing and storage.
• Transformed a legacy Java MapReduce job into a Spark-based pipeline in GCP Dataproc, boosting performance by 80%.
• Revamped Elasticsearch indices retention policies.
• Developed a partition management schedules solution for Hive and S3 to automate handling of external tables based on retention policies. (InBuilt Retention Policies did suffice an adequate solution)
• Implemented Proper Object Data Models in the Java-based Kafka Consumer to enhance code extensibility and design.
• Incorporated self-management certification security standardization for Postgres Data Warehouses.

Jul 2021 – Apr 2023 · 1 yr 9 mos · Toronto, Ontario, Canada

• Data Engineering
• Collaborative Pipeline Development: Partnered with three product teams to build efficient and scalable streaming and batch data pipelines.
• AWS Data Pipeline Design: Engineered streaming data solutions using AWS Kinesis Analytics, SageMaker, and Feature Store.
• Batch Data Ingestion: Constructed a batch pipeline to ingest S3 data into DynamoDB using a PubSub architecture with AWS Lambda and Kinesis Data Streams.
• End-to-End Streaming Pipeline: Developed a streaming pipeline to transform Snowflake data in AWS using S3, asynchronous AWS Lambda, SQS, Snowpipe, and Kinesis Firehose—reducing ETL processing time from 3 hours to 5 minutes.
• ML Pipeline Optimization: Reduced machine learning pipeline latency through code optimization and advanced caching strategies to minimize API calls.
• Software Engineering
• API Development: Designed and implemented APIs with API Gateway, AWS Lambda, and DynamoDB.
• Automated Deployments: Automated Lambda function deployments using the AWS boto3 API for lab environments.
• CI/CD Implementation: Built a CI/CD cycle using Jenkins and CloudFormation for three products in production environments.
• MLOps Lifecycle Integration: Integrated and designed the MLOps lifecycle for generic machine learning product development.
• Best Practices Enforcement: Ensured adherence to top software engineering practices across the entire backend architecture.

Mar 2021 – Jun 2021 · 3 mos

• Responsible for teaching the following modules.
• Training students to use Python tools like numpy, pandas, matplotlib for performing statistical operation and do data analysis using the same.
• Illustrating data engineering to perform ETL operations using different tools.
• Explaining cloud and its infrastructure for building data analytics solutions on Azure
• Teaching Stats and Machine learning algorithms – Hypothesis testing, Linear and Logistic regression, Support vector machine, Naïve Bayes, Decision Tree, XGBoost

Aug 2019 – Jul 2021 · 1 yr 11 mos · Calgary, Alberta, Canada

• Ensured continuous provisioning of cleaned structured data for machine learning production systems.
• Building architecture for batch and stream processing using the Azure stack involving SQL server, Blob storage, Azure analysis service, Azure data factory, Azure data lake, Cosmos DB.
• Developed CI/CD pipleines using Jenkins, Docker, MySQL and bash scripts in collaboration with automation team.
• Designed and developed production level pipelines using big stack of Apache Spark, Kafka and Airflow.
• Improved previous data pipelines with Hive and PySpark to reduce turn around time by 70%.
• Migrated SQL-server loads from on-prem to cloud using techniques that provide near to zero downtime.
• Tuning and optimization of SQL queries for faster data transformation during ETL operations, data extraction and analysis across large volumes of data.
• Administered and Engineered Jenkins for managing Build, Test and Deploy chain, SVN/GIT with Dev/Test/Prod Branching Model for releases.

Apr 2018 – May 2019 · 1 yr 1 mo · Ahmedabad Area, India

• Worked on the Backend Development of Aditya Birla Capital Finance Components.
• Created APIs, Services, Serializers, Models for the new elements of the ABC Finance Components.
• Comprehensive usage of Version Control System (GIT) before joining the team in the ABC project.
• Recommended changes for Back-end Solutions through feedback via continuous integration meetings and analyzed the stakeholder’s involvement in the usage of such a huge project.
• Implementation of ABC Finance Backend with the help of Python Django Framework and Rewrote Efficient code from the old modules of Version 1.0 to make it compatible for the new version.

May 2017 – Jul 2017 · 2 mos · Pune Area, India

• • Development of Modules of Sites Management System that provides ERP Solution that encapsulates features related to Engineering, Esite, MIS etc.