Jul 2021 – Sep 2022 · 1 yr 2 mos · India

• Building finance applications at compass for real estate agents which shows the state of their business in near real time using GRPC microservices, Kafka, Airflow, Aurora DB, Kubernetes.
• Envisioned and lead the revamp of existing architecture to a more simple and scalable architecture. Broke down the revamp plan into phases keeping in mind that the product features planned ahead are not impacted.
• Led features from inception to release and worked extensively with product and other engineering teams to handle ambiguities.
• Introduced JPA/Hibernate for simplifying DB interaction and framework for integration tests for writing simple and completely testable code which enhanced the productivity of the developers in the team.
• Managed three engineers( 1 SDE-2 and 2 SDE-1) full time for 5 months.
• Worked on operational excellence of the system on a regular basis.

Jul 2020 – Jul 2021 · 1 yr

• Working in Associates Payments team which is responsible for identifying attributed orders and compute earnings for amazon associates and issue invoices to eligible associates across the world using SQS, RDS, DyanmoDb, Elastic Search, Spark, Redshift and AWS
• Org: Affiliate Marketing (https://affiliate-program.amazon.in/)
• Designed large scale distributed watchlist feature in MiniTV app using DynamoDb, SQS, Redis and AWS
• Mentored junior engineers and took the initiative to reduce the oncall tickets to improve operational excellence.

Jul 2018 – Jun 2020 · 1 yr 11 mos · Bengaluru, Karnataka, India

• Co-designer and implementor of the initial version of machine learning platform for realtime predictions and offline predictions over big data using Mlflow, Spark, Kafka, Hadoop. 20+ ML models running on the the platform processing over 1TB of data.
• Working on a large scale distributed resilient user profile store handling 30K RPS on an average using HBase, Aerospike, Kafka and Resilience4j.
• Designed and implemented near real time ingestion of time series data in ORC format into HDFS from Kakfa using Secor which helped data scientists to fetch user data and train and run their ML models on it.
• Reduced time for the spark job processing over 1 month of time series data by 5x by using local encryption and scanning HFiles (HBase raw files)
• Mentored junior engineers and worked on operational excellence of ML platform single handedly

Jan 2018 – Jun 2018 · 5 mos

Jul 2015 – Dec 2017 · 2 yrs 5 mos

• Built data platform using Storm, Kafka, Kylin, HBase, Redis, Hive, Hadoop and AWS at Helpshift
• Implemented stateless web service over REST for exposing analytics data rendered on the dashboard
• Implemented job scheduler service for reporting, ETL for Hive and Kylin cube jobs
• Redesigned the existing architecture to be GDPR compliant with minimum performance hit for customers in EU
• Implemented APIs for exposing agent performance analytics data used by PowerBI
• Implemented real time topology for calculating agent performance metrics stored in HBase using Storm
• Implemented stateless web service for exposing agent performance data using HBase as the primary store and Redis as the cache layer.
• Implemented backup system for events data using Secor
• Reduced time of generation of tags report by 30X by using bulk aggregation feature of Elastic Search
• Increased RPS of Storm ETL pipeline by 2X by tuning workers parallelism
• Reduced time for calculation of MAU by 4X by tuning the map reduce job