Feb 2019 – Jan 2023 · 3 yrs 11 mos · Gurgaon, India

• Key achievements:
• 1. Building a reliable and central data lake (using Apache Hudi) for ~800 OLTP tables (MySQL + Postgres) using log based CDC over Kafka Connect and streaming pipeline, with reduced refresh SLA from 24 hours to every half hour along with lesser operational costs and better data quality.
• 2. Real time applications around business observability and pipelines using Apache Flink and Apache Pinot.
• 3. Automation for creation and deployment of ETLs over Airflow DAGs for non-technical end users using simplest of configurations.
• 4. Integration of several external services and building clickstream analytics as CDP.

Jul 2018 – Feb 2019 · 7 mos · Gurgaon, Haryana, India

• Responsible for automation of extraction in contracts' obligation and metadata by:
• 1. Data warehousing and Data ingestion using Data pipeline tools (Rabbitmq, Kafka, Spark).
• 2. Document parsing using Aspose.
• 3. Extraction using Machine learning and NLP (Stanford NLP and spacy) models.
• 4. Storage of information in document based DB (MongoDB).
• 5. Classification and categorization using scikit-learn classifiers, gradient boosting, etc.
• 6. Text sequence classification by Deep learn models (keras, Tensorflow), using RNN, Attention networks, auto encoders, sequence models.
• 7. Flask, queuing system, Asynchronous non-blocking processing using asyncio for providing ML as a service.

Jun 2017 – Jul 2018 · 1 yr 1 mo · Noida Area, India

• Responsible for development of up and new features for Resdex, Resdex Mobile APIs and ensuring healthy state of web application.
• Developed several new spring boot and symfony microservices. Also involved in transition and refactoring of older features into new microservice architecture (php to spring boot).

Jun 2016 – May 2017 · 11 mos · Noida Area, India

• Developed several mailer and search feaures for resdex, also built mobile apis recruiters' app, OTP, authentication, notifications, etc.