Jan 2023 – Dec 2023 · 11 mos · Bengaluru, Karnataka, India

• IDP Team

Jul 2021 – Dec 2022 · 1 yr 5 mos · Bengaluru, Karnataka, India

• Core Data & Metrics team; responsibility is to prepare core datasets for analytics purposes & calculate core metrics for Twitter. Large workload of greater than 80 million QPS and PB of data.
• Led & delivered Project Stork, which involves migration of a Scalding Map Reduce job to a Dataflow job.
• The job ingests Twitter engagements & impressions onto GCS & BigQuery on a scale of 3.8 B records/hour or 200 TB/day.
• Gathered customer requirements, proposed PRD & TDD, tested out a PoC, analysing cost, followed by a quick implementation & a methodical deployment.
• Improved upon the SLA of the sink dataset by ~6-7 hours, revamped the architecture and modernized the pipelines for faster development & easier adoption.
• Worked as a primary contributor to Project Data Validation Framework, a tool that helps establish credibility of the data generated by any modernized Twitter pipelines by validating the datasets against the benchmark datasets generated by existing pipelines.
• Proposed a modular & extensible LLD & led the implementation effort, allowing for customized plugins & validation checks, smooth customer interface, & supporting all possible Data Completeness & Correctness checks out of the box;
• Reduces the developer’s effort of performing manual data validation(~1 month effort) to merely specifying a few configurations(~2-3 days effort).
• Powered by a batch Dataflow pipeline, scales well on order of PB of data by running validation checks in less than an hour.
• Working on Project Eraser, which helps scrub personal data off the Twitter datasets in order to maintain them PDP compliant.
• Implemented different scrubbing logic, to analyse for efficiency & resource consumption against different scales of datasets'(TB or PB).
• Helped in the onboarding & served as a mentor to an SDE-1 & also an intern, by conducting 1:1 sessions to help them ramp up, & also debug technical issues.

Feb 2021 – Jul 2021 · 5 mos · Bengaluru, Karnataka, India

• Developed an efficient and scalable real-time logging filter system for the Real Time Bidding Platform, that delivers 9 billion impressions/day.
• Working with the Programmatic Buying Team, shifted internal service calls from JSON to Smile data format, reducing the payload by 43%.
• Shifted access-logs logging from default Tomcat configuration to log4j loggers, improving the logging rate by 30 times.

May 2020 – Jul 2020 · 2 mos

• Built a RESTful API using Spring Boot, Hibernate, Visa APIs \& Java Cryptography API to process secure, reliable & shareable payment links.
• Improved the responsiveness of Visa Pay application by 40% by issuing signals to the backend API when the link gets created on merchant end.

Jun 2019 – Aug 2019 · 2 mos

• • Devised an algorithm for building Distributed Representations of Collusive Tweets submitted to blackmarket services, achieving an accuracy of 83%