Sep 2022 – Jun 2025 · 2 yrs 9 mos

Oct 2020 – Sep 2022 · 1 yr 11 mos

Mar 2020 – Oct 2020 · 7 mos

• Helped kick start a potential $300,000 cost saving initiative to reduce dependency on external vendors for data availability by writing an internal kstream app to process 5k messages/sec.
• Migrated team's 25+ jobs from an internal tool to airflow to allow better maintainability and remove support issues with previous tool as part of day away. This in turn led to reduction of support issues from twice a week to 0 per week.
• Migrated application critical jobs from an expensive internal tool to dedicated spark jobs on airflow thereby progressing the deprecation path for the tool.

Mar 2018 – Mar 2020 · 2 yrs

• Worked on writing a GraphQL server as a replacement for the current sproc + proxy architecture used for surfacing analytical data to hotel partners. Led the complete process from requirement gathering, design discussion with team, development, business uat, performance testing and production deployment.
• Led the development of a business critical feature(potential revenue impact $21M) in current spark streaming pipeline along with one another teammate. Was responsible for driving the business and product level communication along with overall code base refactor into a common library to facilitate development speed and support future use cases. Simultaneously provided the solution to a streaming problem where system produced correct result for out of order logically dependant messages.
• Reduced the processing time and increased the throughput of a spark application from 12 hours to 2 hours by parallelizing HDFS copy requests and changing the algorithm to remove persist calls.
• Improved the processing speed of an existing spark job by removing database dependencies for I/O (down to 6hrs from 9hrs). Reduced the overall cost of processing from $4k/request to $2k/request by migrating the spark application to common data lake for I/O. Created an automated testing pipeline which checked for data quality and reported the status on Kibana. Tech used - Spark, S3, Qubole, Hive, NodeJS, Elasticsearch, Kibana
• Re-designed an existing spark application to conform to micro-service architecture by removing hard database dependencies and adding API support. Linked the spark application to a central web application to help improve maintainability and scalability. Tech used - NodeJS
• Designed a fully self served UI for analyst community to automate big data batch job requests and align data quality with the existing streaming system while simultaneously refreshing downstream systems. Tech used - NodeJS, AngularJS, Scala, Spark, Qubole Data Service.

Jul 2016 – Mar 2018 · 1 yr 8 mos

• Working as a part of Competitive Intelligence(CI) team which provides on-demand and real time data to analysts, market managers, hotel partners on EPC(Expedia Partner Central) to provide insights into Expedia's competitive landscape.
• Developed a cloud based solution that involves reprocessing billions of scrapes of raw historical data in minutes to provide actionable data about competitors to consumers. Technologies involved - Spark, SparkSQL, SQL, Redshift, S3, Hive, Qubole(QDS).
• Worked on distributed computing that involves writing complex SQL code using Scala with Spark as the compute engine.
• Managed end to end delivery of moving processing of ETLs to AWS in near real time and setup Operational Excellence, data accuracy and code quality checks.
• Good understanding of distributed technologies like Spark, Hive, Hadoop, Tez.

Aug 2015 – May 2016 · 9 mos · NSIT

• Took various workshops on getting started with programming/competitive programming.
• Organized programming contests on judges.

Jun 2015 – Jul 2015 · 1 mo · Greater Hyderabad Area

• Worked on creating a backstage prototype of Microsoft Visio for iOS platform.
• Figured out the key elements of the existing code base of Microsoft's office apps and integrated functionality according to UI and backstage requirements,