Jan 2020 – Mar 2022 · 2 yrs 2 mos · Bengaluru, Karnataka

• Developed Live Publisher which takes real time data from GS- Proprietary table data structure and put it in Kafka in a GS-Proprietary data format.
• Developed EOD Publisher which takes real time data from GS- Proprietary table data structure and put it in Kafka in a GS-Proprietary data format.
• Worked single handedly with Arctic team for the integration of their Streaming API and wrote consumers for Live and EOD Data which reads data from Kafka and put it in Chunkstore which is a GS-Proprietary time series database for long term persistence via this Streaming API.
• Reformatted many repos to simplify micro-services.
• Worked on developing API which gives historical trend of market data.
• Worked on developing APIs for a governance app which dictates the
• responsibility and tracks the user who have made changes to an asset.
• Worked on writing EOD Market Service which provides data to clients in a
• streaming format or REST API.
• Improved an existing system by 92% by working on a POC which took data
• from an external system and made it readily available in our Cache.
• Designed a Prober system to send metrics to Prometheus client to set up an
• alarming system in case of failures.

Apr 2018 – Jan 2020 · 1 yr 9 mos

• 1) Generic Cube Framework: Developed in-house OLAP cube framework which gives the user the flexibility to slice, dice and aggregate data while performing joins with various data sources such as HBase, RDBMS, etc., plugging in user-defined functions and exploding rows with user-defined code. Also, the framework had a rich availability of user-defined aggregate functions including some difficult unique count UDAFs scenarios also which uses HyperLogLog.
• 2) Feature Factory Framework: Developed Feature Factory which combines in-house OLAP Aggregation cube framework and pluggable features which can also involve machine learning. This framework is being used across teams in Mobileum for reporting and detecting frauds.
• 3) Generic Reconciliation Engine: Built a framework which has the capability to reconcile any two data streams on the basis of rules/instructions being specified in a configuration file. This framework was used to reconcile almost 100-110 streams for our client Progresif Cellular, Brunei.
• 5) IRSF DB: Developed a framework to reconcile and resolve the number ranges from various streams.
• 6) SIP Voice Data Model: SIP model project involved writing Hadoop Record Readers for decoding the capture packets i.e. Pcaps, developing a Spark Streaming application for getting the dialogs in real-time and writing a batch job to generate all the voice calls from the data.
• 7) CLI Fraud Detection: Designed, devised and wrote rules with which one can detect Caller-Line Identification frauds from the voice data.
• 8) Roaming Insights legacy system backend was ported by me from pig to Hadoop, Oozie and Spark. This involved thinking of many reporting angles such as generalizing time granularities configuration for each Stream for which reporting was to be done.
• 9) Did many of the optimization processes such as a reduction in Hbase read time, spark job time, hive, etc.

Jul 2017 – Mar 2018 · 8 mos

• Solving various use case with Big Data Analytics

Apr 2017 – Jun 2017 · 2 mos · Gurgaon, India

Jul 2015 – Mar 2017 · 1 yr 8 mos · Gurgaon, India

• Worked in serving and optimization of Brand and Product Ads.
• Used Netty based Rest Express for REST APIs.
• Designed a data flow to ensure consistency of data on all servers using Cassandra and AWS S3.
• Implemented Lucene based indexing to reduce heap size and load on Ad Servers.
• Designed a Codahale metric based monitoring framework to identify production issues and to
• monitor health of the system.
• Worked on a system which manages the budget of the campaigns.
• Also used aerospike for a certain component in AdServer.

May 2014 – Jun 2014 · 1 mo

Aug 2011 – May 2015 · 3 yrs 9 mos

Jun 2013 – Jul 2013 · 1 mo