Jan 2020 – Jul 2023 · 3 yrs 6 mos · Bengaluru, Karnataka, India

• I am a part of the Finance2.0 program which has a primary objective of tech modernisation. This involves the design and development of a central data platform called CILL which stores financial transaction feeds from 200+ source systems with multi-millions worth of transactions per day. The platform comprises of both batch and streaming based pipelines for data processing
• I am responsible for some of the key initiatives which include the following
• Provide technical leadership and guidance to multiple teams within the CILL program.
• Collaborate with the business and the various stakeholders to define and prioritise the technical roadmap for new features in CILL data platform
• Defining the Disaster recovery strategy/plan for CILL data platform
• Defining and building technical solutions to deliver operational excellence expectations via alerts/notifications
• Architecture and Design for the Archival Strategy for the project
• Define and drive the technical solutions/Architecture for new and existing functionality for the various components of the CILL platform

Jun 2019 – Dec 2019 · 6 mos · Greater Bengaluru Area

• Development / design of data lake built on amazon AWS cloud in order to ingest /store petabytes of cyber security related data generated by the various Symantec products. The system provides “Data as a service” which is then by used by the various application teams to run their analytics apps.
• My responsibilities were the following
• Building/designing a high performing data pipeline for ingestion of the data into the data lake
• Lead/mentor a team of junior engineers to ensure the delivery of high quality components
• Performance tuning of the ingestion pipelines to ensure faster processing and a quick turn around time.

Jan 2015 – May 2019 · 4 yrs 4 mos · Greater Bengaluru Area

• Elastic Search as a datastore to perform aggregations
• Design the elastic search datastore for optimal storage and querying of data from elastic search
• Performance tuning of elasticsearch w.r.t storage and faster query responses.
• Development of pipeline for ingestion of data from kafka into elasticsearch
• Specialities : ELK stack, Apache kafka, storm

Dec 2010 – Dec 2014 · 4 yrs · Greater Bengaluru Area

• Involved in the development of the Narus Targeted Intercept product which intercepts the traffic on the wire, does a deep packet analysis and extracts the information from the packets which is then rendered and displayed to the analysts. This will enable the analysts to have a view on what is actually passed through the wire.
• I was responsible for some of the key initiatives that involved migration of data from a normal file store to a distributed database. This includes
• Development / Design for using HBASE as a datastore for storing files of varied sizes.
• Instrumental in increasing the HBASE write throughput to 2-3Gbps in a 9 node cluster, which again involves cluster tuning, data batching, table design changes.
• Designed, executed the usage of HBASE for the storage of the rendered artefacts, which basically involves the storage of varied datasets from 10KB to 2MB
• Developed a framework for measuring the various aspects of the HBASE performance
• a. Write/ Read throughput
• b. Rate at which records get deleted

Feb 2010 – Dec 2010 · 10 mos · Greater Bengaluru Area

• Involved in the development of a product which enables centralized management of the cluster. The centralized cluster manager will perform following functionalities like
• process life cycle management at various nodes
• Collecting the statistics of process at various nodes
• Installing/uninstalling processes at nodes
• Notifications / Alarms concerning the cluster health

Jul 2005 – Jan 2010 · 4 yrs 6 mos · Greater Bengaluru Area

• Development of a taxnonomy generator that was used to create taxonomies from unstructured data for various domains using concepts of semantic web and based on the user inputs for the taxonomies. The taxonomies generated greatly helped in increasing the relevancy and the accuracy of the search results, when given as a input to any taxonomy based search engine. The taxonomy generation was done on a huge corpus which consists of GBs of data.
• Performance tuning of the application and large object handling played a key role here in addition to taxonomy generation.