Mar 2022 – Present · 4 yrs 2 mos

• Here are some of the exciting challenges that I work on
• Current I am responsible for design and implementation of cloud data warehouse solution for Ad platform. This involves several things including
• designing data models and schema design right from staging layer until data marts
• CI/CD automation to implement all the data warehouse components including users, roles, privileges, schema etc. Providing seamless integration of data warehouse platform with CI/CD automation
• Design and implement automated pipelines for data quality checks and defining the rules to ensure quality of data
• Implementing Audit, balancing and control. A framework for auditing of all pipelines to provides status and recency of every pipeline along with historical data for trend analysis.
• Metrics and telemetry of data warehouse platform which can not only track the availabilit, but also the cost at various levels, adoption rate, growth etc.
• I am also collaborating with data pipeline team to define the data ingestion pattern for data pipelines. Migrating data pipelines from on-prem to AWS making scalable platform for data ingestion, transformation and processing.
• As a platform engineering I am also working on building several solutions which has direct impact on the platform across multiple teams
• Currently i am working on creating a platform level solution for change data capture. This will provide a centralised pub/sub model to delivery near real-time data between producers and consumers.
• A centrail CI/CD framework and pipelines to implement schema change management, leading to version control and governance on data deployments.
• Wearing a hat of database engineering, I review application design across Ad platform to understand the use case and suggest the right storage solution, schema design and access pattern.
• I am also working on automations with AWS platform development team to build several automation to provision components in AWS using IAAC

Jun 2017 – Apr 2022 · 4 yrs 10 mos