Nov 2021 – May 2023 · 1 yr 6 mos · London, England, United Kingdom

Sep 2016 – Oct 2021 · 5 yrs 1 mo · London, England, United Kingdom

• Working on K8s,AWS,Kafka,Nifi,Hadoop ,Docker,Mesos,Marathon,MongoDB,Jenkins,Chef,Vagrant,Flume,Sqoop,K8s,Kerberos,IBM CDC
• Currently
• Taking Big-Data group in Top Bank with Transformation Journey.
• Owning DevOps and Containerisation Adoption in Transformation Journey and changing ways of working.
• From Building Solution to Planning,Roadmap,R&R,Rollout,Adoption
• Running HDP Cluster on containers with Kubernetes as orchestrator.
• CI/CD pipelines on containers with Kubernetes.
• Measuring Benefits during/after Transformation Journey.
• Owning [Big Data-Infra for Data Lake Solution in Multitenant Environment for Top Investment Bank-DevOps-CICD-Security-DR-Monitoring ] Building Strong Production Support Team
• 1)Leading the discussions with client for setting up Infra,Deployment and Monitoring and bring those to conclusions.
• 2)Design and build batch and real time Multitenant Secure Hadoop Cluster following Lambda Architecture.
• 3)Setup Deployment Model CICD using Jenkins and Ansible.
• 4)Build Strong SRE/Support Team for Production Support and Issue resolutions.
• 5)Leading Incident Management and Post-Mortem calls related to Production Issues.
• 6)Participating in Design Discussion and contribute to resolve issues faced in Big Data Multitenant Environment.

Feb 2016 – Sep 2016 · 7 mos · Gurgaon, India

• AWS Stack,Kafka,Hadoop,Docker,Mesos,Marathon,Nginx,Jenkins,Chef,Vagrant,Flume,Sqoop
• 1)Setting up Hadoop Cluster for various APIs.
• 2)Setting up Private Docker Registry.
• 3)Setting up staging docker based kafka cluster on Mesos/Marathon.
• 4)Setting up production Kafka Cluster for various API across organization.
• 5)Setting up Kafka-Flume-Hdfs pipeline for real-time dump of kafka event in hdfs for analytics.
• 6)Migrated All API from Unix System to docker containers on Mesos/Marathon.
• 7)Setting up standalone dev envirnments using Vagrant.
• 8)Service discovery on Mesos using consul.
• 9)Provisioning EC2, RDS, ElasticCache instances on AWS.
• 10)Load Balancers and Nginx configuration.
• 11)Modifying chef recipes.
• 12)Work with developers to setup dev environment using Vagrant,AWS and deploying APIs on live platform using GoCD/Jenkins.
• 13)Currently working on setting on SMACK(Spark,Mesos,Akka,Cassandra,Kafka) Cluster.
• 14)Monitoring Askme.com and Big Data Infra.

Aug 2011 – Jan 2016 · 4 yrs 5 mos · Gurgaon, India

• Deployment of Guavus Analytics on Tier1/Tier2 ISP in US.
• Onsite critical upgrade support from hadoop to hadoop yarn.
• Hadoop Software installation and Port configuration
• Configure Name Node High Availability
• Hadoop Cluster Software patching and upgrades.
• Database connectivity for the Hadoop Cluster
• HDFS management and monitoring.
• HDFS support and maintenance.
• Cluster maintenance including creation, addition and removal of data or name nodes
• Manage and review Hadoop/Oozie log files.
• Configuration,Running,Troubleshooting Oozie MR Jobs.
• Module wise data validation from raw flow received, data dropped,annotated data and data showing up on UI.
• Performance testing in staging environment to verify the system performance, UI performance and Modulewise functionality.
• Modularise Data integrity and Data Validation practices.
• UAT with customer.
• Interacting with sales, solutions Architect, customer from/during E2E deployment activities.
• Reviewing and updating SRD and PRD.
• Work closely with Product Development, Product Manager and other stake holders to collaborate on Bugs and issues which require deep expertise.
• Perform deep dives into both systemic and latent reliability issues; partner with software and systems engineers across the organization to produce and roll out fixes
• Customer facing role, interacting with customer and resolving issues raised by customer.
• Writing MOPs and IOGs.
• Apply in-depth analysis of hadoop based workload, project-based work, design solutions to issues, and evaluate their effectiveness

Jun 2011 – Jul 2011 · 1 mo

Jun 2010 – Jul 2011 · 1 yr 1 mo

• In this role , i work closely with engineers, product managers, and partners across various markets globally to deploy, maintain and support the solution while insuring uptime and availability of Yahoo Sponcered Search and related properties.I troubleshoot and resolve cases related to the delivery of Yahoo Sponcered Search services.
• I am responsible to roll out and manage state of the art infrastructure, systems and tools that help Y! to deliver uninterrupted services 24x7 to millions of users worldwide based on ground-breaking features while ensuring a high level of scalability, uptime, and responsiveness. This also includes responsibilities such as responding to outages.
• 1)I work with development/QA team to prepare/configure the necessary infrastructure components required for the development, testing ,staging & release of products components from time to time.
• 2)Deployment of products/software to various environments, work closely with release team to deploy products to production environment addressing scalability, availability, performance and monitoring capability.
• 3)Root Cause Analysis,troubleshooting and fixing of problems which impacts system stability, availability, scalability and performance.
• 4)Handing production change management,process improvements and documentation.

Jun 2008 – May 2010 · 1 yr 11 mos

• Configuration, Administration and maintenance of HPC thin clients clusters.
• L2/L3 linux support.
• Kernel compilation,optimization,applying patches
• Analyzing kernel crash dumps using - Kexec and kdump
• Troubleshooting application issues.
• Patch implementation & maintenance, Licensing.
• Service Management – NIS/DHCP/NFS/Samba/AUTOFS/FTP/SSH etc.
• Handling File system related issues, Mounting file systems, Network share FS, Auto mounting.
• Monitoring performance of servers i.e. CPU utilization, Memory usage,performance and benchmarking etc
• Monitoring Analysing OS logs- Monitoring System error logs, Reports error logs etc
• User issues/client issues monitoring and handling.
• Building Open Source Software and RPMs
• Security Administration – Add/remove/Troubleshoot IPTables Security Policies.

Nov 2007 – May 2008 · 6 mos · New Delhi Area, India

Feb 2006 – Oct 2007 · 1 yr 8 mos