Arullaldivakar Jayapalan — SRE (Site Reliability Engineer)

Accomplished Senior Hadoop Site Reliability Engineer with a decade of comprehensive experience in the dynamic realm of Big Data and Cloud technologies. My journey commenced at IBM as a Hadoop Support Engineer, evolving into an impactful role as a System Architect for Big Data platforms at NetApp, and subsequently contributing significantly as a Senior Site Reliability Engineer at Visa and Apple India Pvt Ltd. Throughout my tenure, I've been adept at orchestrating and leading diverse teams, handling multiple clusters (with up to 150 nodes), and architecting robust solutions for various high-stake projects. My expertise spans a wide technological spectrum, encompassing Cloudera, Hortonworks, AWS, Kubernetes, Docker, and a host of tools within the Hadoop ecosystem. Noteworthy accomplishments include spearheading the design and deployment of pivotal tools like AP-Notebook@AWS, instrumental in driving data exploratory works at Apple India. At Visa, I played a key role in integrating kube-prometheus for observability, migrating legacy systems to Kubeflow, and introducing custom authentication via Dex for Kubernetes. My proficiency extends beyond technical skills, encompassing project leadership, capacity planning, architecting hybrid cloud solutions, and implementing robust security measures using Kerberos, Apache Sentry, and Ranger. I hold esteemed certifications as a Cloudera Certified Hadoop Admin Engineer and AWS Certified Solutions Architect, underpinning my expertise. Additionally, my educational background in Aeronautical Engineering serves as a testament to my strong foundation in problem-solving and analytical thinking. My contributions have consistently received recognition, from architecting efficient migration processes to automating cluster builds, earning accolades and awards for innovation, dedication, and superior performance. I'm eager to leverage my extensive experience, technical prowess, and leadership skills to drive innovation, optimize operations, and contribute significantly to new challenges and opportunities in the evolving landscape of Big Data and Cloud technologies.

Stackforce AI infers this person is a Big Data and Cloud Infrastructure expert with a strong focus on SRE and Kubernetes.

Location: Bengaluru, Karnataka, India

Experience: 12 yrs 10 mos

Skills

Site Reliability Engineering
Hadoop
Kubernetes
Machine Learning
Big Data
Hadoop Administration
Cluster Automation
Etl

Career Highlights

Led migration of legacy systems to modern cloud solutions.
Architected robust big data platforms with significant cost reductions.
Recognized for innovation and performance excellence in multiple roles.

Work Experience

Apple

Lead SRE Engineer (5 yrs 11 mos)

Visa

Staff Software Engineer (7 mos)

NetApp

Bigdata Platform Architect (System Architect) (2 yrs 6 mos)

Infogain

Hadoop Admin (9 mos)

IBM India Private Limited

Hadoop Bigdata Admin (3 yrs 1 mo)

Education

Bachelor of Engineering (B.E.) at Madras Institute of Technology Campus

Arullaldivakar Jayapalan

SRE (Site Reliability Engineer)

Bengaluru, Karnataka, India12 yrs 10 mos experience

Most Likely To SwitchHighly Stable

Key Highlights

Led migration of legacy systems to modern cloud solutions.
Architected robust big data platforms with significant cost reductions.
Recognized for innovation and performance excellence in multiple roles.

Stackforce AI infers this person is a Big Data and Cloud Infrastructure expert with a strong focus on SRE and Kubernetes.

Contact

Skills

Core Skills

Site Reliability EngineeringHadoopKubernetesMachine LearningBig DataHadoop AdministrationCluster AutomationEtl

Other Skills

Team LeadershipgrafanaprometheusKubeflowDexAirflowCassandraHBasePentahoGo (Programming Language)TerraformSplunkShell ScriptingCore JavaCustomer Service

About

Experience

12 yrs 10 mos

Total Experience

2 yrs 6 mos

Average Tenure

5 yrs 11 mos

Current Experience

Apple

Lead SRE Engineer

Jun 2020 – Present · 5 yrs 11 mos · Hyderabad, Telangana, India · On-site

HadoopTeam LeadershipSite Reliability Engineering

Visa

Staff Software Engineer

Nov 2019 – Jun 2020 · 7 mos · Bengaluru, Karnataka, India · On-site

Visa Kubernetes Platform (Pharos)
Visa’s Machine Learning Platform (MLP) is a fully managed, multi-tenant, scalable platform to build and support offline Machine
Accomplishments:
Kubernetes Integration: Integrated kube-prometheus, Kubeflow, Dex for superior observability and reliability on Kubernetes.
Legacy Migration: Successfully migrated legacy solutions to Kubeflow, ensuring seamless operations of critical applications.
Airflow Onboarding: Onboarded the Airflow application to the Kubernetes platform for streamlined workflow management.
Kubernetes-Cluster Authentication: Integrated Kubernetes clusters with Dex for enhanced custom authentication.
Enhanced Platform Metrics: Designed and implemented availability & reliability dashboards for tier 1 applications running on Kubernetes.

grafanaprometheusKubernetesMachine Learning

Netapp

Bigdata Platform Architect (System Architect)

May 2017 – Nov 2019 · 2 yrs 6 mos · Bengaluru Area, India

NetApp is a seller of Storage device and AutoSupport (“ASUP”) is the “phone home” technology available to NetApp customers that subscribe to NetApp’s AutoSupport service and enables NetApp products to automatically send configuration, log and performance data through SMTP, HTTP or HTTPS protocols to NetApp data centers. Here we are administrating 140 node hadoop cluster with the use of Cloudera manager and various hadoop eco system techologies
Architectured and designed the entire Hadoop Migration Process for flume, hive and Hbase data without downtime
Single handed excution of the entire bigdata migration activity
Architecuted and implemented the entire Platform with NFS datastore thereby reduce 50% of the total infrastructure cost and license cost.
Lead the entired Datstax cassandra migration from tradiotional Thrift based to CQL based storage
Implemented NFS as backend storage for the entire Datalake cluster(hdfs,hive,hbase) using the Netapp NFS connector Solution.
Implemented efficient hadoop data archive policy to HDFS, NFS and NetApp Storage Grid Webscale (s3)
Automated and build the CDH cluster in aws cloud using ansible and CDH director API.
Achitected Hybrid Cloud solution using the NetApp Private storage(NPS) & AWS

Big DataCassandraHadoop

Infogain

Hadoop Admin

Aug 2016 – May 2017 · 9 mos · Bangalore · On-site

ASUP5 Redesign: Redesigned ASUP5 ecosystem, decreasing infrastructure costs by 40% through storage and compute decoupling.
End-to-End Cluster Automation: Automated CDH & Datastax cluster builds, significantly reducing build times by 60%.
Datastax Cluster Automation: Automated deployment and management of Datastax clusters, improving operational efficiency by 80%.
Hadoop & Cassandra Optimization: Fine-tuned Hadoop and Cassandra clusters, enhancing performance by 30% and 25% respectively.
Cluster Capacity Planning: Executed well-defined capacity planning for Cassandra & Hadoop clusters, ensuring optimized resource utilization.

HadoopHBaseHadoop AdministrationCluster Automation

Ibm india private limited

Hadoop Bigdata Admin

Jul 2013 – Aug 2016 · 3 yrs 1 mo · Bengaluru Area, India

RESPONSIBILITIES
Monitoring/Controlling the cluster via Cloudera Manager
Executing Map reduce jobs on Cluster.
Bring up/down the application for cluster upgradation/issues.
Performing deployments on Flume Blades and resolving the issues.
Creating & Running hive queries to generate the reports for analysing purpose.
Working closely On ETL and trouble shooting the issues.
Working closely with vendors such as Pentaho,Cloudera,Splunk
Performing Deployments on Pentaho nodes and Scheduling the jobs using Pentaho
Monitoring the solr/cassandra performance using Datastax.
Interacting with Internal Customers and end users via SNOW.
Developing and Maintaining Hive queries(Hql)and Hbase queries in CLI(opsware) and GUI(Hue)
File system management and monitoring.
Knowledge of HBase and Hive administration.
Partitioning of tables on Hive, Adding/Dropping of partitions
Importing and exporting data from HDFS to RDBMS vice versa using Sqoop.
Monitor and report data usage across the clusters.
Additional Responsiblities:
Performing Data Analytics using Apache Spark/Mapreduce progarmming models.
Debug MR Jobs and creating adoch Hive UDF’s for Business reporting.
Splunk Developer for reporting and analysis of the logs and input websrteam data.