Girish Palli

Data Engineer

Bengaluru, Karnataka, India10 yrs 10 mos experience
Most Likely To SwitchAI ML Practitioner

Key Highlights

  • Over 10 years of experience in Data Engineering and DevOps.
  • Expert in building scalable, automated data pipelines.
  • Proven success in cloud migration and infrastructure optimization.
Stackforce AI infers this person is a Data Engineering and DevOps specialist with a focus on Cloud Infrastructure and Automation.

Contact

Skills

Core Skills

Cloud Infrastructure & DevopsData Platform EngineeringCloud Migration & OptimizationData Transformation With Dbt

Other Skills

Big LakeBigQueryC++ LanguageCodefreshDB2DBTData AnalysisData Build Tool (DBT)Data EngineeringData IntegrationData TransformationDataHubDataPlexDataProcDataStage

About

Experienced Data Engineer & DevOps Specialist | Cloud Infrastructure & Data Platform Engineer With over 10+ years of experience in Data Engineering and DevOps, I specialize in building scalable, automated data pipelines and cloud platforms. I thrive in environments that leverage modern cloud technologies, automation, and data transformation to deliver high-performance data systems. My expertise spans from cloud migration and infrastructure automation to data platform engineering, and I’m skilled in collaborating across teams to design and implement solutions that drive business value. Key Competencies & Achievements: Cloud Infrastructure & DevOps: Experienced in automating infrastructure on Google Cloud Platform (GCP) using Terraform, Codefresh, and GitHub Actions. Built end-to-end CI/CD pipelines that streamline deployment and scaling of data systems, improving efficiency and reducing deployment errors . Data Platform Engineering: Led the creation of a centralized Data Hub, delivering data products for business consumers. This platform improved access to data insights, driving a 30% increase in data product engagement. Data Transformation with DBT: Skilled in DBT to automate and manage data transformations, ensuring accurate and timely reporting. Implemented reusable data models and data quality checks, improving the reliability and accessibility of business-critical data. Cloud Migration & Optimization: Managed successful migrations of on-premise data infrastructure to GCP, optimizing processing workflows and reducing operational costs by 20%. Expertise with BigQuery, GCS , DataPlex , DataProc , Dataflow ,GMS(RBAC) and Pub/Sub to optimize data performance and reduce latency. SQL & Data Warehousing: Proficient in SQL, building optimised queries for large-scale data processing and ensuring seamless integration across multiple systems. Expertise in designing and maintaining high-performance data warehouses. Collaboration & Problem-Solving: Proven ability to collaborate with data scientists, business analysts, and IT teams to solve complex data challenges. Known for my strong problem-solving skills and ability to deliver continuous improvements in data workflows. I am passionate about leveraging cutting-edge technologies like GCP, DBT, and DevOps practices to automate and scale data workflows. With a strong foundation in cloud infrastructure and data engineering, I thrive in delivering next-generation data solutions that enable business intelligence and data-driven decision-making.

Experience

10 yrs 10 mos
Total Experience
2 yrs 8 mos
Average Tenure
4 yrs 9 mos
Current Experience

Anz

Data Engineer

Aug 2021Present · 4 yrs 9 mos · Bangalore Urban, Karnataka, India

  • Data Engineer & DevOps Specialist
  • Responsibilities:
  • Designed and implemented automated infrastructure on Google Cloud Platform (GCP) using Terraform, Codefresh, and GitHub Actions, enabling end-to-end CI/CD pipelines that reduced deployment times by 30%.
  • Led the migration of on-prem data systems to GCP, optimizing cloud storage and reducing operational costs by 20%, leveraging services like BigQuery, GCS, DataProc, GMS , DataPlex and Pub/Sub for better scalability and data accessibility.
  • Built and optimized a centralized Data Hub that supports data products for consumers, improving data accessibility and driving a 30% increase in user engagement and data utilization.
  • Developed and managed automated data transformations using DBT to streamline reporting processes, ensuring data accuracy and reducing ETL processing time by 25%.
  • Provisioned and managed cloud infrastructure with Terraform to support dynamic scaling, automate routine tasks, and enable efficient cloud resource management.
  • Collaborated with cross-functional teams to design scalable data pipelines, ensuring data quality, integrity, and performance across the platform, resulting in improved real-time analytics.
  • Optimized complex SQL queries for large-scale data warehousing systems, reducing query runtime by 20% and improving overall system efficiency.
TerraformCodefreshGitHub ActionsBigQueryGCSDataProc+6

Hsbc

Software Engineer

May 2019Aug 2021 · 2 yrs 3 mos · Hyderabad, Telangana, India

Optum

Software Engineer

Jul 2018Apr 2019 · 9 mos · Hyderabad, Telangana, India

Walmart mss

IBM InfoSphere DataStage and QualityStage Developer

May 2015Jun 2018 · 3 yrs 1 mo

  • Working with IBM- InfoSphere DataStage and QualityStage (Etl Tool)
  • Provides technical assistance to team and evaluate all codes.
  • Develop and implement tests on all data stage jobs.
  • Monitor all data stage jobs and provide production support to all.
  • Design and analyze all ETL jobs edition.
  • Manage work and apply various performing techniques.
  • Analyze work and implement all business regulations.
  • Coordinate with team members and administer all onsite and offshore work packages.
  • Schedule and plan all data stage tasks.
  • Review all functional business applications and specifications.
  • Design various block diagrams and logic flowcharts.
  • Maintain and perform tests on all Software Development Life Cycle and ensure required product.
  • Document all program level and user level processes.
  • Design and maintain various data warehouse.
  • Prepare ETL architecture and associate procedures.
  • Analyze performance and monitor work with capacity planning.
  • Roles & Responsibilites:-
  • Understand Requirements.
  • Prepare Questions.
  • Logical design (flow chart)
  • Physical design (coding)
  • Unit test
  • Performance Tuning
  • Peer view – if it is ok, proceed to next step, if else start again from 3 step
  • Design Turnover Document (DTD)/ Technical Design Document (TDD)/ Detailed Design Document (DDD) — > produces job Report
  • Back ups
  • Job sequencing

Education

Jawaharlal Nehru Technological University

"BTECH" — Computer Science Engineering

Jan 2009Jan 2013

Stackforce found 33 more professionals with Cloud Infrastructure & Devops & Data Platform Engineering

Explore similar profiles based on matching skills and experience