Narendra Reddy

Associate Consultant

Bengaluru, Karnataka, India9 yrs experience

Highly Stable

Key Highlights

Over 8.5 years of experience as an Azure Data Engineer.
Expert in optimizing data processing pipelines.
Strong business awareness from product lifecycle involvement.

Stackforce AI infers this person is a Data Engineering professional with expertise in cloud-based data solutions and pipeline optimization.

Contact

Skills

Core Skills

Azure Data FactoryData ProcessingContinuous IntegrationContinuous Delivery (ci/cd)PythonHadoop

Other Skills

Data PreparationAgile EnvironmentData PipelinesData IngestionData QualitySQLAzure DatabricksApache SparkJenkinsAnsiblePySparkScala in SparkGraphQLSqoopSpark-sql

About

Software professional with over 8.5+ years of experience specializing as a Azure Data Engineer. My greatest strength is business awareness acquired by being involved in product life cycle from development to deployment in an agile environment.

Experience

Concentrix catalyst

2 roles

Consultant

Promoted

Sep 2024 – Present · 1 yr 6 mos · Hyderabad, Telangana, India · Hybrid

Software professional with over 8.5+ years of experience specializing as a Azure Data Engineer. My
greatest strength is business awareness acquired by being involved in product life cycle from development to
deployment in an agile environment.

Data PreparationAzure Data FactoryAgile EnvironmentData ProcessingData PipelinesData Ingestion+4

Consultant

Sep 2024 – Oct 2025 · 1 yr 1 mo · Hyderabad, Telangana, India · Hybrid

JenkinsContinuous IntegrationContinuous Delivery (CI/CD)

Infosys

2 roles

Senior Associate Consultant

Mar 2022 – Sep 2024 · 2 yrs 6 mos

JenkinsAnsible

Senior Associate Engineer

Mar 2022 – Aug 2024 · 2 yrs 5 mos

Involved in designing and building entire product from scratch. Optimized existing pipelines and rewritten existing code in UDF to data frame. Having good experience in migrating code from pandas to spark. Having good experience in using unity catalog and delta tables. I have written re-usable python and pyspark code for all pipelines. Involved in processing the data using pyspark and scala spark and dump data back to respective places (S3 or NoSQL server). Performed data processing on huge data understanding the requirement and analyzing which transformation will be used with better performance and type of joins helps us to reduce the shuffle. Helped in optimizing the transformations which helped to reduce usage of resource in EC2 instance. Written backend service using python and graph-ql queries for providing data to frontend. Involved in designing and building entire product from scratch.

PythonPySparkScala in SparkData ProcessingData QualityGraphQL

Tech mahindra

3 roles

Data Engineer

Promoted

Dec 2019 – Mar 2022 · 2 yrs 3 mos

Pulling the data from client location using Sqoop or Spark-sql. Storing the data on HDFS and Creating HIVE tables for initial level of cleaning data. Combing the data from different tables as single table (JOINS). After that processing the data by using Spark and Python as required for analysis (Normalization, Encodings, Missing Values etc.,). Using UDF’S, transformation and actions for processing the data, Optimizing the time complexity by checking the UI how data is Shuffling and Ram consuming depending on that benchmarking the executor core and ram.

SqoopSpark-sqlHDFSHIVEPythonData Processing+1

Software Engineer

Mar 2017 – Mar 2022 · 5 yrs

Software Engineer at TechMahindra

deploypods

Data Engineer

Mar 2017 – Nov 2019 · 2 yrs 8 mos

Implemented hive partitions & bucketing according to the end users’ queries. Analyzed the file formats which one to use in current scenarios. Spark Jobs for Data Preparation for clients. Worked as Data Engineer for an Insurance client in developing their provisioning system. Worked as Data Engineer for an Aircraft client in developing their provisioning system. Writing the cron jobs for automatic the jobs on monthly or Weekly level from data pulling, processing and running inference and pushing the data back to servers.

hive partitionsSpark Jobscron jobsData Processing