Durga Venkatesh Kothapalli

Data Engineer

United States4 yrs 4 mos experience

Most Likely To Switch

Key Highlights

Expert in Data Engineering and Machine Learning.
Proficient in cloud services including Azure and GCP.
Strong background in Big Data technologies.

Stackforce AI infers this person is a Data Engineering expert with strong capabilities in Big Data and Machine Learning.

Contact

Skills

Core Skills

Data EngineeringMachine LearningDatabase ManagementBig Data Engineering

Other Skills

PythonSQLKafkaAzureSnowflakeTableauOracleJavaSASBig DataGCPHadoopPySparkAWSProblem Solving

Experience

4 yrs 4 mos

Total Experience

1 yr 5 mos

Average Tenure

2 yrs 5 mos

Current Experience

Yesjexpert

Senior Data Engineer

Jan 2024 – Present · 2 yrs 5 mos · India · Remote

Collaborated with Data Engineers and Data Architects to define backend data product requirements, including
aggregations, materialized views, and tables for visualization.
Demonstrated expertise with machine learning algorithms such as logistic regression, random forest, XGboost, KNN,
SVM, neural network, linear regression, lasso regression, and k-means.
Developed Kafka producer and consumer applications within a Kafka cluster setup, leveraging Zookeeper.
Created and published customized interactive Tableau reports and dashboards, along with scheduling data refreshes
using Tableau Desktop.
Managed data ingestion into Azure data services, including Azure Data Factory and Databricks.
Extensively utilized Python and SQL for data cleaning, aggregation, and processing tasks.
Constructed machine learning models, including SVM, random forest, and XGboost, for identifying potential new
business cases using Python Scikit-learn.
Provided IaaS support to clients on Microsoft Azure cloud & Optimized Snowflake views to enhance performance.
Developed jobflows using Python-based Apache Airflow for job automation.
Designed complex data mappings in Informatica and DataStage to load data from diverse sources into the data
warehouse.
Responsible for importing data from PostgreSQL to HDFS and HIVE using SQOOP, with experience in migrating

PythonSQLKafkaAzureSnowflakeTableau+2

Jiordino

Data Engineer

Jul 2021 – Sep 2022 · 1 yr 2 mos · Hyderabad, Telangana, India · Remote

Oracle 10g/11g RAC and Single Instance Databases management.
Database upgrades from 10g to 11g & Cron jobs for automated RMAN backups.
Oracle instance monitoring with Oracle Enterprise Manager.
Backup, Restoration, and Recovery using RMAN &Data migration using Export/Import and Oracle Streams.
Index reorganization and database refresh & Oracle Support (SR) process.
Remedy Incident, Change, and Service Request tickets

OracleJavaSASDatabase Management

Imr metallurgical resources ag

Big Data Engineer

Sep 2020 – Jun 2021 · 9 mos · Mumbai, Maharashtra, India · Remote

Data extraction, transformation, loading, and integration in data warehouse, operational data stores, and master data
management.
Processing data in Alteryx to create TDE for tableau reporting.
Experience with GCP Big Query for ETL transformations and Spark job development.
Analyzing Hadoop cluster and big data analytic tools including MapReduce, Hive, and PySpark.
Creation of yaml files for data sources and glue table stack creation.
Data validation from SQL Server to Snowflake.
Writing PySpark scripts for data quality checks and automation with Apache Airflow.
Senior Data Engineer
Hadoop cluster installation using various distributions.
ELASTIC MAPREDUCE (EMR) and Amazon AWS EC2 instance setup.
Hive/SQL queries conversion into Spark RDD and PySpark transformations.
Databricks Python notebooks for data processing in Azure data lake storage.
Azure cloud and Databricks deployment and configuration.
AWS Lambda functions and IAM roles for Python script execution.
Snowflake Data Platform Solution Architecture consulting.
Unit testing with Junit and automation environment setup.
Lambda Deployment function configuration for S3 events and System architecture documentation.

Big DataGCPHadoopPySparkAWSBig Data Engineering

Education

Stony Brook University

Master's degree

Aug 2022 – Dec 2022

National Institute of Technology Karnataka

Bachelor of Technology - BTech

Aug 2016 – Aug 2020

Stackforce found 100+ more professionals with Data Engineering & Machine Learning

Explore similar profiles based on matching skills and experience

Ronan Collobert

Research Scientist

at Apple

Menlo Park, United States25 yrs 1 mo exp

Machine LearningResearch

Mike Thacker-Cooke

Director of Recruitment

at KDR Talent Solutions

Middlewich, United Kingdom13 yrs 11 mos exp

Data RecruitmentTechnology RecruitmentBusiness Intelligence RecruitmentData Management Recruitment

Deepak Gupta

Talent Acquisition Group-Leadership Hiring (PAN India)

at EPAM Systems

Delhi, India16 yrs 3 mos exp

Talent AcquisitionLeadership Hiring

Mani Bhushan

Engineering leader

at OrbitronAI.com

Bengaluru, India10 yrs 7 mos exp

System ArchitectureCross-functional Team LeadershipGeospatial DataGeospatial IntelligenceAmazon Web Services (AWS)

Cheng Niu

Distinguished Research Scientist

at News Break

San Francisco, United States20 yrs 5 mos exp

Machine LearningNatural Language ProcessingArtificial Intelligence

Rebecca Lotus

Author & Speaker

at The Human Work

Steilacoom, United States16 yrs 9 mos exp

LeadershipWritingSoftware ManagementEngineering ManagementAPI Development

Emma Hershberger (Cockram)

Software Engineering Manager

at Ulteig

Wabasha, United States4 yrs 9 mos exp

Project ManagementSoftware ArchitectureContinuous Integration and Continuous Delivery (CI/CD)Full-Stack DevelopmentAzure DevOps

Mohammad Yeasin Arafat

Assistant Manager (Hr & Admin, Accounts, People & Culture, Liaison, Events & Training)

at Bfinit a cosmopolitan place for clients & products

Dhaka, Bangladesh9 yrs exp

HR AdministrationTraining ManagementAccount ManagementOperations ManagementVendor Management