Durga Venkatesh Kothapalli

Data Engineer

United States4 yrs 4 mos experience
Most Likely To Switch

Key Highlights

  • Expert in Data Engineering and Machine Learning.
  • Proficient in cloud services including Azure and GCP.
  • Strong background in Big Data technologies.
Stackforce AI infers this person is a Data Engineering expert with strong capabilities in Big Data and Machine Learning.

Contact

Skills

Core Skills

Data EngineeringMachine LearningDatabase ManagementBig Data Engineering

Other Skills

PythonSQLKafkaAzureSnowflakeTableauOracleJavaSASBig DataGCPHadoopPySparkAWSProblem Solving

Experience

4 yrs 4 mos
Total Experience
1 yr 5 mos
Average Tenure
2 yrs 5 mos
Current Experience

Yesjexpert

Senior Data Engineer

Jan 2024Present · 2 yrs 5 mos · India · Remote

  • Collaborated with Data Engineers and Data Architects to define backend data product requirements, including
  • aggregations, materialized views, and tables for visualization.
  • Demonstrated expertise with machine learning algorithms such as logistic regression, random forest, XGboost, KNN,
  • SVM, neural network, linear regression, lasso regression, and k-means.
  • Developed Kafka producer and consumer applications within a Kafka cluster setup, leveraging Zookeeper.
  • Created and published customized interactive Tableau reports and dashboards, along with scheduling data refreshes
  • using Tableau Desktop.
  • Managed data ingestion into Azure data services, including Azure Data Factory and Databricks.
  • Extensively utilized Python and SQL for data cleaning, aggregation, and processing tasks.
  • Constructed machine learning models, including SVM, random forest, and XGboost, for identifying potential new
  • business cases using Python Scikit-learn.
  • Provided IaaS support to clients on Microsoft Azure cloud & Optimized Snowflake views to enhance performance.
  • Developed jobflows using Python-based Apache Airflow for job automation.
  • Designed complex data mappings in Informatica and DataStage to load data from diverse sources into the data
  • warehouse.
  • Responsible for importing data from PostgreSQL to HDFS and HIVE using SQOOP, with experience in migrating
PythonSQLKafkaAzureSnowflakeTableau+2

Jiordino

Data Engineer

Jul 2021Sep 2022 · 1 yr 2 mos · Hyderabad, Telangana, India · Remote

  • Oracle 10g/11g RAC and Single Instance Databases management.
  • Database upgrades from 10g to 11g & Cron jobs for automated RMAN backups.
  • Oracle instance monitoring with Oracle Enterprise Manager.
  • Backup, Restoration, and Recovery using RMAN &Data migration using Export/Import and Oracle Streams.
  • Index reorganization and database refresh & Oracle Support (SR) process.
  • Remedy Incident, Change, and Service Request tickets
OracleJavaSASDatabase Management

Imr metallurgical resources ag

Big Data Engineer

Sep 2020Jun 2021 · 9 mos · Mumbai, Maharashtra, India · Remote

  • Data extraction, transformation, loading, and integration in data warehouse, operational data stores, and master data
  • management.
  • Processing data in Alteryx to create TDE for tableau reporting.
  • Experience with GCP Big Query for ETL transformations and Spark job development.
  • Analyzing Hadoop cluster and big data analytic tools including MapReduce, Hive, and PySpark.
  • Creation of yaml files for data sources and glue table stack creation.
  • Data validation from SQL Server to Snowflake.
  • Writing PySpark scripts for data quality checks and automation with Apache Airflow.
  • Senior Data Engineer
  • Hadoop cluster installation using various distributions.
  • ELASTIC MAPREDUCE (EMR) and Amazon AWS EC2 instance setup.
  • Hive/SQL queries conversion into Spark RDD and PySpark transformations.
  • Databricks Python notebooks for data processing in Azure data lake storage.
  • Azure cloud and Databricks deployment and configuration.
  • AWS Lambda functions and IAM roles for Python script execution.
  • Snowflake Data Platform Solution Architecture consulting.
  • Unit testing with Junit and automation environment setup.
  • Lambda Deployment function configuration for S3 events and System architecture documentation.
Big DataGCPHadoopPySparkAWSBig Data Engineering

Education

Stony Brook University

Master's degree

Aug 2022Dec 2022

National Institute of Technology Karnataka

Bachelor of Technology - BTech

Aug 2016Aug 2020

Stackforce found 100+ more professionals with Data Engineering & Machine Learning

Explore similar profiles based on matching skills and experience