S

Susmitha Kanagala

Data Engineer

Charlotte, North Carolina, United States0 mo experience

Key Highlights

  • Expert in cloud data engineering with Azure, GCP, and AWS.
  • Proficient in building data pipelines and ETL processes.
  • Strong background in data migration and transformation.
Stackforce AI infers this person is a Data Engineer specializing in cloud data solutions across multiple platforms.

Contact

Skills

Core Skills

Microsoft AzureApache SparkGoogle Cloud Platform (gcp)Azure Data FactorySnowflakeInformatica PowercenterPython

Other Skills

Azure Data LakeAzure Data BricksAmazon Web Services (AWS)ScalaPL/SQLSQLPython (Programming Language)AirflowTalend Open StudioBig DataC (Programming Language)JavaHTML5Azure Databricks

About

Data Engineer with experience in the Information Technology field, includes development and Implementation of various applications in cloud platform includes building data intensive applications, tackling challenging problems, collecting, transforming and sorting data in the banking field. Enjoy creative problem solving and getting exposure on multiple projects, and would excel in the collaborative environment. • Proficient in working with Azure Blob Storage, Azure Data Lake Storage, Azure Data Factory, Azure SQL Data Warehouse, Azure Data Bricks and on Python, SQL and PL/SQL concepts. • Worked on migrating and transforming data from on premise to cloud and between cloud services by creating Azure data factory pipelines & data flows using different ADF activities and components. • Experienced in writing PySpark Applications to connect to different cloud services like Azure SQL DB, Azure Postgres, Azure Synapse Analytics, ADLS, AWS S3 and performing different data transformations based on the business requirements. • Familiar with Azure DevOps to deploy ADF pipelines and ARM Templates into other environments by creating release pipelines. • Worked along with DevOps team to create environment specific Configuration yaml files to deploy code through CI/CD process by creating artifacts using a central repository. • Familiar with Processing of Real-Time Streaming data using Azure Event Hubs and Azure stream Analytics and visualizing the results using Power BI. • Had good knowledge and Hands on with the ETL tool Informatica, Talend to understand the existing flows, to modify the flow and to create the data flow based on the requirement. • Worked in Google Cloud in a source-consumer application using different GCP services like Google Cloud Storage, Cloud Data proc, Big Query, Google Cloud PUB/SUB, Google Cloud Composer etc. • Creation of airflow DAGs using python to orchestrate the data flow using Google cloud services. • Extensively worked with source code management and version control tools like Git, GitHub. • Familiar with Agile way of working, used JIRA to track work progress and Confluence to prepare and manage technical documentations. • Worked with AWS services S3, Redshift, AWS EMR, EC2 and AWS Glue to migrate and to transform data and also to create end to end data flow. • Familiar with components of Hadoop Ecosystem like HDFS, HIVE and HQL. • Knowledge on writing test cases and test plan to validate the data completeness and data accuracy.

Experience

0 mo
Total Experience
--
Average Tenure
--
Current Experience

Ubs

Cloud Data Engineer

Jul 2023Present · 2 yrs 11 mos · Charlotte, North Carolina, United States · Remote

Microsoft AzureAzure Data LakeAzure Data FactoryAzure Data BricksApache SparkAmazon Web Services (AWS)+3

Dun & bradstreet

Data Engineering Sr Analyst

Dec 2021Apr 2023 · 1 yr 4 mos

  • Responsibilities:
  • Understanding data model, business and technical requirements to understand data flow and data processing for source-consumer applications.
  • Using Meta Data Driven architecture, storing and fetching file level metadata from UI using API end points in spark application.
  • Wrote Py-Spark Code for implementing data validations, business rules and transformations in multiple levels in Cloud Storage.
  • Created and scheduled airflow DAGs using python to orchestrate the data flow using airflow operators, custom operators and also connecting to different cloud services.
  • Writing unit test cases, test plan and test strategies to validate the functions and code.
  • Worked in migrating the code to different environments like DEV, QA, UAT and PROD.
Python (Programming Language)Google Cloud Platform (GCP)Apache SparkAirflowSQLAmazon Web Services (AWS)

Ubs

Data Engineer

Sep 2020Oct 2021 · 1 yr 1 mo

  • Responsibilities:
  • Responsible for data migration from on-premise source databases like Oracle, MySQL etc. to target Cloud databases by creating views on the source data, using Cloud Storage as staging area.
  • Involved in testing the Quality of Data and correctness of data migrated by writing PL/SQL and SQL Queries.
  • Used Meta Data Driven architecture to configure file level metadata, transformations and validations of each file for multiple sources.
  • Responsible for Data ingestion from multiple sources and Processing of batch/stream data into target Cloud database using Spark, Data Bricks, Python, Cloud Integration tools, SQL.
  • Responsible for creation of delta tables and worked with delta lake house.
  • Responsible for ADF templates deployment using Azure DevOps and Continuous Integration as part of CI/CD process.
Microsoft AzureAzure Data LakeAmazon Web Services (AWS)Apache SparkSQLAzure Databricks+3

Rbl bank

Data Engineer

Nov 2019Sep 2020 · 10 mos

  • Responsibilities:
  • Responsible for on-premises-to-cloud jobs migration, understanding existing Informatica data flow and transformations.
  • Implementing the jobs in Azure Data Factory using ADF, ADLS, Azure Sql Server and Data bricks.
  • Ingesting data from on-premises Source Database(Oracle/MYSQL) tables to ADLS using Linked services and pieplines in ADF based on requirements.
  • Loading data into target Azure SQL server data warehouse by creating external stages, schemas and tables with proper distributions using Azure Data Factory Pipelines.
  • Wrote SQL, PL/SQL, stored procedures, triggers and cursors for implementing business rules and transformations.
Microsoft AzurePython (Programming Language)Apache SparkSQLAzure DatabricksAzure Data Factory+1

Länsförsäkringar

Data Engineer

Jul 2018Oct 2019 · 1 yr 3 mos

  • Responsibilities:
  • Wrote PL/SQL and SQL stored procedures to fetch the data from on-premise Oracle Data base to generate files.
  • Used python-snowflake connector to connect to snowflake from ADF and created external stages, schemas and tables to store data in Snowflake.
  • wrote PL/SQQL and SQL Scripts, Stored Procedures to perform transformations on the data in ADLS and finally loading to Azure SQL data warehouse by creating external tables and tables in SQL data warehouse.
  • Connected to SQL server from Azure SQL data warehouse using SQL Authentication to run SQL Queries to create tables and to load data.
  • Created pipelines in Azure Data Factory to execute the Stored Procedures, to perform required transformations on data in SQL DW.
Microsoft AzurePython (Programming Language)SQLAzure DatabricksSnowflakeAzure Data Factory+1

Capgemini

Software Developer

Jan 2017Jan 2018 · 1 yr

Python (Programming Language)SQLInformatica PowerCenterTalend Open StudioPython

Education

National Institute of Technology Calicut

Bachelor of Technology - BTech — Computer Science

Sri Chaitanya College of Education

Secondary Education

Stackforce found 100+ more professionals with Microsoft Azure & Apache Spark

Explore similar profiles based on matching skills and experience