Durga Prasad Choudhury

Product Manager

Pathapatnam, Andhra Pradesh, India3 yrs 10 mos experience

Highly Stable

Key Highlights

Over 4 years of experience in cloud-native data solutions.
Expertise in AWS, Azure, and GCP with certifications.
Proven track record in building scalable data architectures.

Stackforce AI infers this person is a Data Engineer specializing in cloud-based data solutions across multiple platforms.

Contact

Skills

Core Skills

Data EngineeringData AnalyticsCyber Security

Other Skills

ADLAWSAWS GlueAWS Identity and Access Management (AWS IAM)AWS LambdaAWS Step FunctionsActive DirectoryAd ServingAirflowAmazon AthenaAmazon QuickSightAmazon RedshiftAmazon S3Amazon Web Services (AWS)Analytics

About

Results-oriented Data Engineer with over 4 years of experience delivering scalable, cloud-native data solutions across AWS, Microsoft Azure, and Google Cloud Platform (GCP). Certified in all three platforms, I specialize in PySpark, DBT, Snowflake, Python, and SQL to design, build, and optimize data pipelines that drive real-time analytics and data-driven decisions. I excel in end-to-end pipeline development, from data ingestion and transformation to validation and optimization, ensuring data quality, performance, and scalability. With strong data modeling expertise and a focus on business impact, I am committed to building efficient, high-performance data architectures. With a proven track record of managing complex workflows and enabling actionable insights, I help organizations leverage data for smarter, faster decision-making.

Experience

3 yrs 10 mos

Total Experience

1 yr 3 mos

Average Tenure

Current Experience

Softborne technology solutions pte. ltd.

2 roles

Analytical Data Engineer II

Promoted

Sep 2023 – Sep 2025 · 2 yrs · On-site

Collaborated with the product management team to identify and prioritize new feature opportunities, leading to timely implementation.
Designed and optimized scalable data models in Reactor Data using Snowflake and BigQuery, improving query performance and reducing processing time.
Developed and maintained real-time and batch data pipelines using BigQuery, Dataflow, Pub/Sub, and Cloud Composer to support data availability in dashboards and analytics.
Built Python-based transformations and leveraged Apache Beam and Spark for large-scale data processing.
Used Source Links to join and map multiple data sources into unified schemas.
Tuned complex SQL queries in Snowflake and BigQuery to optimize performance and manage large-scale retail datasets.
Worked closely with cross-functional teams to implement tailored data solutions.
Conducted thorough validation and reconciliation checks between source systems and data warehouse to ensure 99%+ data accuracy.
Built custom RESTful API connectors using Airbyte for ingesting external data sources.
Deployed and managed connectors using Cloud Functions and Cloud Run for scalable and serverless execution.
Developed dynamic dashboards to track KPIs such as sales performance and customer lifetime value.
Created and maintained transformation logic using DBT, promoting modular, testable, and reusable data models.

Data PipelinesData MiningPython (Programming Language)Database Management System (DBMS)SnowflakeApache Kafka+21

Data Engineer II

Sep 2023 – Sep 2025 · 2 yrs · On-site

Cross-team CollaborationData Architects

Career break

Caregiving

Jul 2023 – Aug 2023 · 1 mo · Pathapatnam, Andhra Pradesh

Kpi partners

Data Engineer I

Jul 2022 – Jul 2023 · 1 yr · Hyderabad, Telangana, India · Hybrid

Created data pipeline using Apache Airflow, Python, and AWS to store weekly raw data.
Designed a multi-user, multi-database app for processing thousands of records.
Managed SQL and NoSQL data modelling using Big Query, Snowflake, and Amazon DynamoDB for scalable data collection and structured storage.
Collaborated with cross-functional teams for pipeline design and business needs.
Developed scalable data ingestion processes and optimized ETL workflows.
Implemented data quality checks and monitoring mechanisms for data integrity.

Data PipelinesAmazon RedshiftData MiningPython (Programming Language)PySparkDatabase Management System (DBMS)+23

Dxc technology

Associate Professional Software Engineer

Aug 2021 – Jun 2022 · 10 mos · Bangalore Urban, Karnataka, India · Remote

Developed and maintained scalable data pipelines using Azure Data Factory and Databricks.
Optimized data storage and retrieval using Azure Synapse Analytics and Delta Lake.
Implemented data security best practices with Azure Key Vault and Role-Based Access Control (RBAC).
Monitored and troubleshot data pipeline failures using Azure Monitor and Log Analytics.
Designed and implemented data models for efficient querying and reporting.
Automated ETL processes to streamline data ingestion and transformation.
Collaborated with data analysts and stakeholders to ensure data accuracy and consistency.
Conducted performance tuning of SQL queries and Spark jobs for optimal execution.

Data PipelinesAmazon Redshiftazure synapseData MiningDatabase Management System (DBMS)Active Directory+27

Cyber crime cell, gurugram police

Cyber Security Intern

Jun 2021 – Jul 2021 · 1 mo · Gurugram, Haryana, India · Remote

• Provide first-level compliance monitoring and investigations. Assist with forensics analysis and fact gathering. Assist with vulnerability assessments and penetration testing for specific applications, services, networks, and servers as required.

Cybercrime InvestigationCyberlawPenetration TestingCyber DefenseProgrammingCyber Threat Hunting (CTH)+5

Google

Google Cloud Ready Facilitator

Apr 2021 – Jun 2021 · 2 mos · India · Remote

The Google Cloud Ready Facilitator program will introduce you to Computing, Application Development, Big Data & Machine Learning using Google Cloud's training platform called Qwiklabs.
I learned each of these topics using self-paced labs that provide you temporary credentials to Google Cloud Platform, so you can learn the cloud using the Real thing – no simulations.

Data MiningDatabase Management System (DBMS)Google Cloud Platform (GCP)Statistical Data AnalysisSQLGoogle Analytics+5

Dxc technology

Project Trainee

Feb 2021 – Apr 2021 · 2 mos · Bengaluru, Karnataka, India · Remote

The project aims to analyze and identify the data quality issues and strategies to mitigate these Issues in the given data. We need to start with a PowerPoint presentation that outlines the approach that we will be taking.
The client has agreed on a 3-week scope with the following 3 phases as follows - Data Exploration: Model Development