Afaque Ahmad

Product Manager

Singapore, Singapore7 yrs 9 mos experience
Highly Stable

Key Highlights

  • Led development of data pipelines processing 50K+ orders daily.
  • Engineered a Customer360 feature store handling over 1TB of data.
  • Automated reporting processes, reducing manual efforts by 95%.
Stackforce AI infers this person is a Data Engineering expert in SaaS with a focus on large-scale data solutions.

Contact

Skills

Core Skills

Data EngineeringData ArchitectureData Quality

Other Skills

AWSAWS Elastic BeanstalkAWS Elastic Container ServiceAWS GlueAWS LambdaAirflowAlgorithm DesignAlgorithmsAmazon CloudWatchAmazon Elastic MapReduce (EMR)Amazon S3Amazon Web Services (AWS)Apache SparkAthenaBig Data

About

I eat codes for breakfast and I'm a data enthusiast. I love solving challenging algorithmic problems related to backend & data engineering bringing over 6 years of experience in this field. I've cracked interviews at Apple, Uber, Atlassian, Databricks As a Senior Data Engineer at QuantumBlack, AI by McKinsey, I collaborate and lead the development of cutting-edge Data & AI solutions for clients across Southeast Asia, focusing on advanced analytics & large-scale data infrastructure. My expertise includes building data lake solutions across AWS and Azure platforms, large scale feature engineering for data science use cases, and development of robust data quality frameworks. Notably, I have engineered data pipelines that seamlessly process over 50K+ orders daily, developed a comprehensive Customer360 feature store on Hadoop handling over ~TBs of data, designed and developed data quality dashboards to detect and report 15+ critical metrics for each dataset Additionally, my passion for knowledge sharing and community contribution is reflected through my active involvement in writing insightful blogs, producing educational videos related to data engineering on my YouTube channel and making significant open-source contributions, particularly for Kedro, as showcased in my Github portfolio Skills: • Programming Languages - Python, Scala, Shell • Processing - MapReduce, Spark • Storage / DWH / Databases - Snowflake, MySQL, Postgres, Hadoop, Hive • Cloud - AWS, Azure • Data Orchestration - Luigi, Airflow, Prefect, Mage • Server Monitoring/Maintenance - Nagios, Icinga2 • Proficient With Git, CI/CD, Docker, Linux Certifications: • AWS Certified Cloud Practitioner • Astronomer Certified Apache Airflow Fundamentals GitHub: https://github.com/afaqueahmad7117 YouTube Channel: https://www.youtube.com/channel/UCYFKQl9VMvtnUHxZ9uFx7qw Contact: afaque.ahmad@theseniorde.com

Experience

7 yrs 9 mos
Total Experience
3 yrs 10 mos
Average Tenure
1 yr 6 mos
Current Experience

Databricks

Solutions Architect, Data & AI

Dec 2024Present · 1 yr 6 mos · Singapore · On-site

  • Leading Large Scale Data Engineering Implementations; Data Migrations; GenAI, RAGs, LLM Engineering; Data Architecture / Best Practices Advisory

Youtube

Content Creator (Data Engineering)

Aug 2023Present · 2 yrs 10 mos

  • Teaching all that I've learnt over the last 7 years; I'm here to make complex data engineering concepts as easy as you could never think it could be

Quantumblack, ai by mckinsey

Principal (Jr.) Data Engineer

Aug 2019Dec 2024 · 5 yrs 4 mos · Singapore · On-site

  • Data Lake Design & Architecture: Led the development of Data Lake on AWS coding 15+ pipelines processing over 50K+ orders sizing several GB daily from diverse data sources powering 3+ data science use cases
  • Large Scale Feature Engineering: Developed a Customer360 Feature Store on Hadoop processing over 1TB of data (100M customers) powering personalization use cases
  • Data Quality Framework: Designed and developed a data quality framework to detect and report 15+ metrics on each dataset from 3 diverse data sources
  • Automated Progress Monitoring: Reduced manual efforts by ~28%, created a real-time pipeline to record progress values, perform aggregations and update PowerBI dashboards for an oil and gas client
  • Ops: Optimized CI/CD pipelines by reducing dependency installation, test duration and using readily available tasks
  • Open Source Contributions: Contributed DeltaTableDataset to Kedro, authored blog on deploying Kedro pipelines on EMR
  • Writer & Data Engineering Advocate: I write on various Data Engineering topics including Apache Spark, Performance Tuning, SQL on LinkedIn, official Kedro Blog (link below)
AWSHadoopData LakeData Quality FrameworkFeature EngineeringData Orchestration+2

Mckinsey & company

Jr. Engagement Manager (JEM)

Aug 2019Dec 2024 · 5 yrs 4 mos · Singapore · On-site

  • Analytics JEM; Empowering clients unlock the potential of Data & AI

Urban company

Data Engineer

Jan 2017Dec 2017 · 11 mos · Gurgaon, India

  • Founding Data Engineer @ UrbanCompany (formerly UrbanClap); ETL Design & Development: Developed and maintained 12+ ETL pipelines extracting data from 5+ sources ingesting into Redshift, processing using pandas, Luigi.
  • Reporting Automation: Automated 12 investor reporting sheets using SQL queries, pandas neatly formatting them using xlsxwriter, eliminating 95% manual efforts.
  • SQL Query Optimization: Optimized and reduced runtime a SQL query of over 600 lines for creating master table from 4.5 hours to 1 hour.
  • Automated Report Mailer: Reduced repetitive efforts of marketing teams by developing an automated mailer that could accept SQL queries, produce reports, and mail the owners at a configurable schedule.
ETLSQLData ProcessingData Engineering

Education

National University of Singapore

Master's Degree — Computer Science

NUS Overseas Colleges

Entrepreneurship Exchange Program — Entrepreneurship/Entrepreneurial Studies

Stackforce found 100+ more professionals with Data Engineering & Data Architecture

Explore similar profiles based on matching skills and experience