B

Biplob Biswas

Software Engineer

Bavaria, Germany11 yrs 10 mos experience
AI ML PractitionerAI Enabled

Key Highlights

  • 10+ years of experience in data architecture and engineering.
  • Expert in building scalable data pipelines and ML platforms.
  • Proven track record of driving revenue through data-driven insights.
Stackforce AI infers this person is a Data Engineering expert with extensive experience in SaaS and Fintech industries.

Contact

Skills

Core Skills

Data EngineeringData ArchitectureMachine LearningCloud Services

Other Skills

AWS AthenaAWS EMRAWS GlueAWS LambdaAWS ServicesAgile MethodologiesAlgorithmsAmazon Web Services (AWS)Apache SparkAurora ServerlessBuilding PerformanceBusiness ConsultingBusiness IntelligenceBusiness RequirementsBusiness Understanding

About

I am a Data enthusiast, having 10+ years of professional experience in architecting data projects, building data pipelines, software development with a keen interest in intriguing use cases involving huge amounts of data in industries. Successfully completed and obtained a Master's Degree in Business Intelligence (MS in IT4BI) having acquired Category A Erasmus Mundus Scholarship provided by the European Commission. Interests: ML Platforms, Architecting Data pipelines, Big Data Processing, Cloud Services, Predictive Analytics, Streaming Data Analysis, Data Mining, Machine Learning

Experience

11 yrs 10 mos
Total Experience
1 yr 11 mos
Average Tenure
1 yr 7 mos
Current Experience

Holidu

Staff Data Engineer

Nov 2024Present · 1 yr 7 mos · Munich, Bavaria, Germany · Hybrid

Stakeholder EngagementData TransformationPySparkData ManagementData CollectionData Infrastructure+6

Thoughtworks

Lead Data and ML Engineer

Sep 2021Nov 2024 · 3 yrs 2 mos · Munich, Bavaria, Germany · Hybrid

  • Architected enterprise-scale ML platform using Kubeflow on Kubernetes, enabling 10+ teams to implement CD4ML pipelines with automated model deployment
  • Orchestrating successful delivery of 3 production ML use cases, driving initiatives to achieve 20% reduction in use case-specific operational overhead by 2024
  • Pioneered Generative AI capabilities by implementing open-source model hosting and RAG architecture using in-memory vector databases, empowering teams with production-ready LLM solutions
  • Tools & Technologies:
  • Kubernetes, Kubeflow, KServe, Terraform, Python, AWS Services (IAM, Glue, S3, EMR on EKS, SageMaker, Athena), Generative AI (LLMs, Multimodal Images), RAG architectures
Amazon Web Services (AWS)Business UnderstandingData StrategiesStakeholder EngagementRelationship DevelopmentData Transformation+23

Burdaforward

2 roles

Senior Data Engineer

Aug 2019Aug 2021 · 2 yrs

  • Engineered scalable ETL pipeline processing 150+ GB daily data across multiple sources, unifying user interactions, impressions, and revenue data
  • Unified siloed data systems by implementing modern data mesh architecture using Snowflake and DBT, enabling cross-functional content analytics
  • Drove 12% increase in annual revenue through automated content performance analysis, identifying high-value article types and engagement patterns
  • Tools & Technologies:
  • AWS Athena, AWS Glue, AWS EMR, MWAA(Managed Airflow), Apache Spark, Snowflake, DBT, Python, Terraform, Gitlab CI
Amazon Web Services (AWS)Business UnderstandingData StrategiesStakeholder EngagementRelationship DevelopmentData Transformation+19

Data Engineer

Oct 2018Jul 2019 · 9 mos

  • Architected and implemented serverless AWS infrastructure using CloudFormation, processing 10M+ daily user interactions and 100K+ news articles for recommendation engine
  • Optimized cloud costs by 35% through event-driven processing and intelligent S3 lifecycle policies
  • Designed scalable data pipeline using AWS services (Lambda, Kinesis, DynamoDB, Aurora Serverless, Redis) enabling data scientists to reduce model training time by 40%
  • Tools & Technologies:
  • AWS cloud, Lambda, Kinesis, Apigateway, Elasticache, Dynamodb, Firehose, EMR, Python3, Scala, YAML
Amazon Web Services (AWS)Business UnderstandingData StrategiesData TransformationCoding StandardsPySpark+14

Airplus international

Big Data Engineer - Consulting

Feb 2017Sep 2018 · 1 yr 7 mos · Neu-Isenburg

  • Architected and delivered a scalable data pipeline using Apache Spark and Kafka, replacing legacy monolithic system, reducing processing time by 35%
  • Built mission-critical ETL workflows for data enrichment and complex business logic implementation, processing huge amount of daily transaction data
  • Integrated COTS solutions with the Hadoop ecosystem, providing data to these systems in near-real time
  • Tools & Technologies:
  • Java, Apache Spark, Kafka, Hadoop (HDFS, YARN), Solr, HBase, Phoenix, Nifi, Flink, Oozie, HDP 2.5, CDH 5.13, Azure Cloud
Business UnderstandingData StrategiesData TransformationPySparkIT ConsultingData Collection+10

Qimia gmbh

Big Data Engineer

Oct 2016Sep 2018 · 1 yr 11 mos · Sankt Augustin

  • Working on the Big Data Technology stack which includes Spark, Hive, Impala, Sqoop, Kakfa and Flume.
  • Setting up of virtual hadoop/CDH cluster using Vagrant and Ansible.
Data TransformationPySparkData IntegrationContinuous Integration and Continuous Delivery (CI/CD)Apache SparkBusiness Consulting+3

Berlin big data center

Student Assistant

Mar 2016Sep 2016 · 6 mos · Berlin Area, Germany

  • Working on distributing big datasets into small manageable chunks using Python and later, would be testing them on scalable systems like Flink and Spark to run machine learning tasks.

Aricent group

Software Engineer

Oct 2012Sep 2014 · 1 yr 11 mos · Gurgaon, India

  • Underwent 3 months of extensive training in C, Data Structures, Unix, and, System Programming. Also basics of Telecom and Networking.
  • Worked for Multinational data networking and telecommunications equipment company, building and maintaining features used in PCU(Packet Control Unit) for 2G networks

Mathematical sciences foundation

Information Technology Intern

Jun 2010Jun 2010 · 0 mo · The Air Force School, Subroto Park

  • Our Task was to study about the structure of .bat files, So as to infect them and then device a way to restore the original file.

Education

Technische Universität Berlin

Master’s Degree — Business Intelligence

Jan 2015Jan 2016

Université de Tours

Master's Degree — Business Intelligence

Jan 2014Jan 2015

Université libre de Bruxelles

Master's Degree — Business Intelligence

Jan 2014Jan 2015

Jamia Millia Islamia

Bachelor in Technology (B.Tech) — Computers

Jan 2008Jan 2012

Raisina Bengali School, C.R.Park

Senior Secondary — Science

Jan 1992Jan 2007

Stackforce found 100+ more professionals with Data Engineering & Data Architecture

Explore similar profiles based on matching skills and experience