Shiv Konar

Product Engineer

United Kingdom13 yrs 3 mos experience

Most Likely To SwitchHighly Stable

Key Highlights

Architected real-time data solutions for 19 million customers.
Delivered £5M in annual savings through engineering optimisations.
Led cloud migrations enhancing data processing efficiency.

Stackforce AI infers this person is a Fintech Data Engineering expert with a focus on real-time analytics and cloud-native solutions.

Contact

Skills

Core Skills

Data EngineeringReal-time Data ProcessingUser Engagement AnalyticsCloud Data MigrationData Lake ArchitectureBusiness Intelligence DevelopmentData Warehousing

Other Skills

AWS Data EngineeringAWS LambdaAgile Team LeadershipAmazon Web Services (AWS)Apache Airflow OrchestrationApache KafkaAuto-Scaling and Resource OptimisationBig DataCampaign Performance AnalyticsCloud Data Migration (Hadoop and AWS to Snowflake)Cloud-Native Engineering (AWS)Cost Optimisation and Vendor ManagementCustomer Experience EnhancementDashboard Design with Power BI and TableauData Architecture

About

✭ I’m a hands-on Data Engineering Lead specialising in Streaming Solutions who loves the intersection of data, cloud, and innovation. Over a decade, I’ve architected and scaled high-performance platforms that power real-time insight, cut costs, and improve user journeys—lately at NatWest, supporting 19 million+ retail customers. ✭ From modernising legacy Hadoop clusters to pioneering streaming engines that classify hundreds of thousands of transactions every minute, I turn messy data challenges into elegant, scalable systems. I build AWS-centric pipelines, lead cloud migrations, and coach cross-functional teams to deliver value fast, always with measurable improvements. ✭ Comfortable across Python, Scala, Spark, Kafka, Airflow, and Snowflake, I pair deep technical know-how with a strong sense of data governance and security. My goal is simple: deliver faster insights, stronger decisions, and better experiences. If you’re building something ambitious with data, let’s connect and explore possibilities.

Experience

Natwest group

4 roles

Data Engineering Lead

Promoted

Jan 2025 – Present · 1 yr 2 mos · London Area, United Kingdom

I’m leading the build of a near-real-time engine that classifies every UK debit-card transaction for 19 million customers. By shifting us from overnight batches to live Kafka-based streams, I’m boosting fraud alerts and UX while setting the foundation for a new, cloud-native data platform.
Key Achievements:
Achieved rapid classification of over 200,000 debit card transactions in under one minute, enabling quicker analytics and informed decisions.
Directed a real-time streaming initiative, driving innovation in transaction processing and reducing time-to-insight for digital banking.
Enhanced fraud detection and real-time engagement by delivering timely insights like overspending alerts and suspicious activity within a 5-minute SLA.

LakehouseScala SparkFraud Detection and PreventionCustomer Experience EnhancementData EngineeringPySpark+11

Lead Data Engineer

Oct 2023 – Jan 2025 · 1 yr 3 mos · London Area, United Kingdom

I built an in-house transaction categorisation and insights platform, cutting our dependence on costly third-party tools and giving retail banking complete control of its data. Alongside the build, I coached junior engineers through our modernisation roadmap, trimming technical debt and tuning the system for peak performance.
Key Achievements:
Boosted mobile app engagement by designing custom features for business banking and commercial clients.
Enhanced processing speed and improved efficiency by launching a near real-time streaming engine for non-booked transactions.
Delivered £5M in annual savings, tripled feature rollout speed, and increased user engagement by 30% through targeted engineering and optimisation.

Big DataUser Engagement AnalyticsAmazon Web Services (AWS)LakehousePerformance OptimisationScala Spark+13

Senior Data Engineer

Promoted

Jun 2022 – Oct 2023 · 1 yr 4 mos · London Area, United Kingdom

I built and maintained data workflows using Scala, PySpark, and Airflow to simplify processing and job control. I led a major data migration from Hadoop and S3 to Snowflake, boosting speed and accuracy. I also set up AWS monitoring tools to improve system visibility and incident response across the board.
Key Achievements:
Scaled system capacity and reliability by architecting AWS data pipelines that process 80 M+ daily transactions for 20 M customers.
Minimised resource waste and boosted processing efficiency through auto-scaling policies and right-sized AWS EMR clusters.
Elevated platform utilisation by 40 % and enhanced production health monitoring by 75 %, securing higher availability and resilience.

Scala SparkAuto-Scaling and Resource OptimisationData EngineeringMonitoring and Operational ResilienceScala and PySpark DevelopmentAWS Data Engineering+5

Data Engineer

Jul 2019 – Jun 2022 · 2 yrs 11 mos · London Area, United Kingdom

I built Spark-Scala data jobs to bring scattered data sources together in an AWS data lake, making storage cleaner and faster. I used near real-time Spark Streaming with AWS Kinesis to feed DynamoDB and S3, and teamed up with analysts to create Tableau dashboards that surfaced key insights and issues.
Key Achievements:
Implemented data security policies using Apache Ranger to ensure compliance and enforce authorised data access.
Deployed eight data integration jobs to production, advancing analytics capabilities and meeting privacy requirements through secure accessibility.
Managed setup and upkeep of Hive and AWS Athena tables, schemas, and views to enable smooth querying and reporting.

Scala SparkData Lake Architecture and IntegrationData EngineeringData Security and Compliance (Apache Ranger)Spark and Scala DevelopmentHive and Athena Table Management+3

Neo@ogilvy

2 roles

Business Intelligence Developer

Promoted

Apr 2014 – Jun 2019 · 5 yrs 2 mos · London Area, United Kingdom

I led the creation of BI reports and dashboards using Power BI and Tableau to help teams make smarter decisions. I also built and maintained end-to-end data pipelines on GCP, making sure data flowed smoothly and reliably across systems to support real-time insights and business performance.
Key Achievements:
Created over 80 real-time campaign performance dashboards for more than 10 media clients, empowering timely insights and data-driven optimisation.
Maintained consistent service delivery by supporting client reporting on a legacy SQL Server data warehouse while leading the migration to Hadoop architecture.

Big DataCampaign Performance AnalyticsBusiness Intelligence DevelopmentGoogle Cloud Platform (GCP) Data EngineeringData IntegrationPython (Programming Language)+4

Data Warehouse Developer

Dec 2012 – Apr 2014 · 1 yr 4 mos · London Area, United Kingdom

I overhauled our data warehouse with Kimball best practices, making it cleaner, faster, and easier to scale. I then built and launched multi-dimensional databases that power SSAS cubes and data-mining models, giving analysts richer, quicker insights and a foundation that’s straightforward to maintain.
Key Achievements:
Improved query performance and user experience by optimising database and dashboard configurations for high efficiency.
Upgraded the data warehouse architecture to support more efficient data storage and accelerate retrieval across business intelligence tools.

Kimball Data Warehousing MethodologyBusiness Intelligence DevelopmentMultidimensional Database DesignData Mining and AnalyticsDatabase Performance TuningData Warehousing