Soumen Ghosh

Software Engineer

Bengaluru, Karnataka, India27 yrs 11 mos experience
Highly Stable

Key Highlights

  • 27 years of industry experience in software architecture.
  • Expertise in Big Data and Analytics solutions.
  • Led migration projects for large-scale data systems.
Stackforce AI infers this person is a Cloud Computing and Big Data expert with extensive experience in enterprise solutions.

Contact

Skills

Core Skills

Cloud ComputingBig DataSecurityIdentity ManagementSoftware ArchitectureE-commerceSoftware DevelopmentData ScienceAdvertising TechnologyTelecommunicationsData AnalysisEmbedded SystemsProject ManagementClient ServicesSearch Technologies

Other Skills

ASN.1 ProtocolsAgile MethodologiesAlgorithmsCosmosDBData MigrationDatabricksDelta LakeDigital Set Top BoxesDistributed SystemsDockerETL FrameworkEnterprise Service BusFraud ManagementFulfilment Management SystemH.323

About

Software Architect, with background in Big Data and Data Migration, Analytics, Distributed Computing and Security, having more than 27 years of industry experience. Specialties: Design, Implementation, Problem Solving, Solution Architecting and Optimization. Strong background in technical fundementals. Strong background in Big Data and Analytics. Has background in data mining and machine learning. Some of my key skills are: Distributed, Scalable, Redundant and Highly available Systems, Spark and Scala, Databricks, SSO and ID Federation, OpenBanking, CDR, Supply Chain and eCommerce, Cloud, Big Data and Analytics, Messaging Frameworks, Software Architecture, Java, Scala, C/C++, JAX-RS, REST, Hadoop, Kafka, PIG, HIVE, Data Mining and Machine Learning, ETL, PL/SQL, PERL, Docker, Telecommunication Data Retrieval and Analysis, Domain Specific Language design and Implementation using ANTLR and PERL, IP Based Telephony (SIP and H.323), Enterprise Systems (3 tier systems), Processing of Large Stream Of Data, Unix/Linux systems programming, Performance analysis and improvements. A brief snapshot of my experience includes: More than 27 years of experience in software design and implementation, More than 8 years experience in Analytics, More than 3 years in Spark/Synapse with CosmosDB, More than 2 years experience in SSO and IDFederation, More than 2 years of experience in Supply Chain, More than 4 years of experience in hadoop, More than 15 years of experience in JAVA, More than 4 years of experience in Cloud, More than 4 years of experience in telecommunication domain, More than 2 years of experience in enterprise management, 3 years of experience in financial software development and maintenance, More than 4 years of experience in internet telephony, More than 2 years of experience in customized search engine development, More than 4 years of experience in ETL and data parsers ,More than 3 years of experience in developing Domain Specific Languages, More than 1 year of experience in writing CAD tools

Experience

27 yrs 11 mos
Total Experience
2 yrs 9 mos
Average Tenure
--
Current Experience

Microsoft

Principal Engineer

Feb 2021Oct 2025 · 4 yrs 8 mos · Hybrid

  • Soumen worked on Synapse and Data Bricks platforms to facilitate analytics on customer data stored in CosmosDB. Soumen added a Merge On Read feature to open source delta lake to support small, local edits. This effort improved delta lake CRUD performance by upto 30% on some queries. Also took active part in speaking to stakeholders like Customers and Product Managers to understand new features, came up with designs and take part in implementations. His focus areas were Delta Lake, spark, Synapse and Databricks.
  • Soumen currently works on Migration charter for CosmosDB. This involves building tools for large scale data movements into CosmosDB from Customer's existing database. There are different tools / technologies available for this, both from MS and outside. The challenge is to understand customer requirements, both around performance and security, and come up with right solutions / tools to faclilitate migration. Soumen is the main technical leader of the migration teams and works on all new challenges posed to the team.
SparkDatabricksCosmosDBDelta LakeCloud ComputingBig Data

Intuit

Principal Engineer

Jun 2019Feb 2021 · 1 yr 8 mos · Greater Bengaluru Area · On-site

  • Soumen worked on federation of user Identity into and out of Intuit, supporting SSO for both outside parties and Intuit application and partners. This helped Intuit to integrate newly acquired entities safely and quickly into Intuit framework. He also worked on facilitating movement of financial data of extremely sensitive nature into and out of Intuit. He worked on cutting edge authentication/authorization protocols like OAuth2.0, OpenIDConnect, SAML. Also worked on enabling Intuit to meet various banking standards like OpenBanking, Australia CDR and PSD2.
SSOOAuth2.0OpenIDConnectSAMLSecurityIdentity Management

@walmartlabs india

Principal Engineer

Apr 2016Jun 2019 · 3 yrs 2 mos · Bengaluru, Karnataka, India · On-site

  • Soumen designed and helped implementing the next generation fulfilment management system for Walmart e-commerce. This replaced the legacy Stirling based fulfilment system and was distribute, reliable and elastic. Also worked on Walmart's enterprise service bus and authorization services.
Fulfilment Management SystemEnterprise Service BusSoftware ArchitectureE-commerce

Oracle

Consulting Member of Technical Staff

Aug 2013Mar 2016 · 2 yrs 7 mos · Bangalore · On-site

  • Soumen worked for the Extensibility team of Enterprise Management (EM) Software. Was deeply involved in design and implementation of various Extensibility platform microservices on SAAS platform for EM. Designed high capacity data ingestion microservice for loading configuration and metric data in platform.
  • Worked on SmallFile support in data ingestion service on HDFS, Platform wide metadata service, Generic Entity Model and other microservices. Worked on proof of concept of scalable deployment using Docker. Individual responsibility also includes mentoring people for various technical roles and conducting periodic training sessions.
  • Worked on entity search platform to handle metric uploaded by large number of entities, and log analytics and reporting using Hadoop.
MicroservicesHDFSDockerSoftware DevelopmentCloud Computing

Yahoo!

Principal Engineer

May 2010Aug 2013 · 3 yrs 3 mos · Bangalore · On-site

  • Soumen Worked on implementing various Machine Learning algorithms for user behaviour prediction. The models predict click and conversion probabilities for Y! RMX Ad Exchange. It involved crunching huge amount of data in Hadoop (Map-Reduce and PIG) as well as building models. The models used were: Decision tree for click and conversion prediction and Expectation Maximization correction modules, Maximum Entropy Models and Random Forest. Also experimented with various other models.
  • Platform: Hadoop
  • Language: Java, PERL, PIG
  • Technologies: HBASE
Machine LearningHadoopJavaData ScienceAdvertising Technology

Connectivasystems

Principle Member of Technical Staff - II

Aug 2005May 2010 · 4 yrs 9 mos · Caluctta

  • Was involved in Architecting, Mentoring, Developing and Researching in Onereview RA and Fraud management software. The areas worked on includes File based stream analysis, Mathematical Models for data analysis, Test Call Generator, Rule Based Fraud, Workflow engine, ETL Framework, Adapter Workbench and Search. Worked on time based aggregation of CDRs to uncover hidden patterns like number having only outgoing and no incoming calls, outgoing calls exceeding a limit in a specific amount of time, numbers staying at a single cell tower for more than a specific amount of time etc. Had very good exposure to telecom data analytics.
  • Languages Used: Java, C++
Fraud ManagementETL FrameworkTelecommunicationsData Analysis

Alumnus software limited

Project Manager

Apr 2005Jul 2005 · 3 mos

  • Managed development of digital set top boxes. Developed a couple of Linux USB device drivers as well.
Digital Set Top BoxesLinux USB Device DriversEmbedded SystemsProject Management

Cognizant technology solutions

Senior Associate / Poject Lead

Jun 2002Apr 2005 · 2 yrs 10 mos

  • Involved in the TECS client services front end for JPMorgan Chase bank. Work involved maintaining/augmenting the SmallTalk UI, Tuxedo/RogueWave middleware and eventually porting the UI to javascript. Mastered SmallTalk in a short space of time and helped others understand.
SmallTalkJavaScriptSoftware DevelopmentClient Services

Astralsys software limited

Project Leader

Dec 2000May 2002 · 1 yr 5 mos · Kolkata, West Bengal, India · On-site

  • Designed and build VOIP products based on H.323 and SIP protocol.
VOIPH.323SIPTelecommunicationsSoftware Development

Intarka inc.

Senior Software Engineer

May 1999Dec 2000 · 1 yr 7 mos · Calcutta · On-site

  • Worked as a developer for a search product with personalized experience. Worked on VC++ and Java, on things like Web Crawling and Natural Language Processing
Web CrawlingNatural Language ProcessingSearch TechnologiesSoftware Development

Motorola india electronics limited

Senior Software Engineer

Jan 1996Jan 1998 · 2 yrs · Kolkata, West Bengal, India · On-site

  • Worked on Iridium Projects initially - developed tools to encode/decode ASN.1 message protocols. Moved on to CAD-VLSI and worked on optimization of logical circuits with transistor sizing and also on various search problems on VLSI floorplan.
ASN.1 ProtocolsVLSI OptimizationEmbedded SystemsSoftware Development

Education

Indian Statistical Institute, Kolkata

M Tech — Computer Science

Jan 1994Jan 1996

Jadavpur University

BE — Electronics and Telecommunication Engineering

Jan 1990Jan 1994

Narendrapur Ramkrishna Mission Residential College

Jan 1988Jan 1990

Stackforce found 100+ more professionals with Cloud Computing & Big Data

Explore similar profiles based on matching skills and experience