P

Pawan Kumar

Senior Software Engineer

Bengaluru, Karnataka, India11 yrs 6 mos experience
Most Likely To Switch

Key Highlights

  • Expert in building scalable data pipelines and frameworks.
  • Proven track record in mentoring junior engineers.
  • Strong background in cybersecurity and data governance.
Stackforce AI infers this person is a SaaS and Cybersecurity expert with strong capabilities in data engineering and software development.

Contact

Skills

Core Skills

Amazon Web Services (aws)PythonApache IcebergAws GlueJavaSpring BootMicrosoft Sql ServerJavascriptPublic SpeakingAlgorithms

Other Skills

.NET FrameworkAirflowAngularApache AirflowApache DruidApache KafkaApache SparkApache TrinoBig DataCC#C++Computational ComplexityData StructuresDatabases

About

As a Senior Engineer in Data Platform at Sophos, I have been involved in creating a new Apache Iceberg-based data pipeline for our data lake. Before this, my primary focus involved creating a new batch detection framework using the data available in the data lake to help the threat hunters detect potentially customer-compromising detections before they happen. I have used my skills in AWS, Java, Spring, Python, Relational, and Non-relational databases to deliver high-quality solutions that increase transparency, accountability, and efficiency for our clients. Additionally, I have been mentoring 5+ junior engineers in software development best practices and enhancing their technical competencies. Before joining Sophos, I worked at o9 Solutions, Goldman Sachs, Eureka AI, and Samsung Research Institute, where I gained valuable experience in developing proprietary databases from scratch, building web applications, handling data pipelines in a large-scale system, building desktop applications, building user tracing systems based on their constantly changing geography, and building UX using cutting-edge technologies. I have a Master's degree in Computer Science and Engineering from IIT Kanpur and a Bachelor's degree in the same field from IIT Guwahati. I am passionate about exploring new horizons, taking on new challenges, and solving intricate problems in the software engineering world.

Experience

Sophos

Senior Software Engineer - II

Oct 2023Present · 2 yrs 5 mos · Bangalore Urban, Karnataka, India · Hybrid

  • At Sophos, I serve as a Senior Engineer within the Data Platform team, tasked with overseeing the end-to-end lifecycle of data within our AWS-based data lake infrastructure. My responsibilities encompass the comprehensive management of data ingestion, storage, processing, and retrieval processes on both Hive as well as Apache Iceberg-based table formats. In addition, I built the batch processing and ETL (Extract, Transform, and Load) framework, using which the threat hunters can detect any compromising situations before they happen. We achieve this using tools/technology like Java, Spring Boot, Python, Airflow, Kafka, Various AWS services, Terraform, Metabase, Trino, Iceberg, Grafana, and so on.
  • Additionally, I have been working as a team lead while mentoring 5+ junior engineers to enhance team productivity.
Amazon Web Services (AWS)PythonApache KafkaSpring Boot

O9 solutions, inc.

Senior Software Engineer - I

Aug 2021Oct 2023 · 2 yrs 2 mos · Bengaluru, Karnataka, India · Hybrid

  • Built a comprehensive auditing framework for o9’s database, improving transparency, accountability, and data integrity across all transactions while cutting transaction tracking time by 90%, enabling faster decision-making.
  • Delivered key customer-driven enhancements such as lightweight and private scenario sharing, directly improving client usability and engagement. Proactively resolved platform bugs before release deadlines, ensuring smooth deployments and higher customer satisfaction.
  • Strengthened data governance by introducing measure-based and time-based ACL frameworks, allowing dynamic, context-driven access control. These innovations enhanced security, compliance, and flexibility in enterprise-scale data operations.
  • Designed and implemented a centralized service health monitoring system that provided real-time visibility into the performance and configuration of Java-based microservices. This solution improved operational reliability and boosted SLA compliance by over 20%, reducing downtime and accelerating root-cause resolution.
JavaMicrosoft SQL ServerREST APIsHBaseSpring BootHadoop+1

Goldman sachs

Software Associate

Jan 2021Jul 2021 · 6 mos · Bangalore Urban, Karnataka, India

  • Undertook the task of augmenting LUMOS, an internal desktop application used by GS. The primary objective was to centralize open workflows, tickets, and notifications for all GS employees within a unified and intuitive desktop app. I added the feature of adding message boards, outage notifications, and getting the least TAT on any open tickets by the support team. This led to improvement in efficiency by over 30%.
JavaJavaScriptNode.jsElectron.jsSpring BootAngular

Eureka ai

2 roles

Senior Software Engineer

Promoted

Jan 2020Jan 2021 · 1 yr · On-site

  • Contact Tracing: Developed a user tracing system utilizing Telco Data to identify potential sources of Covid-19 infection with user consent. Incorporated the use of Point of Interest data to jog the user's memory and identify potential infection sites. Our solution was used by a few governments to track and minimize the infection resulting from Covid-19.
  • Eureka Omni: Developed a versatile tool called Omni, which enables the creation of customized dashboards using pre-existing or new widgets to generate reports in diverse formats. The solution provides a streamlined approach to report creation and empowers clients to effortlessly share and disseminate information with stakeholders.
JavaJavaScriptElasticsearchHiveReact.jsREST APIs+1

Software Engineer

Jun 2018Dec 2019 · 1 yr 6 mos · On-site

  • Eureka User Profile Store: Designed and developed the UI and backend of the Eureka User Profile Store utilizing RESTful APIs. The platform allowed for exploration of Telco Users and extraction of specific user sections using multiple filters. Additionally, Implemented a suite of data visualization tools that dynamically responded to selected filters, empowering clients to generate intelligence reports for their target users. The success of this initiative resulted in 6 telcos across 5 countries contracting our services.
  • Campaign Management System: Designed and implemented a comprehensive system enabling Campaign Directors to create and oversee targeted campaigns for specific customer segments. One of the key challenges of this project was effectively matching the optimal users for each campaign during simultaneous runs, a challenge successfully addressed through innovative solutions and advanced algorithms.
  • Credit Score: Developed an end-to-end system capable of seamlessly ingesting credit score data from the Eureka AI engine in Hive to Elasticsearch. The solution also features the ability for clients to retrieve user credit scores via mobile number or national ID, providing a user-friendly and efficient means of accessing critical information.
  • Built an advanced data transformation tool, "pipecore", capable of automating the seamless integration of data from various Telcos into our proprietary internal format, optimized for use with AI engines. This tool also facilitated efficient transfer of data between databases, including but not limited to Hive and Elasticsearch, ensuring smooth and uninterrupted data flow across our systems, and hence enhancing our data pipeline efficiency by over 3x.
JavaJavaScriptElasticsearchReact.jsPythonApache Druid

Indian institute of technology, kanpur

Graduate Teaching Assistant

Jul 2016May 2018 · 1 yr 10 mos · Kanpur Area, India

  • Served as a Teaching Assistant for the Computer Science Department at IIT Kanpur, where I provided academic support to students. Additionally, contributed my expertise as a Teaching Assistant for the Design and Analysis of Algorithms course, facilitating student learning and academic success.
JavaJavaScriptPythonREST APIs

Samsung electronics

2 roles

Senior Software Engineer

Mar 2016Jul 2016 · 4 mos · On-site

  • Implemented a significant upgrade to the Samsung Email system by introducing an innovative IFTT (If this then that) feature. The primary goal of this upgrade was to achieve a zero Inbox by automatically generating events, tasks, and reminders based on email content. Additionally, incorporated a priority inbox capability utilizing advanced information retrieval techniques to enhance user productivity and streamline email management.
Public SpeakingAlgorithmsData Structures

Software Engineer

Jun 2014Mar 2016 · 1 yr 9 mos · On-site

  • Engineered a robust and scalable billing system that enables system administrators to efficiently monitor call, SMS, and data usage for both personal and enterprise accounts through a unified dashboard. As a result, this initiative yielded significant cost savings for the company by mitigating expenses related to personal usage.
  • Led the development of a Google Drive add-on for Samsung Email, allowing mail attachments to be directly integrated with Google Drive. This comprehensive solution improved email management efficiency, enhancing overall productivity while ensuring data integrity and accessibility.
JavaJavaScriptPython

Education

Indian Institute of Technology, Kanpur

Master of Science - MS — Computer Science

Jul 2016May 2018

Indian Institute of Technology, Guwahati

Bachelor of Technology - BTech — Computer Science

Jul 2010Apr 2014

Mother's International Academy, Patna

Higher Secondary School

Apr 2008Mar 2010

Brilliant Public School

High School

Jan 2005Jan 2008

Stackforce found 100+ more professionals with Amazon Web Services (aws) & Python

Explore similar profiles based on matching skills and experience