krishna khadka

Data Engineer

Bengaluru, Karnataka, India5 yrs 11 mos experience

AI ML PractitionerHighly Stable

Key Highlights

Expert in building scalable data platforms on Databricks and AWS.
Proficient in optimizing Spark and SQL workloads for performance.
Strong background in data governance and quality frameworks.

Stackforce AI infers this person is a Data Engineering expert in SaaS environments with a focus on cloud solutions.

Contact

Skills

Core Skills

Data EngineeringData ArchitectureSoftware EngineeringCloud Computing

Other Skills

AWSAlgorithm DesignAlgorithmsApache KafkaApache SparkArtificial Intelligence (AI)Artificial Neural NetworksAzure DatabricksAzure DevOps ServicesBig Data AnalyticsCloud ApplicationsCloud DeploymentContinuous Integration and Continuous Delivery (CI/CD)Data GovernanceData Mining

About

Senior Data Engineer specializing in building scalable, reliable, and cost-efficient data platforms on Databricks and AWS. Experienced in designing high-throughput pipelines, optimizing Spark and SQL workloads, and implementing strong data quality and governance frameworks. Skilled in data architecture, distributed processing, and infrastructure automation with Terraform and Kubernetes. Passionate about transforming complex datasets into actionable insights and continuously improving performance, reliability, and data-driven decision making.

Experience

5 yrs 11 mos

Total Experience

2 yrs 3 mos

Average Tenure

1 yr 5 mos

Current Experience

Arctic wolf

Senior Data Engineer

Jan 2025 – Present · 1 yr 5 mos · Bengaluru, Karnataka, India · Hybrid

Data Engineering solutions on petabyte-scale data lake on Databricks‬/AWS
Designing scalable pipelines to reliably process terabytes of security observations daily,‬
‭ ensuring high throughput and low latency.‬
Optimizing SQL and Spark queries for efficient data retrieval, reducing compute costs and improving‬
‭ query performance across large datasets.‬
Driving performance tuning, monitoring, and cost optimization of Databricks jobs and Delta tables.‬
Enhancing data reliability with schema validation, error handling, and automated quality checks‬
‭ across ingestion workflows.‬
Robust data governance and RBAC-based access controls, ensuring secure handling of‬
‭ sensitive cybersecurity datasets.‬
‭

DatabricksAWSSQLSparkData GovernanceData Engineering+1

Outsystems

Senior Data Engineer

Dec 2023 – Jan 2025 · 1 yr 1 mo · Bengaluru, Karnataka, India · Hybrid

Data architecture design on AWS for data storage, data‬‭ processing, and analysis‬
Development and deployment of data pipelines and models‬‭ for ETL using Apache Spark and Kafka‬
Implemented scalable data processing solutions using‭ Apache Spark on the Databricks platform including delta‬‭ tables, jobs, workflows, dashboards, and alerts‬
Proficient in SQL, leveraging advanced querying techniques‬‭ and optimization strategies to manipulate and extract‬‭ insights from large datasets on Databricks and Snowflake‬
Data Reporting and Analytics using Opensearch‬
‬‭- Monitoring and Optimizing the Kubernetes environment as‬‭ well as Infrastructure as code using terraform for resources‬
Integrating schema registry for efficient management and‬‭ governance of telemetry data‬
‬‭- Designed and implemented data lineage solutions to track‬‭ data flow and transformations across the pipeline.‬
Designed and implemented distributed systems to ensure‬‭ scalability, reliability, and high availability

AWSApache SparkKafkaSQLKubernetesTerraform+2

Opentext

3 roles

Senior Software Engineer

Promoted

Mar 2022 – Dec 2023 · 1 yr 9 mos · On-site

Designed and developed secure REST APIs for seamless integration and communication between multiple systems and applications, incorporating robust authentication and authorization mechanisms like OAuth and JWT.
Orchestrated the successful migration of on-premises products to the cloud, leveraging cloud deployment technologies to streamline and optimize scalability, availability, and performance.
Implemented Infrastructure as Code (IaC) using Terraform to automate the setup of cloud resources for SaaS products.
Led the development of a password management service as a SaaS product, ensuring robust security measures and seamless user experience.

REST APIsTerraformCloud DeploymentOAuthJWTSoftware Engineering+1

Software Engineer

Jul 2020 – Mar 2022 · 1 yr 8 mos · On-site

(Formerly MicroFocus)
Developed an Analytics Dashboard for Access Manager, focusing on logs analytics and management. Leveraged technologies such as Elasticsearch, Logstash, Kibana, Apache Kafka, Apache Spark, etc.
Utilized Docker and Kubernetes to design and implement microservices for the development of a containerized product.
Streamlined product upgrades with the implementation of an Upgrade Assistant and channel upgrade feature, enabling one-click upgrades.

ElasticsearchDockerKubernetesApache KafkaApache SparkSoftware Engineering+1

Software Engineer Intern

Jan 2020 – Jun 2020 · 5 mos · On-site

(Formerly MicroFocus)
Designed and implemented a DNS certificate issuing client utilizing the ACME protocol, which was integrated seamlessly into the product.
Acquired extensive knowledge of various technologies including SAML, Single-sign-on, OAuth, LDAP, etc.
Implemented secure coding practices to mitigate Cross-site scripting (XSS) attacks and other malicious content injections.

SAMLOAuthSecure Coding