Nitish Dobriyal

Data Engineer

Dehradun, Uttarakhand, India7 yrs 1 mo experience

Most Likely To SwitchHighly Stable

Key Highlights

Expert in AWS and Databricks for data solutions.
Proven track record in building efficient data pipelines.
Strong foundation in machine learning and data analysis.

Stackforce AI infers this person is a Data Engineer specializing in cloud-based data solutions and machine learning applications.

Contact

Skills

Core Skills

DatabricksAwsEtlData PipelineCi/cdData EngineeringWeb DevelopmentMachine Learning

Other Skills

AWS LambdaAgile MethodologiesAlgorithmsAmazon Elastic MapReduce (EMR)Apache AirflowApache KafkaApache Spark StreamingBMC Control-MBig DataBitbucketC++CSSCascading Style Sheets (CSS)Control-MData Modeling

About

Experienced Data Engineer with 5+ years of experience into Data Engineering. My professional skills include: -> AWS Services: S3, Redshift, Lambda, EC2, EMR, Dynamo DB, SQS, SNS -> Databricks: Delta Lake, Workflows, Asset Bundles, Autoloader, Unity Catalog -> Open source: Apache Spark, Spark Streaming, Apache Hive, Apache Hadoop, Sqoop -> Programming: Python, Unix -> Databases/DWHs: Mysql, Oracle, Redshift, Delta Lake -> Orchestration Tools: Apache Airflow, Stonebranch UAC, Control M -> DevOps: Bitbucket, GIT, JIRA, CI/CD, Jenkins -> Data Science: Numpy, Pandas, Matplotlib, Seaborn, EDA, Inferential Statistics, Machine Learning -> Good knowledge of Kafka and messaging queue frameworks Passionate about learning, developing, and building efficient solutions by solving real-time business problems proficiently.

Experience

7 yrs 1 mo

Total Experience

3 yrs 6 mos

Average Tenure

4 yrs 3 mos

Current Experience

Tiger analytics

2 roles

Data Engineer

Jul 2024 – Present · 1 yr 11 mos

Senior Software Engineer - Big Data

Mar 2022 – Jul 2024 · 2 yrs 4 mos

● Worked on building a curated layer on Lakehouse harnessing data from harmonized layers to craft various dimension tables. Utilized Databricks Notebooks for transformations and workflows for full load and incremental load processing from harmonized to curation layers. Applying SCD type1, and type 2 for merge strategies. Using Delta table optimizations, improved query response times and and overall system
efficiency. Utilized Databricks asset bundles to streamline deployment and management of Infrastructure as Code (IaC) artifacts (workflows, tasks)
● Worked on developing a Data Ingestion solution using AWS Serverless methods. Utilised AWS Lambda, SQS, SNS, S3 and DynamoDB for the same
● Worked on migration of more than 100 ETL jobs(Hive, Pyspark/Python based) to different AWS Datalake.
● Developed a data pipeline to transform moving JSON Data coming from Kafka source to write it to Redshift Spectrum on the s3 layer. Orchestrated the pipeline with Apache Airflow.
● Spearheaded the creation of an automated CICD pipeline for deploying Redshift objects, improving, efficiency and enabling continuous delivery.

DatabricksAWS LambdaSQSSNSS3DynamoDB+4

Infosys

3 roles

Senior Systems Engineer

Promoted

Oct 2021 – Mar 2022 · 5 mos

● Designed and developed data ingestion pipelines and infrastructure for various Payment Messages.
● Worked on data processing, data modeling, batch side scripting and orchestration
● Other tasks include unit testing, code deployment. Tech used: Python, SQL, S3, Shell Scripting, Control-M

PythonSQLS3Shell ScriptingControl-MData Engineering

Systems Engineer

Aug 2019 – Sep 2021 · 2 yrs 1 mo

● Developed applications for Netreveal(Proprietory tool).
● Worked on writing shell scripts for automated file ingestion.
● Worked on writing SQL scripts for handling application backend.

Shell ScriptingSQL

Systems Engineer Intern

Jan 2019 – May 2019 · 4 mos · Mysore

● Developed 'Hunger Heroes' web app to create an interlink between NGOs and Vendors for distribution of waste food across the city.
● Worked on Relational Modelling for the several entities related to the business.
● Used PHP, MySQL, HTML, CSS, JS to build Vendor and Lab modules and web interfaces separately for the project.

PHPMySQLHTMLCSSJavaScriptWeb Development

Sunfox technologies

Summer Intern

May 2018 – Jul 2018 · 2 mos · Dehra Dun, Uttarakhand, India

Project Spandan:
Worked on analyzying ECG Generated output using Python Numpy, Pandas and Matplotlib.
For predictive model used different ML and DL libraries like sci-kit learn, Keras to classify 4 different types of Arrhythmia cases.

PythonNumpyPandasMatplotlibKerasSci-kit Learn+1