Lakshmi B

Data Engineer

Hyderabad, Telangana, India7 yrs 9 mos experience

Most Likely To SwitchHighly Stable

Key Highlights

Designed scalable data pipelines for actionable insights.
Achieved 40% reduction in ETL latency through optimization.
Expert in Big Data technologies and cloud platforms.

Stackforce AI infers this person is a Data Engineering expert in Healthcare and Financial services with a strong focus on Big Data technologies.

Contact

Skills

Core Skills

Data EngineeringEtl Development

Other Skills

ADLS Gen2AnsibleApache ImpalaApache SparkApache Spark StreamingApache SqoopAzure Data FactoryAzure Data LakeAzure DatabricksBatch ProcessingBig DataBig Data AnalyticsBitbucketCDCCSS

About

Hello! I'm Lakshmi, a passionate and results-driven Data Engineer with hands-on experience in Big Data, Cloud technologies, and Data Warehousing. I have experience in designing and building robust, scalable data pipelines that transform raw data into actionable business insights. With a strong background in ETL development, analytics solutions, and end-to-end system design, I have delivered successful projects for leading clients across banking, hospitality, and health insurance domains. I have Expertise in building ETL pipelines using Big Data technologies Pyspark, Spark SQL, Azure Databricks, Azure Data Factory on Azure Cloud and Snowflake. Proficient in handling large-scale data processing, optimizing Spark jobs, and applying complex business logic transformations. Experience in both Agile and Waterfall methodologies, working across the full Product lifecycle — from requirement gathering to production deployment. A self-starter and fast learner who thrives in both independent and collaborative environments. Always curious, always growing — a true data enthusiast who embraces continuous learning and upskilling. Core Skills & Tools : Cloud & Data Platforms: ADF, Databricks, Snowflake, Data Lake, Lake House Big Data: Apache Spark (Core, SQL), Hive, YARN, MapReduce, Sqoop Programming: Python, SQL, Unix Shell Scripting Databases: Oracle (11g), Exadata, Netezza, Teradata Schedulers & ETL: Autosys, Tivoli, Informatica DevOps & Version Control: Git, Bitbucket, SVN, Ansible, Jira Tools: Putty, Toad, WinSQL, SQL Developer

Experience

7 yrs 9 mos

Total Experience

3 yrs 10 mos

Average Tenure

4 yrs 7 mos

Current Experience

Optum

2 roles

Data Engineer

Promoted

Mar 2023 – Present · 3 yrs 2 mos · Hyderabad, Telangana, India

1. Designed and developed batch and real-time streaming ETL projects using PySpark and Lakehouse technologies in Databricks, implementing Medallion Architecture.
2. Managed end-to-end process, from data profiling and quality checks to ingestion, transformation, and building the reporting layer to support data-driven solutions.
3. Optimized long-running Spark jobs through advanced performance optimization techniques resulting in a 40% reduction in ETL latency.
4 . Ingested and transformed data from multiple source systems into centralized DW like Snowflake & Delta table.
5. Implemented SCD1/SCD2 logic, designed Fact and Dimension models, and applied Star and Snowflake schema architectures to ensure data accuracy, integrity, and consistency
6. Ensured pipelines are cost-effective and scalable , enabling seamless growth as data volumes increased by 15% year-over- year .
7. Enabled 20% faster executive reporting with optimized Snowflake queries.
8. Collaborated with stakeholder and cross-functional teams to understand data requirements and design data-driven solutions, including analytical and reporting systems, to support business decision making and optimize operational insights.

PySparkLakehouse technologiesDatabricksETLSnowflakeData ingestion+3

Associate Data Analyst

Sep 2021 – Feb 2023 · 1 yr 5 mos · Hyderabad, Telangana, India

1. Responsible to build and maintain the end to end Data Process for the Modernized Reporting Suite.
2. Migrated legacy HQL scripts to highly optimized PySpark scripts, utilizing Spark's distributed processing power & Optimization Techniques to streamline data workflows and achieve a 50% reduction in processing time, while enhancing scalability and performance.
3. Worked on the optimization of the ETL pipelines. Tuned the slow Performing spark jobs to use optimization techniques and resources in an optimal way to reduce the overall processing time by 35%. 4. Monitored and maintained data solutions, resolved issues, and optimized performance.
5. Collaborate closely with business stakeholders and cross-functional teams to design and implement data-driven solutions that align with business objectives and deliver measurable impact

PySparkETLData ProcessData workflowsOptimization techniquesData Engineering+1

Tata consultancy services

2 roles

System Engineer

Jun 2019 – Aug 2021 · 2 yrs 2 mos

ETL developer responsible to build ETL pipelines using Hadoop and pyspark framework.
Worked in Banking project on credit risk and deposit data platform.
Responsible to build data pipelines using shell scripts and pysparks scripts making sure of lineage, governance and security of data across modeling layer.

HadoopPySparkETLData governanceData securityData Engineering+1

Assistant System Engineer

Jun 2018 – Jun 2019 · 1 yr

Worked in Data warehousing and reporting projects as an ETL developer, responsible to build pipelines using Hadoop, Hive ,Sqoop and traditional RDBMS, Unix shell scripts.
Worked in Hospitality project, responsible to warehouse the historical and present data to do inventory and batch processing on estimation of reservation and revenue for each property. The processes data will be used for reporting purpose

HadoopHiveSqoopUnix shell scriptingData warehousingData Engineering+1

South central railway

Summer Intern

May 2017 – Jun 2017 · 1 mo

Developed a website for the power generation Department of railways using HTML5, CSS, JavaScript and Oracle database for the backend Connectivity. At Software Development Center, South Central Railway, Vijayawada.

HTML5CSSJavaScriptOracle database