Balaji Rajagopalan

Associate Consultant

Hamburg, Hamburg, Germany26 yrs 4 mos experience
Most Likely To SwitchHighly Stable

Key Highlights

  • Expert in data engineering and architecture.
  • Proven track record in anomaly detection and predictive modeling.
  • Strong experience with diverse technology stacks.
Stackforce AI infers this person is a Data Engineer with strong expertise in Fintech and SaaS environments.

Contact

Skills

Core Skills

Data EngineeringData ScienceTechnical LeadershipSoftware Development

Other Skills

Agile MethodologiesComputer ScienceDistributed SystemsETL pipelinesEclipseExtract, Transform, Load (ETL)JMS queueJava Enterprise EditionLinuxPandas (Software)PerlPython (Programming Language)Unixairflowandroid

About

Experienced kotlin/scala/java developer, working closely with data science team, great knowledge of working with diverse technology stacks such as, hadoop, yarn,spark/hive, and real time streaming with apache flink/kafka, JMS message queues/RabbitMQ, reactive frameworks such as vert.x, play, spring framework, spring batch for pipeline creation, micro services development for implementing rest interface and no sql stores such as redis/redshift. Specialties: scala, java,ruby, c++,c j2ee,big data ecosystem hive/spark, nosql redis,redshfit,database postgresql, oracle, solid, mysql, containers such as docker, kubernetes, queueing system such as rabbitmq, jms,kafka, with experience in developing services using micro service architecture, and used angular js,html,css for front end development. I am passionate about coding, reactive programming excites me, scala is an awesome language that suites this need. These days I am looking into machine learning algorithms, practicing problems on both supervised and unsupervised learning.

Experience

26 yrs 4 mos
Total Experience
2 yrs 1 mo
Average Tenure
4 yrs 8 mos
Current Experience

Thoughtworks

Senior Data Consultant

Oct 2021Present · 4 yrs 8 mos · Hamburg, Germany

  • Joined thoughtworks team worked with a Pharmaceutical as a client, and a investment banking client, trying to implement data mesh architecture, along with data vault modelling. Worked on setting with databricks workspace, fault detection algorithm using flink, video/text embedding generation, data quality monitoring using anomalo.
data mesh architecturedata vault modellingdatabricksfault detection algorithmflinkdata quality monitoring+2

Jimdo

Senior Data Engineer

Mar 2020Sep 2021 · 1 yr 6 mos · Hamburg, Hamburg

  • Jimdo is one of early companies enabling small business building website and providing them a digital platform. I have joined the data team, we use redshift as warehouse, with ETL pipelines mostly written in python. Did some work with anomaly dedution of input raw data and worked on predictive model for forcasting the payment using simple linear regression, involved in exploratory data analysis including deployment of model in production.
redshiftETL pipelinesanomaly deductionpredictive modelingexploratory data analysisData Engineering+1

Deposit solutions

Senior Data Engineer

Jan 2019Feb 2020 · 1 yr 1 mo · Hamburg Area, Germany

  • Working on streaming solution on flink for anomaly deduction, the plan is to use simple rules to begin with and use data science later on to deduct anomalies in the transaction data related to the fintech application. The Building new reporting platform at deposit solutions.Spark was used for ETL. Developed micro services using kotlin.
flinkanomaly deductionstreaming solutionsparkmicroservicesData Engineering

Smaato

Senior Data Engineer

Dec 2017Dec 2018 · 1 yr · Hamburg Area, Germany

  • Worked along side data science team with some interesting work with traffic curation for eliminating traffic which has less potential revenue.
  • I used to work with data team , involved writing workflows using airflow to crunch to data from Kafka using spark jobs. I was involved in moving the message broker from Kafka to kinesis, found a critical bug with kpl library in AWS which was causing the service that hosting the kinesis process cpu hit 100%, worked with the AWS team to get that fixed. The data team workflow processes about billions of ad request messages per day or 12 terabyte of data. Working on streaming processing system with flink to reduce the time from source to sink from 6 hours to 15 minutes.
airflowsparkkafkakinesisstreaming processingData Engineering

Philips

Technical Lead

Aug 2016Nov 2017 · 1 yr 3 mos · Bengaluru Area, India

  • 1. Solved a connection issue with elastic search where the client connection where getting dropped, tuned the ping timeout parameter which solved the issue.
  • 2. Debugged a production issue with elastic search, suggested to eliminate the use of dynamic indices to static index so schema changes do not cause the parse failure of data.
  • 3. Suggested couple of platform improvement such as adding a metric collection system for tracking the pipeline jobs and export functionality to get the data out of hive to external storage such as redshift/postgresql, these were taken up in subsequent releases.
elastic searchpipeline improvementmetric collectionTechnical Leadership

Ola (ani technologies pvt ltd)

Software Development Engineer III

Jan 2016Jul 2016 · 6 mos · Bangalore

  • 1. Worked with apache flink to read the data from the kakfa streams and insert to sql sink as aggregates, which is input to peak price algorithm. Got to work closely with data sciences team which was fun.
  • 2. Made the execution of peak pricing algorithm horizontally scalable using the redis queue to distribute the load to nodes in the cluster.
apache flinkkafkaredispeak pricing algorithmSoftware Development

Flipkart.com

Software Development Engineer III

Oct 2014Dec 2015 · 1 yr 2 mos · Bangalore

  • 1. Fixed a complex multi-threaded issue with java android app that we developed for managing warehouse operations, the toughest part of the issue with gathering logs from the devices, which were later moved to elastic search.
  • 2. Implemented command pattern in the backend code for bunch of server rest api which cleaned up the readability of the code.
  • 3. Implement a data driven way for invoking the next intend in the android UI which provided a greater flexibility to configure the UI without code changes.
javaandroidmulti-threadingelastic searchSoftware Development

Oracle

Consulting member of technical staff

Nov 2012Oct 2014 · 1 yr 11 mos · Bangalore

  • Made an important architecture contribution to the EM service where I suggested to use middleware technology such as JMS queue as opposed to oracle advance queueing, had to prove by suggestion through performance numbers to get the buy in.
middleware technologyJMS queueSoftware Development

Yahoo!

Principal Engineer Yahoo Bangalore

Jan 2010Oct 2012 · 2 yrs 9 mos · Bengaluru Area, India

  • Debugging a really tough multi-threaded issue with transport software which was used to get data from front end to the back end server.

Kulicke and soffa

Senior Software engineer

Oct 2007Dec 2009 · 2 yrs 2 mos · Greater Philadelphia Area

  • Worked the crystal space 3D engine to display the labels for the wires and bond site. The outlines were created using blender a 3D tool, and the sprite was rendered with varying length, width and angle.

Trueposition

software engineer

Nov 2004Oct 2007 · 2 yrs 11 mos · Greater Philadelphia Area

  • 1. Fixed a bottleneck issue in performance data collection module there was an accidental sleep in code by a fellow developer was causing the issue, debugged the issue with timestamps from the logs.
  • 2. Helped the customer support engineering to identify the root cause of the problem.of RMI exception NoSuchObjectException was thrown when the RMI gets into bad state and the reason for this was a temporary network outage.

Velankani software private limited

software engineer

Jan 2004Jan 2007 · 3 yrs

  • I was consultant with velankani working for true position

General bandwidth

software engineer

Jan 2001Oct 2004 · 3 yrs 9 mos · Austin, Texas Area

  • There was a call trace module which was written by another team and incorrect data was not flowing into our system from that module, so I debugged and helped the team fix the issue, this was outside my area of responsibility.

Motorola india electronics ltd

software engineer

May 1999Dec 2000 · 1 yr 7 mos

  • Debugged an interoperability issue of snmp agent with other subsystem, the other team was claiming things are fine from their end, got the tcpdump of response from the component proved there an issue with response, had fun doing really low level debugging.

Education

Drexel University

MS — Computer Science

Jan 2005Jan 2009

National Institute of Technology, Tiruchirappalli

Bachelor in Engineering — Electronics and Communication Engg

Jan 1996Jan 1999

Alpha matriculation higher secondary school

Matriculation XStd

Jan 1981Jan 1993

Stackforce found 100+ more professionals with Data Engineering & Data Science

Explore similar profiles based on matching skills and experience