Oct 2021 – May 2023 · 1 yr 7 mos · New York City Metropolitan Area

• Open source contributor in RADICAL Pilot funded by DoE and NSF
• Research in Linux Kernel Memory Management
• Contributed to Ansible Stacks, and Android Tooling
• Recipient of Paul Panayotatos Endowed Research Grant for Digital Sustainability and Privacy -
• https://www.linkedin.com/posts/rutgers-electrical-and-computer-engineering1_kartik-modi-awarded-the-2022-paul-panayotatos-activity-6914589575860428800-SGxk/
• Areas of Focus: HPC - Supercomputing, Distributed Computing, and Operating Systems

Apr 2021 – Aug 2021 · 4 mos

• Designed a framework for incremental, real-time data marts for data quality, analytics, and dashboarding.
• The datamart framework uses a combination of Spring, Spark Streaming, and Postgres functions using Kafka as a source in the same build package
• Identified frequently accessed dataset, converting into datamarts for significantly reduced data scan, improving debuggability, logging, auditing, column-level access control based on Postgres schema isolation

Jan 2020 – Mar 2021 · 1 yr 2 mos

• Designed a DWH in Timeseries, Incremental store in TimescaleDB for cataloging, discovery, data marts, ACL data quality - (Failed - Designed again in Postgres later)
• Architectural high-level contribution in Schema Registry Design
• Contribution in in-house Presto setup on ec2 with terraform, ansible, and monitoring & autoscaling scripts
• Async Logging implementation in SDI - reduce latency from ~4ms to ~2ms, improved I/O, containers reduced by 30%
• CPS Metrics - Analyze the data generated by CPS project for it's correctness, completeness, accuracy, etc.

Jul 2018 – Dec 2019 · 1 yr 5 mos

• Architected, Designed and Owned CPS(Cost Per Shipment) - Activity-based costing. Batch based historical data analysis jobs with Spark Df for cost allocation at shipment level
• Resolving several spark CPS issues related to concurrent writes, handling write failures, shuffle issues, memory issues, duplicated data etc and gained expertise in tuning internals of Spark.
• Developed unified standards for data accessing from different data sources like Datalake(s3), DWH(redshift), RealtimeDB(postgres), Hive(Athena) in CPS
• Developed a Reactive Microservice for exposing Api's of CPS on Non-blocking Reactor Library with SpringWebflux, Phoenix(Hbase).
• Developed delhivery's centralized stream API in Flask for producing data in Kafka ~10ms latency.
• Optimized stream API with design of 2 level in-memory caching after profiling and battle-tested stream API for serving up millions of of requests per minute reducing latency <4ms.
• Developed Datalake(s3) & real-time db(postgres) pipelines on Kafka Connect Framework.
• Initiated and helped adoption of Stream processing on KStreams across org, solved small business requirements with State stores
• Debugged Kafka broker, connector jvm head/core dump analysis, connect issues involving concurrent kafka connect's thread management, broker's disk, memory and load issues with the help of jvm tools (jstack, visualvm, eclipse mat, gceasy), newrelic insights(nrql), ganglia and distributed tracing & cloudwatch metrics.
• Workarounds build and deployment issues and later redesigned deployment and monitoring 300+ pipelines with detailed monitoring, automation and alarms using Jenkins, docker, ELK, Newrelic, Cloudwatch, NFS.
• Developed a proprietary in-house library of UMS(user management system) caching layers (in-memory & redis) for other microservices packaged into python whl.
• Design and Code Reviewer for team of 9 with managing 3 teammates

Nov 2017 – Jul 2018 · 8 mos

• Developed several custom features like SSO, Periodic Jobs in Reporting - Redash (Open Source BI Tool)
• Migrated Reporting Service to Docker ECS Orchestration with runtime scalability for bulk reports reducing latencies by 40% in some cases.
• FT for single scheduler for reporting with RPC comm. increasing it's SLA
• DbCleanup: Developed distributed highly FT microservice for database cleaning with a master-slave arch.

May 2016 – Nov 2017 · 1 yr 6 mos

• Debugged several convoluted L4 issues like concurrency problems involving deadlocks, thread-safety in data structures, thread signaling, deadlock detector, locking hierarchy.
• Patched performance hit bugs causing network choking, high load-avg, cpu spikes after detailed profiling analysis.
• Received positive feedback on RCAs to rarely recreated issues usually involving network trace dumps, large heap/core dump analysis, jstack states mapping with os threads while debugging server slowness, gc-analysis, debugging os level issues with wireshark, mitmproxy, tcpdump, strace, gdb, auditd, abrtd, sar logs, nagios
• Authored HLD for near-real time Business Dashboarding involving several caching layers, H2 database, optimized query plans improving performance upto 39% on denormalized tables, and exposing dynamic stats definition creation through Rhino engine.
• Developed generalized highly-configurable and user-customizable dashboard UI with tons of widgets, charts, and visualizations using GWT(Google Web Toolkit) and it's libraries.
• Conducted Design Optimizations while migrating older data-allocation APIs of single to (batch or stream) based, in-house event-driven Rule Engine from sync to aysnc calls
• Debugged low-level real-time calling issues with the Asterisk call server and workarounds with event interactions during critical downtimes.
• Primary Owner for "StatsManager" and db issues (pg_locks, bloating, xid, query plans, plpgsql, materialized view, pgpool, postgis, replication, lag, pg_toast, row corruption)
• Product: http://www.ameyo.com/
• Company: http://www.drishti-soft.in/
• Tech: OSGi, Postgres, GWT, Hibernate, DACX (In-house Java SOA Java Framework), Eclipse EMF, Corosync, CentOS, Python, PHP, Codegens(In-house), C
• Accomplishments:
• Rated mostly "A+" grade in Design Reviews and Code Reviews of major features.
• Service Champion Award for delivering Business Analytics Dashboards
• Third Position in Tech Hunkaar Hackathon

Oct 2015 – Dec 2015 · 2 mos

• Front-End Development, Client Change Requests, UI/UX Wireframes.
• Tech: Angular, jQuery, Bootstrap, Photoshop, Illustrator
• http://www.creativetechnos.com/

May 2015 – Jun 2015 · 1 mo

• Full-day Internship -
• Responsible for backend API development for upcoming product Retrofin (Restaurant App) in CakePHP
• Packaging and Deployment automation on EC2
• Redesigned some parts of URL Shortener http://k7g.in/
• Flagship Product: https://www.wijungle.com/
• Company: http://www.httpcart.com/

Feb 2014 – Feb 2016 · 2 yrs

• Offered Web Application, Web Designs, and Android App Development Services.
• Gained 15 Clients worldwide primarily from North Americas and Europe.
• Top-Rated Service on Fiverr.com
• Portfolio: http://www.emodiz.com/