Abhishek Gupta

Product Manager

San Jose, California, United States17 yrs 5 mos experience
Highly Stable

Key Highlights

  • Led efficiency initiatives saving hundreds of millions.
  • Architected cloud-native 3D visualization technologies.
  • Managed high-performance computing projects at scale.
Stackforce AI infers this person is a Cloud Infrastructure and HPC expert with a focus on efficiency and security.

Contact

Skills

Core Skills

Infrastructure EfficiencyCost OptimizationPerformance EngineeringCloud ComputingHigh Performance ComputingSecurityTechnical ConsultingCloud Security

Other Skills

3D visualizationAMQPAlgorithmsApache MesosApplication securityCC++C/C++CI/CDCloud architectureCloud technologiesCloud-native technologiesCommunicationComputer ScienceContainer security

About

Seasoned technical engineering manager, 3.5 years EM exp (1.5 at Meta, 2 at Schlumberger STIC), 3 different teams: - Currently EM at Meta, inherited 8 people team with multiple senior ICs (IC6, IC7s), delivering a) fleet-wide efficiency tools, power observability for billion $ infra b) applied efficiency wins (autoscaling, stacking, load balancing AI etc) saving 100s of Millions. - Prev, managed and scaled team (4 to 8) to deliver capacity sufficiency and efficiency for major Meta products such as Instagram, Messenger, WhatsApp, AI infra. - At SLB innovation center, architected and managed team to deliver 3D viz in the cloud, O(months workflow) to O(seconds), scaled team from 2 to 9 15+ years as in infra, public+private cloud runtimes and efficiency, some notable examples - Initiated and tech-led Messenger architecture re-deployment, re-mapping 2 billion+ user, landing $100+M impact - Initiated systems for efficiency opportunity discovery, 200M worth opportunities discovered, 50 Million were landed. - Initiated and tech led tools for Meta efficiency visibility, monthly capacity report, sent to Product VPs, educated leadership and ICs, ran performance academies - CS PhD from UIUC (HPC+cloud). B-tech gold medalist from IIT Roorkee. 10+ patents and 30+ technical research papers. Previously, I lead innovation projects at Schlumberger innovation (STIC). My team (which I grew from 2 to 6 full time people, 10 with interns) has delivered several impactful POCs, and handed over significant subset to business stakeholder. Key success include cloud-native 3D viz, (https://www.slb.com/campaigns/gaia.aspx), HPC in Cloud with GCP, 100X faster production optimization, next gen oilfield visualization with game engines. Contributed as technical leader, architect, hands-on roles at Schlumberger, Intel, HP Labs, UIUC, Microsoft. CS PhD from UIUC. B-tech gold medalist from IIT Roorkee. Outstanding ability green card & O-1 visa.

Experience

Databricks

Software Engineering Manager - cost efficiency

Sep 2024Present · 1 yr 6 mos · Mountain View, California, United States · On-site

  • Heading efforts on cloud efficiency and cost optimization at Databricks

Meta

2 roles

Engineering Manager TLM - Infrastructure efficiency

May 2022Sep 2024 · 2 yrs 4 mos

  • Leading infra efficiency team at Meta, supporting multiple horizontal efficiency efforts, domain expert in deployment efficiency efforts around containerization, hardware replacement efficiency, load balancing, storage buffer management, compression
  • Previously - Manage and technically lead team focusing on Performance engineering in Capacity Engineering and Analysis (CEA) org for major products at Meta, including Instagram, Messenger, WhatsApp, Video. Also, aspects of cloud demand management and efficiency.
Infrastructure efficiencyContainerizationLoad balancingStorage buffer managementCost optimization

Tech Lead Engineer on Efficiency, Performance, capacity infrastructure

Nov 2019May 2022 · 2 yrs 6 mos

  • Infrastructure engineering at Meta
  • Current focus - Efficiency tracking and improvements @Meta - includes fleet wide efficiency aspects:
  • monitoring and reporting different aspects of utilization, software efficiency and regressions, hardware efficiency, including how to make them actionable
  • Tech led and devised systematic detection of software regressions from load test of production applications, improve quality and accuracy, landed multimillion $ savings
  • initiated systems for visibility of fleet-wide efficiency opportunities with quantifiable impact
  • Motivating efficiency and performance through company-wide performance education, created trainings
  • technically lead xfn teams to deliver efficiency tools, data pipelines
  • communicate from engineers to Directors/VPs on overall efficiency state.
  • Previously, Drove Capacity planning and Efficiency initiatives of Meta infrastructure for Messenger. Landed deployment re-architecture of how users are mapped to regions to optimize for efficiency and capacity fungibility.
  • Improved HHVM performance through ML based approach and autotuning of performance parameters. Paper under submission. Multi million $$ savings.
  • Capacity planning and performance improvements during increased load during COVID 19.
Efficiency trackingMonitoringReportingPerformance educationInfrastructure efficiencyPerformance engineering

Schlumberger

2 roles

Software Engineering Manager - HPC and Visualization

Dec 2017Nov 2019 · 1 yr 11 mos

  • Led and managed projects in the High performance computing and 3D visualization innovation at STIC center. In addition to my technical hands-on work on HPC projects on GCP cloud, I was also responsible for:
  • technically lead and prioritize innovation projects
  • work with business stakeholders to ensure alignment
  • work with silicon valley ecosystems - startups, Google, NVidia, universities (Stanford, Berkeley) to ensure we can benefit from cutting-edge advancements in computer science
  • present HPC vision and innovation projects to customers, stakeholder, partners
  • grow the team, hence recruiting and interviewing
  • One of my major accomplishments has been the successful handover of cloud remote visualization prototype done at STIC to a tech center (https://www.slb.com/campaigns/gaia.aspx). The project has been commercially being offered to clients as part of our DELFI visualization experience. I also presented it at GCP booth at Supercomputing conference 2018 and Nvidia GTC conference 2019. The success outcome is handover of POCs to business stakeholder. Key successes include cloud-native 3D viz, Other key successful initiatives include HPC in Cloud with GKE, 100X faster production optimization, next gen oilfield visualization with game engines.
  • I have grown the team from 2 to 6 full time people, for summer of 2019, I recruited 4 interns as well. The team has delivered:
  • highly parallel and scalable production network optimizations using the Google cloud Kubernetes engine
  • applying game engine technology to oilfield visualization, technologies such as Unity and MapLarge
  • prototyped next generation visualization technologies such as NVidia Index, webRTC, point clouds for seismic visualization
  • HPC technologies for machine learning using frameworks such as Kubeflow, distributed Tensorflow, Impala
  • technology monitoring of HPC architectures and ecosystem such as TPU, quantum computing etc.
High Performance Computing3D visualizationCloud-native technologiesCloud Computing

Software Architect at STIC - Cloud, Security, HPC

Jan 2017Nov 2019 · 2 yrs 10 mos

  • Applying cutting edge cloud, security, and computer science technologies to HPC application and systems at Schlumberger Software and Technology Innovation Center (STIC) in Silicon Valley. Part of Schlumberger's journey of move to digitalization, specifically moving workloads and services to cloud.
  • Most recently, I have been looking at frameworks and HPC infrastructure to scale ML training and RL. I am investigating Horovod, Kubeflow, IMPALA among others.
  • Architect and build REST and gRPC high performance services for seismic processing and I/O using Google cloud (GCP) from ground-up. Micro-services architecture. Dealt with:
  • Performance and scalability: XX performance improvements through several optimizations - caching and parallel processing, (auto) scalability.
  • Security: authentication, authorization, encryption etc
  • Resilience through replication, retries, failure tolerance.
  • Continuous integration and continuous deployment (CI/CD) using Jenkins and VSTS.
  • Code quality through static analysis tools, cppcheck, PMD, sonar, and unit test, integration tests
  • Also, drive security initiatives at the center to address security issues at infrastructure, platform, and application level in cloud - firewalls, auth, VPNs etc. In addition the following:
  • architecture and implementation of the first data source for the rendering architecture, which was a proof point for the visualization architecture and allowed us to differentiate, without doubt, the cloud visualization product.
  • took the initiative for a large part of the security efforts done for STIC (automated firewall scans, user scans, etc.).
  • mentored interns and new hires, also did GCP admin training in Houston for cloud security
  • was one of the key contributors for the collaboration with other tech centers
  • made hands-on contribution to security at STIC by writing scripts for periodic monitoring and alerting of firewall related security incidents
Cloud architectureSecurity frameworksMicroservicesCloud ComputingSecurity

Santa clara university

Guest Faculty Professor - Cloud Computing

Jan 2017Jan 2021 · 4 yrs · San Francisco Bay Area

  • Adjunct Professor in Computer Engineering at SCU from Jan 2017. Teaching graduate level course on Cloud Computing in Winter quarter 2017: includes lectures, assignments, exams, projects. So far I have taught this class 4 times, I generally teach it in winter quarter (Jan to March) every year and update the course according to latest development in the field
  • For cloud computing class:
  • Got AWS and Google cloud grants for student projects
  • Class of 35 graduate students, 8 projects with IEEE style project reports - helped them develop the projects and plan potential conference publications
  • Designed course syllabus and material, got access to presentation slides from publishers and adapted them to meet academia and industry trends in cloud computing
  • developed homeworks and mid-terms to give students both theoretical as well as hands-on knowledge of cloud computing including cloud providers like AWS, GCE and cloud technologies such as kvm, mapreduce, docker containers.
Cloud ComputingCourse designProject supervision

Self

Technical Consultant/advisor

May 2016May 2018 · 2 yrs · San Francisco Bay Area

  • Technical consulting for short-lived projects and technical advise , some of the clients I have helped:
  • Sociallist - helped infuse devops process into Google App Engine app - created staging and prod versions, helped fix multiple app engine specific issues including security and monitoring (https://sociallist.io/)
  • Advised as a SLURM expert to a DOAR client (https://www.doar.com/experts) - included running SLURM scripts and validating SLURM capabilities on supercomputers
  • Advised JISA group on micro-service architecture and devops
  • Occasionally act as subject matter expert on Deepbench and Clarity
  • Technical Skillset
  • Private and public cloud: Kubernetes, Docker, Mesos, SDI, KVM, Google cloud, GCS, GKE, Endpoints, Amazon EC2, Azure, Openstack
  • Fullstack/Backend/Distributed: scalable microservices, Python Django, Flask, auth, gRPC, protobuf, REST, swagger
  • Security: Host security - SELinux, iptables, Host IDS, Suricata, Web firewall WAF, Intel TxT, Network security - IPSec, OpenVPN, Regex, IDS. Application security - static and dynamic analysis. Container security
  • Perf optimization, HPC, MPI, OpenMP, shell scripting, C/C++, VTunes
  • HPC for ML using Kubeflow, Tensorflow, Impala
  • Misc: Zabbix, ElasticSearch, Logstash, Kibana (ELK), PubSub, AMQP, CI/CD, Git, Scrum, SLURM.
Technical consultingDevOps processesMicro-service architectureTechnical Consulting

Intel corporation

Cloud Security Architect

Jul 2014Jan 2017 · 2 yrs 6 mos · San Francisco Bay Area

  • One of my key accomplishments was Intel CIT 3.2 Docker container integrity. I was the technical lead on it, I architected and prototyped the solution and later led a team to deliver the product. The software has since then open sourced (https://github.com/opencit/opencit/wiki/Open-CIT-3.2-Product-Guide#60-image-integrity-vm-and-docker)
  • My other accomplishments were
  • Research and develop solutions for cloud and data center security - data security and network security including SDN/NFV and SDI
  • Architect and perform pathfinding for trusted computing in cloud - such as Docker containers and VMs in openstack clouds
  • Trusted Docker containers, integration with Openstack
  • Feature awareness in Mesos, Kubernetes, Docker Swarm
  • H/W assisted performance acceleration of critical security workloads such as VPN, IPsec
Cloud securityData securityNetwork securityCloud Security

Hewlett-packard laboratories

3 roles

Research Associate Intern

May 2012Aug 2012 · 3 mos

  • Researched techniques for application-aware VM placement in cloud (mentor Dr.
  • Dejan Milojicic)

Visiting Researcher/Contingent Worker

Sep 2011Sep 2013 · 2 yrs

  • Designed and implemented methods for bridging HPC-cloud divide. Presently, collaborating on performance evaluation and simulation of next-generation systems for data-intensive applications

Research Associate Intern

May 2011Aug 2011 · 3 mos

  • Evaluated the performance and mapping of HPC applications in cloud (mentor
  • Dr. Dejan Milojicic)

University of illinois at urbana-champaign

Graduate Research Assistant

Aug 2009Jun 2014 · 4 yrs 10 mos · Urbana-Champaign Area

  • Worked on various projects: large-scale HPC applications, parallel runtime systems, and schedulers for both clouds and HPC (select projects lised below).

Microsoft

Software Design Engineer

Jul 2008May 2009 · 10 mos · Greater Hyderabad Area

  • Worked in the team Microsoft CRM (Customer Relationship Management)

Oracle

Software Engineer Intern

May 2007Jul 2007 · 2 mos · Bangalore

  • Worked on Apache Tomcat server deployment
  • Worked on multi-threaded SOAP request generator

Education

University of Illinois Urbana-Champaign

Doctor of Philosophy (Ph.D.) — Computer Science

Jan 2009Jan 2014

Stanford University

XINE229 - Leading Innovation — Organizational Leadership

Jan 2019Jan 2019

University of Illinois Urbana-Champaign

MS — Computer Science

Jan 2009Jan 2011

Indian Institute of Technology, Roorkee

Bachelor of Technology (B.Tech.) — Computer Science

Jan 2004Jan 2008

Stackforce found 100+ more professionals with Infrastructure Efficiency & Cost Optimization

Explore similar profiles based on matching skills and experience