About Experience Skills Education Get in Touch
Building what's next

Manan Bhatia

Senior Software Engineer

Building and modernizing large-scale distributed data pipelines for cloud security platforms. Deep expertise in streaming infrastructure, batch and stream processing, and multi-cloud environments.

0+ Years Experience
0+ Complex Services Built
0+ Languages in Production
$M+ Cloud Costs Saved
pipeline.rs
async fn process_stream(ctx: &Context) {
    let consumer = KafkaConsumer::new(
        &ctx.config.brokers,
        &ctx.config.topic,
    ).await?;

    while let Some(msg) = consumer.recv().await {
        let batch = msg.compute_offsets();
        lakehouse.write(batch).await?;
    }
}

Engineering at Scale

I'm a Senior Software Engineer at Fortinet, where I build and modernize large-scale distributed systems microservices for cloud security platforms.

I specialize in building high-throughput distributed systems — from streaming architectures and real-time data processing to fault-tolerant platform infrastructure at enterprise scale.

Streaming Infrastructure

Kafka, Redpanda, Spark-based real-time and batch processing at massive scale

Platform Reliability

Building watchdog monitors, error tracking systems, and self-healing infrastructure

Multi-Cloud

Deep expertise across AWS, GCP, and Azure — designing cloud-agnostic solutions

Beyond the Code

When I'm not writing stream processing algorithms, you'll find me exploring New York's art scene — from gallery openings in Chelsea to hidden installations in Tribeca. On weekends, I'm usually hunting down the city's best speakeasies for their mocktail menus. I don't drink alcohol, but I'm a sucker for a well-crafted mocky.

Gallery Hopper

If there's an opening, I'm probably there

Mocktail Connoisseur

Zero proof, full flavor — one speakeasy at a time

What I've Built

and where

Senior Software Engineer

Fortinet (via acquisition of Lacework)

June 2023 — Present New York, NY

Audit Pipeline Modernization & Streaming

  • Architected Spark-based stream processing system consuming AWS CloudTrail logs from Kafka, computing batch-hour offsets for the in-house data lakehouse
  • Led Azure Activity Logs modernization — refactored legacy pipeline to stream via Kafka with Spark summarization, cutting Snowflake dependency and storage costs
  • Built a metadata service writing event boundaries to Kafka, enabling Spark SQL batch windows — now foundational for all pipelines migrating off Snowflake
  • Solved large payload failures with chunked 1MB writes and checkpointing; discovered and fixed a legacy bug causing partial data downloads
  • Spearheaded GCP migration for 300+ customers with zero data loss, reducing migration time by 500% through full automation and Terraform integration

Platform Reliability & Performance

  • Diagnosed critical Rust Kafka consumer failure; built persistent error tracking with auto-restart, differentiating ephemeral vs. real errors
  • Built Snowflake connection pool watchdog in Rust using thread channels to detect hanging ODBC connections and restart K8s pods
  • Designed Java config poller with stateful-retry for AWS key deletion, reducing on-call incidents by 50%
  • Implemented Kafka consumer offset bootstrapping for new customer onboarding

Infrastructure & Storage Optimization

  • Migrated Kafka clusters to Kubernetes with cohort-based topic migration using data volume segmentation
  • Completed EKS pipeline modernization for Snowflake deprecation — virtual topic epoch switches, full task lifecycle management
  • Leveraged Google Pub/Sub audit logs to improve throughput by 100% and reduce alert response from hours to seconds

Cross-Functional Leadership

  • Resolved multiple high-priority SEVs including GKE audit log fixes and Security Hub enablement
  • Built Azure AD events processor in Rust reading from customer Event Hubs
  • Partnered with Principal Engineers to drive architecture decisions for new pipeline services
  • Developed CLI capabilities in Go and modified ingestion services in Java — improving stability by 60%
Kafka Spark Rust Java Go Kubernetes AWS GCP Azure Terraform

Software Engineering Intern

Lacework

May 2022 — Aug 2022 Mountain View, CA
  • Implemented the company's first Kubernetes-based Event Driven Autoscaling system for pod auto-scaling
  • Built Apache Kafka pipeline to migrate from AWS Kinesis — reducing costs and making ingestion cloud-agnostic; deployed across US, EU, and AU
Kubernetes KEDA Kafka AWS Kinesis

Engineering Intern

Lacework

May 2021 — Aug 2021 Mountain View, CA
  • Built end-to-end data visibility metrics in Java with Wavefront dashboards for full ingestion observability
  • Built HTTP endpoint for async Snowflake queries with unit tests and feature-flagged production deployments
Java Snowflake Wavefront K8s

Technical Arsenal

Languages

Java Rust Go Python C C++ SQL JavaScript HTML/CSS

Streaming & Data

Apache Kafka Redpanda Apache Spark Spark SQL Snowflake Data Lakehouse

Cloud & Infrastructure

AWS GCP Azure Kubernetes Terraform EKS GKE EC2 S3 CloudTrail Pub/Sub Event Hubs

Tools & Practices

GitHub Actions CI/CD Feature Flags Distributed Systems Microservices

Foundation

Bachelor of Science in Computer Science

Purdue University West Lafayette, IN

GPA: 3.71 Dean's List — All Semesters Class of 2023

Let's Connect

Interested in discussing distributed systems, cloud infrastructure, or potential opportunities? I'd love to hear from you.