I like figuring out how things work. And making them work better.
I’m a data engineer with 6+ years of experience in consulting, manufacturing, and fintech. I’ve built a Databricks platform from scratch at AGCO and run Payla’s data platform as its sole data engineer.
I enjoy getting into the details—why a pipeline is slow, where a number comes from, or what someone actually needs from a report. Most days that means working with Python, SQL, Spark, and AWS, and talking to the people who use the data.
From consulting for manufacturing teams to owning a fintech data platform.
MAR 2025 — PRESENT
Munich, Germany
Data Engineer @ Payla Services GmbH
Running the data platform behind payments, reporting, and financial operations.
Since June 2025, I’ve owned the platform as the sole data engineer, working directly with leadership and the Finance, Risk, Integration, and Site Reliability Engineering teams.
What I worked on
Built PySpark and Polars pipelines for batch, streaming, change data capture, reconciliation, and near-real-time financial processing.
Delivered one-time and daily bank reporting files to AWS S3 using Dagster and Databricks.
Resolved production incidents across Amazon MSK, Kafka Connect, MongoDB change data capture, Databricks, and Kubernetes during migrations and reporting cycles.
Tuned Kubernetes CPU and memory allocation for more than 50 bronze tables to reduce infrastructure costs.
Automated releases with GitLab CI/CD and merge trains, publishing Python wheels to S3 and managing versioned deployments.
Built a GitHub Copilot custom agent to identify memory-heavy Databricks jobs and propose resource changes, changelog updates, and validation steps.
100+
Unit and integration tests
≈95%
Coverage across transformations, orchestration, and SDK integrations
Building the Enterprise Data Hub on Databricks from scratch.
Helped migrate AWS-native workloads to Databricks on AWS, establishing Unity Catalog, access management, jobs, Git integration, Databricks Asset Bundles, and GitHub Actions.
Migrated and optimized Spark, SQL, Kafka, and Databricks workloads: improved reporting speed by 30%, reduced latency by 25%, and improved Unity Catalog query performance by up to 3×.
Proposed and delivered Salesforce ingestion with AWS AppFlow, replacing long-running custom REST API jobs in Glue.
Ran Databricks workshops for data engineers and business teams to support adoption of the new platform.
Worked across client engineering and business teams to deliver cloud and data solutions. Translated requirements into user stories, estimated work, and delivered across Scrum sprints.
Improved Spark and SQL processing performance by 20% through caching, partitioning, and workflow tuning.
Technology AnalystJan 2023 – Mar 2024
Senior Systems EngineerJan 2022 – Dec 2022
Systems EngineerDec 2019 – Dec 2021
AWSSparkSQLConsultingScrumManufacturing
03
Education & credentials
JUL 2015 — JUL 2019
Bachelor of Engineering
Information Science
Visvesvaraya Technological University Bengaluru, India
CERTIFICATION
Databricks Certified
Data Engineer Associate
English — fluent German — A1, currently learning B1
04
What people say
Excerpts from recommendations shared by colleagues on LinkedIn.
“Arpit is amazing at understanding very complex setups in a very short period of time and really interested to understand the whole picture so he can support others.”
“Throughout our joint time Arpit improved all of (my) reports and found ways which helped me solve issues with data retrieval (some of which I didn’t even know I had).”
Claudia KafkaCFO, Payla Services GmbHSeptember 2026 · Managed me directly
“Arpit was instrumental in deploying a new lakehouse on Databricks. He handled it from the design stage all the way up to the go-live phase.”
“In addition, he drove the Proof of Concept, aligned and set up the environment, and took charge of onboarding and training the data engineers.”
“Arpit consistently demonstrated not only a deep technical proficiency but also an ability to work collaboratively and take on significant responsibilities.”
Sabine PagelSenior Digital Transformation & Data LeaderSeptember 2024 · Senior colleague
“Arpit’s expertise in Amazon Web Services (AWS) and PySpark was instrumental in architecting and implementing robust data solutions and pipelines that met and exceeded our project requirements.”
“Beyond his technical expertise, Arpit is a collaborative team player who communicates effectively, fosters a positive working environment and approaches challenges with a solutions-oriented mindset.”
Alper KocaData Architect, Bucher HydraulicsMarch 2024 · Collaborated across companies
“Arpit has been an amazing data engineer to work with.He is very dedicated and sincere in his effort to the assigned work.He adapts any skill within a week and make sure not to miss the delivery timeline.”
Ankur SrivastavaCloud & Big Data ArchitectFebruary 2025 · Worked on the same team
