Role: Senior Data Platforms Developer / Lead | Dates: Jan 2025 - Dec 2025 | Status: Completed
Technologies: Azure Databricks, Delta Live Tables (DLT), Spark Structured Streaming, Kafka, Azure Event Hub, Redis, Python, Terraform, Azure DevOps
Leading the technical design and implementation for migrating core components of the enterprise data platform from a nightly batch refresh model to real-time data ingestion and processing. This involves designing and building streaming pipelines, integrating with messaging queues (Kafka, Event Hub), leveraging caching solutions (Redis), and defining infrastructure using Terraform. Also contributing to the Data Platform Governance committee overseeing these changes.