Designed and maintained a production-grade Data Lake for the Disney Identity team using AWS S3 and Databricks to centralize disparate enterprise data sources. Automated API Integration by developing custom transformers in Spark Scala to generate JSON schemas from Swagger API definitions, streamlining data ingestion. Eliminated high manual overhead by building an automated schema comparison tool that instantly identifies structural changes and ensures data modeling consistency across environments.

Automated the ingestion and parsing of new data entries using Spark Scala, facilitating seamless integration for accounting systems. Troubleshot pipeline instability and ensured high-quality delivery through aggressive QA development and automated unit testing for every code change. Maintained and scheduled robust Databricks jobs across DEV, QA, and PRD environments to guarantee 24/7 reliability of financial data pipelines.

Automated data transformation processes for the multi-billion-dollar integration of Disney and Hulu accounting systems using Spark and Python. Designed and maintained automated reporting systems in MicroStrategy using SQL, delivering analyst-ready models via email for executive decision-making. Troubleshot and monitored pipeline outputs for major integration projects using Databricks notebooks, PySpark, and SQL to ensure data integrity. Directly addressed pain points of pipeline downtime by debugging bugs and errors in production environments, significantly reducing manual troubleshooting time.

Maintained a production-grade QA platform tracking daily jobs, utilizing SQL Server as the core engine and Azure Power Apps for a streamlined user interface. Automated ETL jobs for enterprise data from APIs and FTPs (JSON, CSV, TXT) using Transact-SQL to design complex merge scripts for reporting tables. Ingested and parsed API data responses using Azure Logic Apps, ensuring reliable movement of source data into Blob Storage for downstream modeling.

Designed a high-performance pipeline for processing contract-level KPIs, translating complex banking logic into reliable analyst-ready models using PySpark. Troubleshot and optimized reliability for core work meshes, reducing process downtime and manual execution time through improved Spark Scala scripting.

Ingested data efficiently from S3 buckets to MongoDB using multi-threading and Python to minimize manual data collection overhead. Designed end-to-end ETL pipelines for diverse enterprise sources to streamline data movement from EC2/EMR to Redshift and Snowflake-compatible warehouses. Automated data load jobs using Python and AWS, achieving a 60% reduction in load time and ensuring consistent delivery for business intelligence. Automated dynamic ad hoc reporting solutions by developing custom ETL jobs using AWS Glue, significantly reducing the manual burden on analysts.

Ingested data from diverse web sources using Python scraping libraries (Selenium, BeautifulSoup, Scrapy) and OpenCV to bypass manual extraction challenges. Maintained high-performance Data Modeling structures using HIVE to store extracted semi-structured data for downstream analysis. Automated the ingestion of complex semi-structured files (JSON, XML) and database tables using DataStage, ensuring a reliable data flow to Oracle systems.