Simera Professional Key (SPK)

LUIS C

Colombia

Senior Data Engineer

5 yrs exp

Luis es un Senior Data Engineer con aproximadamente 5 años de trayectoria en datos, especializado en Azure Databricks, Python, PySpark, SQL, Azure Data Factory y arquitecturas Lakehouse/Medallion. Cuenta con certificaciones Databricks Data Engineer Professional y Associate, además de una maestría en Inteligencia Artificial. Ha liderado iniciativas de optimización, gobierno de datos y modernización de plataformas, incluyendo implementaciones de Unity Catalog y automatización CI/CD con Azure Dev…

Skills

  • Azure
  • data modeling
  • Deep Learning
  • Docker
  • Git
  • Neural Networks
  • NLP
  • Python
  • Spark
  • SQL
  • 3D Modeling
  • Azure DevOps
  • Data Engineer
  • Data Factory
  • DataBricks
  • Keras
  • Neural Network
  • Phyton
  • Power BI
  • Pyspark
  • Scikit-learn
  • Snowflake
  • Splunk
  • Python/R
  • Data Governance
  • Data engineering
  • PowerBI
  • Python3
  • FastAPI
  • deepl learning
  • Paython
  • Langchain
  • MSQL
  • Streamlit
  • AWS S3
  • Fast API
  • Data Quality
  • Apache Airflow
  • SOQL
  • Azure data factory
  • Azure Synapse
  • Spark SQL
  • TSQL
  • Neural Networks (CNNs
  • Pyrhon
  • SparkSQL
  • Unity Catalog

LUIS C

Colombia

Senior Data Engineer

5 yrs exp

Luis es un Senior Data Engineer con aproximadamente 5 años de trayectoria en datos, especializado en Azure Databricks, Python, PySpark, SQL, Azure Data Factory y arquitecturas Lakehouse/Medallion. Cuenta con certificaciones Databricks Data Engineer Professional y Associate, además de una maestría en Inteligencia Artificial. Ha liderado iniciativas de optimización, gobierno de datos y modernización de plataformas, incluyendo implementaciones de Unity Catalog y automatización CI/CD con Azure Dev…

Skills

  • Azure
  • data modeling
  • Deep Learning
  • Docker
  • Git
  • Neural Networks
  • NLP
  • Python
  • Spark
  • SQL
  • 3D Modeling
  • Azure DevOps
  • Data Engineer
  • Data Factory
  • DataBricks
  • Keras
  • Neural Network
  • Phyton
  • Power BI
  • Pyspark
  • Scikit-learn
  • Snowflake
  • Splunk
  • Python/R
  • Data Governance
  • Data engineering
  • PowerBI
  • Python3
  • FastAPI
  • deepl learning
  • Paython
  • Langchain
  • MSQL
  • Streamlit
  • AWS S3
  • Fast API
  • Data Quality
  • Apache Airflow
  • SOQL
  • Azure data factory
  • Azure Synapse
  • Spark SQL
  • TSQL
  • Neural Networks (CNNs
  • Pyrhon
  • SparkSQL
  • Unity Catalog

Senior Data Engineer

CI&T
October 2024 - present

Project: Analytical Gateway & Cloud Infrastructure. Optimized business PySpark/SQL logic in Databricks reducing critical reporting pipeline execution time from 126 hours to under 30 minutes while ensuring 100% data accuracy. Automated CI/CD pipelines in Azure DevOps, migrating manual deployment workflows to multi-stage pipelines (Dev/UAT/Prod). Architected a high-performance API using FastAPI and Databricks SQL Warehouse, enabling secure, low-latency data access for global analytical layers. Built declarative data pipelines with Delta Live Tables and Spark Declarative Pipelines, defining transformations as streaming and batch tables with built-in expectations for data quality and automatic dependency management. Project: Modern Data Warehouse Migration. Led the migration of legacy financial systems (Oracle EPBCS, PostgreSQL) into Snowflake, leveraging dbt to build a modular Medallion Architecture and orchestrating via Azure Data Factory. Facilitated technical strategy and demonstrations in English for US-based stakeholders to ensure business-technical alignment.

Data Engineer

Summan SAS
May 2024 - October 2024

Managed the comprehensive administration of the Databricks workspace, implementing enterprise data governance through Unity Catalog and External Locations. Designed incremental and streaming ingestion with Databricks Auto Loader and Spark Structured Streaming, processing large volumes of JSON, Parquet, and HTML files into the Bronze layer of a Medallion architecture with schema evolution handling. Ensured seamless data integration with Azure Synapse Serverless SQL to optimize query capabilities. Contributed to the design of the Medallion ETL architecture, enabling secure and efficient data democratization across company analytics layers.

Data Engineer

Experis Colombia
November 2023 - May 2024

Responsible for the maintenance and evolution of Delta Lake Houses in Databricks, aligning technical integrity with strategic business needs. Redesigned the ETL ecosystem, achieving a significant operational impact by reducing delta table ingestion and update times from 8 hours to under 60 minutes. Developed optimized workflows for distributed analytics utilizing orchestration tools like Azure Data Factory and distributed processing languages such as PySpark.

Analytics Engineer

MVM Software Engineering
August 2022 - November 2023

Designed and maintained large-scale batch data pipelines on Azure Synapse and Azure Data Factory, using SQL and PySpark for complex transformations across multiple source systems. Built orchestrated ETL/ELT workflows with scheduling, dependency handling, and failure recovery to deliver reliable, analytics-ready datasets. Delivered Power BI dashboards on the curated layer for cost and project execution visibility, translating technical metrics into business insights.

Analytics Engineer

Pi Epsilon SAS
October 2021 - July 2022

Led the development of Business Intelligence solutions by creating advanced dashboards in Power BI for the analytical monitoring of infrastructure projects. Managed the visualization of key financial indicators, including Earned Value Analysis (EVA), billing reports, and cost control.

Smart Scores

Communication
85
Role Fit
98
Adaptability
85
Problem-solving
90

Smart Skills

beta