Azure Data Engineer · Chennai, India
Azure Data Engineer with 3+ years of professional experience and hands-on experience building data pipelines using Azure Data Factory, Azure Databricks, PySpark, Delta Lake, ADLS Gen2 and Microsoft Fabric.
Focused on metadata-driven orchestration, data validation, pipeline observability, issue resolution and dependable downstream data delivery.
SQL Server, Files, APIs
Azure Data Factory, Fabric Data Factory
Schema checks, file validation, data-quality rules
PySpark, Mapping Data Flows, SQL
ADLS Gen2, Delta Lake, Fabric Lakehouse, Azure SQL
Technical Focus
Platforms and engineering patterns used across ingestion, transformation, validation, lakehouse processing and pipeline reliability.
Selected Work
Hands-on projects demonstrating ingestion design, validation, incremental processing, lakehouse workflows and production-oriented controls.
Microsoft Fabric
Lakehouse and Data Quality
A Microsoft Fabric ingestion workflow that discovers incoming files, validates trusted office CSV files and loads approved records into a curated Lakehouse table while routing unsupported or schema-incompatible files to quarantine.
Azure Data Factory + Azure Databricks
Metadata-Driven Quarterly Processing
A completed quarterly source-to-Raw platform combining metadata-driven ADF orchestration, Databricks validation, Delta Lake processing, Unity Catalog registration and protected watermark updates.
Azure Data Factory
Metadata-Driven Ingestion
A configurable Azure Data Factory ingestion framework that reads source behaviour from metadata and processes SQL Server, HTTP or REST and file-based sources through reusable pipeline components.
Azure Databricks
PySpark and Data Quality
A retail data pipeline that ingests order and order-item data from Amazon S3, stages it in ADLS Gen2 and applies Databricks PySpark validation before loading approved records into Azure SQL Database.
Azure Data Factory
Mapping Data Flow and Conditional Routing
A parameterised Azure Data Factory pipeline that reads movie data from Azure Blob Storage, applies Mapping Data Flow transformations and routes validated outputs to Azure SQL Database or ADLS Gen2 according to release-year rules.
Engineering Approach
A pipeline is not complete only because data moved successfully. Validation, security, monitoring and recovery behaviour also need to be considered.
Clarify source structure, target requirements, data volume, refresh pattern and failure expectations.
Use reusable and parameterised components to move data from files, databases or APIs.
Check schema, required fields, duplicates, reference values and record quality before promotion.
Apply maintainable PySpark, SQL or Mapping Data Flow logic based on the platform and use case.
Organise data into landing, raw, validated, curated or medallion layers using appropriate file and table formats.
Capture execution status, errors and row counts, then support safe retries, quarantine handling and controlled reruns.
Experience Snapshot
Currently working as an Associate Data Engineer at Ensono Technologies LLP, developing metadata-driven ingestion, Databricks lakehouse processing, validation and incremental-loading controls while supporting pipeline issue resolution.
Credentials
7 completed certifications and badges support my cloud, data and delivery foundation.
Amazon Web Services Training and Certification
Amazon Web Services Training and Certification
Contact
Explore my experience and project case studies, or contact me regarding relevant Azure Data Engineering roles.
