Professional Profile

About Me

Azure Data Engineering with a Development and Reliability Focus

I am an Azure Data Engineer based in Chennai, India with more than three years of professional experience. My current direction centres on building dependable ingestion and lakehouse workflows across Azure data platforms.

My work is development-focused: I build metadata-driven ingestion, Databricks processing, Delta Lake pipelines and validation controls. I also support pipeline issue investigation and resolution so data delivery remains dependable.

Alongside professional implementation work, I develop portfolio case studies to explore Microsoft Fabric, Mapping Data Flows, validation-first ingestion and maintainable data-quality controls.

Pushpak Vootla, Azure Data Engineer

Working Principles

How I Approach Data Engineering

  1. 01

    Understand the Data Contract

    Clarify the source structure, refresh pattern, expected output and downstream dependency before implementation.

  2. 02

    Validate Before Promotion

    Apply schema, file, record and business-rule checks before data reaches a trusted layer.

  3. 03

    Design for Supportability

    Capture execution status, failure context, row counts and stage-level information needed to investigate pipeline issues.

  4. 04

    Protect Reruns and Watermarks

    Ensure repeated processing and incremental state changes follow controlled success conditions.

  5. 05

    Keep the Design Explainable

    Use clear configuration, consistent naming and documentation so the pipeline can be understood and maintained.

Capabilities

Current Technical Focus

Technologies and practices I currently use or continue to develop through implementation and portfolio work.

Data Orchestration

Pipeline orchestration, parameterisation and configuration-driven data movement.

Azure Data FactoryFabric Data FactoryMetadata-driven pipelinesMaster-child pipelinesParameterised pipelinesParameterised datasetsDynamic expressionsIncremental loadingConditional routing

Data Processing

Batch transformation, validation and notebook-based processing.

Azure DatabricksPySparkSQLPythonMapping Data FlowsDatabricks NotebooksFabric Notebooks

Lakehouse and Storage

Cloud storage and table technologies used across Landing, Raw and curated processing.

ADLS Gen2Delta LakeFabric LakehouseParquetAzure Blob StorageAzure SQL DatabaseSQL Server

Data Quality and Reliability

Controls used to prevent invalid-data promotion and improve operational clarity.

Schema validationRow-count validationDuplicate detectionNull validationReference-data validationQuarantine handlingRejected-record handlingAudit metadataFramework loggingWatermark controlRetry handlingPipeline notificationsRerun-safe processingArchive-before-delete

Governance and Security

Access, credential and configuration practices used across approved implementations.

Unity CatalogAzure Key VaultManaged Identity fundamentalsSecure linked servicesRole-based access fundamentalsSecret separation from codeEnvironment-aware configuration

Development and Delivery

Development and documentation practices supporting maintainable data engineering.

GitGitHubAzure DevOps fundamentalsTechnical documentationData pipeline troubleshootingEnvironment configurationJSON configurationREST and HTTP ingestion

Credentials

Certifications

Professional certifications and learning credentials across Azure, AWS and supporting delivery tools.

Next Steps

Career Direction

I am focused on Azure Data Engineering roles where I can contribute to reliable ingestion, transformation, lakehouse processing and operationally clear data platforms.

I am particularly interested in opportunities involving Azure Data Factory, Microsoft Fabric, Azure Databricks, PySpark, Delta Lake and metadata-driven data engineering.

View ExperienceView Resume