Data Engineering Manager

Job Overview

Overview

The Data Engineering Manager is responsible for

• Design and build a metadata-driven file ingestion framework on Databricks aligned with medallion architecture
• Lead onboarding of datasets using pattern-based ingestion (not one-off pipelines)
• Define and enforce data contracts including schema, keys, and data quality rules
• Implement scalable ingestion patterns supporting header / no-header / schema evolution scenarios
• Drive table-driven configuration model to eliminate dependency on static YAML-based onboarding
• Integrate ingestion framework with governance tools (e.g., DataHub) for lineage, discovery, and ownership
• Ensure strong data quality, auditability, and observability through centralized logging and control tables
• Collaborate with business and source system teams to understand data semantics and define keys
• Establish best practices for file ingestion, schema management, and incremental processing
• Lead vendor teams to ensure delivery aligns with target architecture and platform standards

Education

• BLD – Bachelor's Level Degree in computer science required
• 15 years of experience in a complex computing environment may be considered in lieu of degree.

Certification/Licensure

• Databricks Certification (Preferred)

Experience

• 8+ years in data engineering with strong experience in modern data platforms (Azure preferred)
• Hands-on expertise in Databricks and PySpark for large-scale data processing
• Strong understanding of medallion architecture (Bronze / Silver / Gold)
• Experience designing metadata-driven or configuration-driven data pipelines
• Deep knowledge of data ingestion patterns (batch, file-based, incremental loads)
• Strong experience with data modeling concepts (keys, SCD, merge strategies)
• Experience implementing data quality frameworks and validation rules
• Proficiency in SQL and distributed data processing concepts
• Experience with Azure Data Lake Storage (ADLS) and file-based ingestion patterns
• Familiarity with CI/CD, Git, and deployment practices in data engineering Preferred Experience
• Experience with data catalog and governance tools (DataHub, Collibra, Alation, etc.)
• Exposure to customer 360 / MDM / CRM data integration (Salesforce, Dynamics, etc.)
• Experience working in healthcare or regulated environments (HIPAA, PHI handling)
• Familiarity with Unity Catalog and data access governance models
• Experience integrating data platforms with workflow tools like ServiceNow for access management
• Exposure to event-driven ingestion patterns (file triggers, streaming, etc.)
• Experience working in large-scale migration or modernization programs ,enterprise data platforms
• Experience enabling self-service data access and governance
• Familiarity with API-driven data integration patterns

We provide market-competitive compensation packages, inclusive of base pay, incentives, and benefits. The base pay rate for Full Time employment is:$116,729.60-$216,777.60. Additional compensation may be available for this role such as shift differentials, standby/on-call, overtime, premiums, extra shift incentives, or bonus opportunities.

About Us

Sentara Health, an integrated, not-for-profit health care delivery system, celebrates more than 135 years in pursuit of its mission – “we improve health every day.” Sentara is one of the largest health systems in the U.S. Mid-Atlantic and Southeast, and among the top 20 largest not-for-profit integrated health systems in the country, with 34,000 employees, 12 hospitals in Virginia and Northeastern North Carolina including 10 hospitals with the prestigious Magnet recognition, and the Sentara Health Plans division which serves more than 1 million members in Virginia and Florida. Sentara is recognized nationally for clinical quality and safety, and is strategically focused on innovation and creating an extraordinary health care experience for our patients and members. Sentara was named a Health Quality Innovator of the Year (2024), was recognized by Forbes as “America’s Best-In-State Employer” (2024), “Best Employer for Veterans” (2022, 2023), and “Best Employer for Women” (2020), and named to IBM Watson Health’s “Top 15 Health Systems” (2021, 2018).

Learn more (Opens in a new tab)