Data Engineer
Data Pipeline Development & ETL/ELT
- Design, develop, and maintain robust ETL/ELT pipelines to ingest, transform, and deliver data across enterprise platforms.
- Build scalable data ingestion frameworks for structured and semi-structured data, including XBRL filings and financial datasets.
- Implement data transformation logic to support analytics, reporting, and regulatory use cases.
- Ensure data pipelines are reliable, performant, and scalable in cloud environments.
- Leverage AI-assisted development tools to accelerate pipeline development, testing, and optimization.
- Develop and manage data solutions leveraging AWS services (e.g., S3, Airflow, DAGs, Glue, Lambda, Redshift).
- Implement and optimize Apache Iceberg table formats for large-scale, ACID-compliant data lakes.
- Support lakehouse architectures that unify data lakes and data warehouses.
- Optimize data storage and retrieval strategies for performance and cost efficiency.
- Enable data platforms that support AI/ML workloads and downstream generative AI use cases.
- Design and implement CI/CD pipelines for data pipelines, infrastructure, and analytics code using tools such as GitHub Actions, GitLab CI, Jenkins, or AWS-native services.
- Automate build, test, and deployment processes for ETL pipelines and data platform components.
- Implement DataOps best practices, including version control, automated testing, environment promotion, and rollback strategies.
- Ensure reproducibility, reliability, and governance of data pipeline deployments across environments.
- Integrate AI-driven testing and monitoring tools to improve pipeline quality and reduce operational risk.
- Design and implement materialized views and other performance optimization techniques to improve query efficiency.
- Tune data pipelines and queries for performance, scalability, and cost.
- Implement partitioning, indexing, and caching strategies aligned to workload patterns.
- Develop pipelines to ingest, parse, and normalize XBRL (eXtensible Business Reporting Language) data.
- Support regulatory and financial data use cases requiring high accuracy and traceability.
- Ensure alignment with data standards and validation rules for financial reporting datasets.
- Apply context engineering principles to ensure data is enriched with meaningful metadata, lineage, and business context.
- Collaborate with Data Architects to support data modeling, schema design, and entity relationships.
- Enable downstream analytics and AI use cases by structuring data for usability, discoverability, and governance.
- Integrate pipelines with enterprise data catalogs and metadata management systems.
- Support automated metadata capture, lineage tracking, and data quality monitoring.
- Ensure alignment with data governance frameworks and standards established by OCDO organizations, including AI data readiness and traceability.
- Collaborate with data architects, analysts, and business stakeholders to understand data needs and deliver solutions.
- Participate in stakeholder listening campaigns, workshops, and data discovery efforts.
- Work in Agile teams to iteratively deliver data capabilities and enhancements.
- Contribute to identifying and implementing AI-driven efficiencies and automation opportunities across the data lifecycle.
- Bachelor’s degree in Computer Science, Engineering, Data Science, or related field.
- 5+ years of experience in data engineering, ETL development, or data platform engineering.
- Strong hands-on experience with:
- ETL/ELT tools and frameworks
- AWS data services (S3, Glue, Lambda, Redshift, etc.)
- Apache Iceberg and modern data lake architectures
- Experience designing and implementing CI/CD pipelines for data platforms and ETL workflows.
- Demonstrated proficiency using AI tools and AI-assisted development workflows (e.g., LLM copilots, automated code generation, pipeline optimization tools).
- Experience processing XBRL or complex financial/regulatory datasets.
- Proficiency in SQL and Python.
- Experience implementing materialized views and query optimization techniques.
- Understanding of data modeling concepts and metadata management.
- Familiarity with data governance, data quality practices, and data readiness for AI/ML use cases.
- Ability to work in Agile, DevOps-oriented environments.
- U.S. Citizenship required; ability to obtain and maintain a federal clearance.
- Experience supporting federal agencies such as SEC, DHS, Treasury, or Federal Reserve System.
- Familiarity with data catalog tools (e.g., Collibra, Alation, ServiceNow).
- Experience with Apache Spark, Kafka, or other distributed data processing frameworks.
- Experience enabling data pipelines for AI/ML or generative AI applications.
- Knowledge of data maturity frameworks (e.g., EDM DCAM, TDWI).
- Exposure to context engineering or semantic data layer design.
- AWS or data engineering certifications.
- Experience with infrastructure-as-code (IaC) tools (e.g., Terraform, CloudFormation) in support of CI/CD pipelines.
Recommended Jobs
Data Integrations Specialist (Team Lead)
Description CLGT Solutions, LLC dba CLOVEHITCH is a Service-Disabled Veteran-Owned small mission-focused business that provides professional solutions in the areas of intelligence, financial manag…
AWS Data Engineer (Capital One)
We are seeking AWS Data Engineers with our client in McLean, VA for a Hybrid role. Job Details: Job Title: AWS Data Engineer Location: McLean, VA Work Mode: Hybrid T…
Senior Mechanical Engineer
Job Description Job Description At Whitman, Requardt & Associates, LLP, we are "People Focused and Project Driven". We have been in business for more than 110 years and we are known for our quali…
Marketing Specialist in Sterling, VA - Full Time
Job Title: Marketing Specialist – Brand Partnerships Position Classification: Exempt Department: Marketing Reports To: Vice President of Marketing Salary-$75,000-$85000 Location-Sterlin…
Quality Assurance Specialist
Job Description Job Description Description: This is a great opportunity for a self-motivated Quality Assurance Specialist with focus on product inspection for military components, products, and…
Engineering Technician I
Public Trust: None Requisition Type: Regular Your Impact Own your opportunity to support our nation's defense. Make an impact by connecting and securing critical operations across the glob…
Deputy Course Chair/Coordinator (Office Management and Communication Training)
Background: Kent, Campa and Kate Incorporated (KCK) is a federal contracting company that provides professional services to Government Agencies. We are currently seeking candidates for a Deputy Co…
Laborer - Asphalt Milling - Hampton Roads
Job Title: Laborer - Asphalt Milling - Hampton Roads Department: Paving Reports to: Paving Foreman Major Role and Purpose : ~ Performs various unskilled tasks that are associated wit…
School Director Intern
Job Description Job Description Job Title: School Director Intern Location: Stafford, VA (Doc Stone) Pay: $20.00/hour Schedule: 40 hours/week | 12-week internship Overview: Children…
Registered Behavior Technician, ABA Therapist
Job Description Job Description At Bee Wise Behavior, we serve children and young adults with Autism Spectrum Disorders. Every child has a treatment team consisting of a Board Certified Behavior …