Senior Data Engineer
We are seeking a Senior Data Engineer to lead the development and optimization of data pipelines, data movement tools, and data preparation services to support advanced analytics and enterprise integration. This role requires deep expertise in handling both structured and unstructured data, designing scalable data architectures, and developing reliable ETL solutions. The ideal candidate has significant experience with both cloud-native and on-premise technologies, and thrives in collaborative, mission-driven environments.
Clearance Requirement: TS/SCI with Polygraph
Key Responsibilities:
Design, implement, and maintain scalable data pipelines using tools such as Spark, Apache Iceberg, Trino, OpenSearch, EMR, NiFi, and Kubernetes
Perform data ingestion, extraction, formatting, and transformation for a wide range of data types and formats
Clean, preprocess, and format data for exploration, analysis, and operational use
Develop ETL tools and scripts and enhance existing pipelines using coding best practices
Work with Data Scientists, analysts, and mission partners to align data pipelines with mission goals
Collaborate with software engineers to configure and maintain data services and back-end components
Ensure data quality, standardization, and provenance through collaboration with testing and data quality teams
Produce comprehensive documentation including ETL mappings, data lineage, code usage guides, and access instructions
Participate in enterprise working groups to define and promote data standards across systems
Support both one-time and ongoing data extraction and transformation efforts across various repositories
Required Qualifications:
10+ years of experience in data lifecycle engineering
Deep expertise in ETL development, pipeline design, and data architecture
Strong coding skills in Python, SQL, Spark, and other data engineering tools
Experience with COTS and open-source tools including ElasticSearch and Apache NiFi
Strong understanding of cloud and on-premise data storage and processing solutions
Experience working in Agile development environments
Proven track record of integrating diverse data sources into enterprise systems
Preferred Qualifications:
Experience deploying solutions on cloud platforms (e.g., AWS EMR, S3, RDS, EKS)
Familiarity with data governance, data lineage tracking, and security requirements
Hands-on experience with container orchestration using Kubernetes
Knowledge of data lake architectures and streaming data processing
Excellent communication skills and experience collaborating with cross-functional teams
Tech Stack & Tools:
Languages & Frameworks: Python, SQL, Spark
Data Tools: Apache NiFi, Apache Iceberg, Trino, ElasticSearch, OpenSearch
Cloud & Infrastructure: AWS EMR, Kubernetes, Docker, On-Prem Systems
DevOps & CI/CD: Git, Agile methodologies, automated testing frameworks
Why Join M1?
Tackle large-scale data engineering challenges with national impact
Work with cutting-edge cloud and open-source data tools
Collaborate with a multidisciplinary team of engineers and analysts
Flexible, mission-driven environment with opportunities for leadership
M1 Technology is an equal opportunity employer and values diversity. We do not discriminate in hiring on the basis of race, color, religion, sex, national origin, age, disability, veteran status, or any other characteristic protected by federal, state, or local law.
Recommended Jobs
Subject Matter Expert
Foreign Resource Development Associates, LLC (FRDA) is a privately held Service-Disabled Veteran-Owned Small Business (SDVOSB) headquartered in Arlington, VA. FRDA's team of dedicated employees and a…
Shower & Bath Installer - Earn Bonuses for Happy Customers!
Job Description Job Description Bathroom Installer We are looking for a highly skilled Bathroom Installer to join our team! In this role, you will be responsible for the efficient installati…
Misc. Steelworker
The employee must have the basic knowledge and skills to fabricate, install and assemble structural steel. Tasks & Experience Required: Should have at least 1 year welding exp. Weld using …
Carpenter
Summary: A Carpenter for BOWA plays an important role working closely with Project Managers helping to manage and complete residential remodeling work with the highest quality craftsmanship. This posi…
Sales and Marketing Specialist
Job Title: Sales and Marketing Specialist Location: Williamsburg, VA Zip Code: 23185 Duration: 8 months Pay Rate: $82.85 /hr. Keyword's: #Williamsburgjobs; #MarketingSpecialistjobs. Start Date: …
Caregiver- URGENTLY HIRING (Haymarket)
Alliance Home Care, LLC is a small business that is supportive, inclusive, professional and our goal is to look for compassionate and caring caregivers, who are dedicated to having a positive impact o…
Senior Manager/Operations Manager
Job Description Job Description Description Senior Manager/Operations Manager WHO WE ARE: STAHL Companies provides the Program Management for its Channel of Commercial Technology c…
Specialty Medical Director (Board Certified / Diplomate / Residency Trained) J207037
**Position Available: Specialty Medical Director - Criticalist, IM – Ashburn, VA** Elevate your career to the next level with this leading role that offers a competitive salary up to $300,000. Embra…
Scribe - Carilion Roanoke Memorial Hospital
How You’ll Help Transform Healthcare: Take this rare job opportunity to apply your skills to a dynamic new setting. Keep vital teams organized, informed, and fully prepared to succeed. At Carili…
Construction Manager I
What We're Looking For At HNTB, you can create a career that is meaningful to you while building communities that matter to all of us. For more than a century, we have been delivering solutions fo…