AI Data Pipeline Engineer
AI Data Pipeline Engineer
Elite Technical is seeking AI Data Pipeline Engineer will design and optimize data pipelines that efficiently move data from sources to AI processing services, ensuring seamless integration with human evaluation workflows. This role requires a deep understanding of AI workflows and the ability to translate complex data requirements into scalable, secure pipelines.
Responsibilities include, but are not limited to:
- Develop and maintain data pipelines using AWS services (Glue, Lambda, Kinesis) or open-source tools (Apache Kafka, NiFi, Airflow) for structured/unstructured data ingestion through embedding and loading into vector databases
- Integrate data with AI models (LLMs, RAG) via RESTful APIs, ensuring low-latency processing and scalability.
- Validate data quality, implement caching (Redis), and optimize pipelines for performance (50-2000 tokens/sec).
- Collaborate with systems administrators to ensure compliance with security and classification requirements.
- Document pipeline workflows and provide support for human evaluation of AI outputs.
This position is a full-time employment opportunity with our client, who recently was awarded the opportunity to build an AI prototype for their government client. This is an exciting opportunity to lead the charge for a cutting-edge government AI related project.
Required Skills
Required Qualifications:- Clearance Required: Must be a U.S. Citizen and possess a current and active TS/SCI clearance granted by the Department of Defense or an Intelligence Community agency. Must be able to pass a Counterintelligence (CI) Polygraph (current CI Polygraph a plus).
- 3+ years in data engineering or AI infrastructure.
- Expertise in data pipeline tools (AWS Glue, Kafka, NiFi) and AI integration.
- Strong problem-solving skills for optimizing data flow and latency.
- Experience with JSON, Python, and API-driven architectures. Education Qualifications: Bachelor-s degree or higher or equivalent experience
Years of Experience:
- 7+ years w/ High School Diploma
- 3+ years w/ Bachelor's Degree, Master-s Degree, or PhD
Desired Qualifications:
- Knowledge of LlamaIndex, Qdrant, and classification-aware data processing.
Recommended Jobs
Full Time Family Practice Job Norfolk, VA
Conveniently situated on the beaches of the Atlantic Ocean and Chesapeake Bay, residents of Norfolk can enjoy some of the best beaches in the United States. The region also offers countless entertainm…
Cyber Security Support for USPTO IT Support
Cyber Security Support for USPTO IT Support Management Applications, Inc., a leading provider of Network Management, Design and Implementation is seeking Cyber Security Support for a long-te…
Front-End Software Developer
Northstrat is seeking an experienced and driven Senior Front-End Software Developer to join our dynamic team. The ideal candidate will have extensive experience in front-end development, bringing a w…
Senior Electrical Engineer
Join our Alexandria team as a Senior Electrical Engineer ! In this leadership role, you’ll manage a team of electrical engineers and work on multidisciplinary projects across sectors including healt…
Accounting Manager
Job Description Job Description Accounting Manager Quantum Leap is the leading technology services company focused on providing operationally focused solutions for select clientele within the …
Business Data Analyst (hybrid)
At Agile Defense we know that action defines the outcome and new challenges require new solutions. That’s why we always look to the future and embrace change with an unmovable spirit and the courage …
Biscuit Maker/Baker-Hardee's Restaurant-Bristol 1, VA (1881)
Overview: At Capstone Restaurant Group, we are dedicated to the fast-food industry and are independently owned and operated as a franchisee in the Hardee’s Restaurant System. We are always searching …
General Superintendent OCONUS
American International Contractors (Special Projects), Inc. (AICI-SP) is currently seeking General Superintendent candidates to work on upcoming secure projects OCONUS. An ACTIVE U.S. Secret…
RN Emergency Room
RN Emergency Room – Salem, VA Pay : $32.33 - $48.50/hr, based on years of experience Night Shift Differential : $3.00/hr Weekend Day Shift Differential : $3.00/hr Weekend Night Shift D…