Data Engineer

Position Overview:

We are seeking a skilled and motivated Data Engineer / Database Engineer to support our growing team providing an enterprise data warehouse and visualization environment. The ideal candidate will have hands-on experience with Microsoft SQL Server, ETL pipelines, and cloud-based data ingestion tools—especially AWS Glue. You will be responsible for maintaining and enhancing our database systems, enabling near real-time data pipelines from source databases (Oracle, SQL, etc) to Microsoft SQL Server, and ensuring secure, efficient, and scalable data access.

Key Responsibilities:

• Support and maintain Microsoft SQL Server environments, with flexibility to adapt to alternative database platforms as the architecture evolves.


• Design and implement data pipelines using AWS Glue for efficient and scalable data ingestion.


• Build and maintain a near real-time data ingestion framework from Oracle databases to Microsoft SQL Server.


• Develop and manage ETL processes, including incremental/delta-based extraction of raw data from source systems.


• Build and optimize relational database schemas, ensuring performance, scalability, and data integrity.


• Implement and manage fine-grained roles and permissions in SQL Server to enforce data security and access control policies.


• Work closely with Data Analysts, Software Engineers, and other stakeholders to ensure data systems meet business requirements.


• Monitor and troubleshoot data pipelines and workflows to ensure reliability and consistency.

Qualifications:

• Proven experience with Microsoft SQL Server administration and development.


• Proficiency in ETL/ELT concepts and experience with tools like AWS Glue, Apache Spark, or equivalent.


• Solid understanding of Oracle database systems and data extraction methodologies (especially delta/change data capture techniques).


• Experience with cloud data platforms, particularly AWS (S3, Glue, RDS, etc.).


• Strong SQL skills and experience with role-based access control in database systems.


• Familiarity with scripting or programming languages (e.g., Python, Shell) for automation and data processing.


• Ability to work independently and communicate effectively in a collaborative team environment.

Preferred Skills (Not Required):

• Proficiency with Oracle GoldenGate or equivalent CDC (Change Data Capture) tools for real-time replication.


• Experience with modern data architectures (e.g., Data Vault 2.0).


• Knowledge of CI/CD practices for data engineering.


• Exposure to data cataloging and metadata management tools.