Data Engineer II OIT

Emory University is a leading research university that fosters excellence and attracts world-class talent to innovate today and prepare leaders for the future. We welcome candidates who can contribute to the excellence of our academic community.

The Data Engineer II is a key contributor to the development and evolution of the Unified Data Platform (UDP) built on Azure Fabric. This role works closely with researchers architects and cross-functional teams to design and deliver scalable high-quality data solutions that support research analytics and clinical insights. The position involves working with complex datasets modern cloud data platforms and emerging technologies. While experience with Azure Fabric is preferred candidates with strong hands-on experience in Azure Synapse or Databricks with PySpark will be equally considered. This role requires a strong foundation in data engineering biomedical informatics principles data governance and compliance standards such as HIPAA. The ideal candidate is a collaborative team player who can translate business and research requirements into efficient and scalable technical solutions.

JOB DESCRIPTION:

• Collaborate as a core member of cross-functional teams including Business Analysts Project Managers Data Analysts and Architects to deliver high-quality solutions within scope and timeline.
• Work directly with researchers and stakeholders to gather analyze and translate requirements into technical solutions.
• Design develop and maintain scalable data pipelines ETL/ELT processes and data integration workflows across multiple systems.
• Build and optimize data solutions that integrate data from disparate sources ensuring data quality integrity and consistency.
• Evaluate emerging technologies and develop proof-of-concepts to support innovation and continuous improvement.
• Apply biomedical informatics standards methodologies and principles to research data solutions.
• Ensure adherence to HIPAA and institutional data governance policies and standards.
• Develop and maintain metadata data standards and data governance processes for complex datasets.
• Create and maintain clear technical documentation including data pipelines workflows and system designs.
• Communicate technical concepts effectively to both technical and non-technical stakeholders.
• Partner with data and system architects and demonstrate an understanding of data modeling concepts and best practices.
• Manage workload effectively and provide timely updates on task progress and deliverables.
• Works as a positive team member of a project that may consist of Business Analysts Project Managers Information Architects Data Analysts and/or Database Administrators to deliver quality applications and components within scope on time and within budget.
• Manages workload effectively and report status of tasks in a timely manner.
• Works directly with researchers to document analyze and translate their needs into technical designs and informatics solutions.
• Participates in the evaluation of emerging technologies and develops proof-of-concepts.
• Contributes to technical teams.
• Follows standard operational procedures and HIPAA regulations.
• Develops strategies for managing complex data sets through maintaining data standards and metadata.
• Applies biomedical informatics technical standards methodologies and principles to research-specific program needs objectives and outcomes.
• Develops complex reports data pipelines and ETL processes from disparate systems and ensures their accuracy.
• Gathers user requirements and creates technical documentation
• Performs other related duties as required.

MINIMUM QUALIFICATIONS:

• A bachelors degree in a related field and three years of related experience OR an equivalent combination of education training and experience.

PREFERRED QUALIFICATIONS:

• Strong experience designing and maintaining data pipelines and ETL/ELT processes
• Proficiency in Python SQL and Apache Spark (PySpark)
• Experience with modern data platforms such as Azure Fabric (preferred) Azure Synapse Serverless or Databricks
• Ability to work with large complex and distributed datasets
• Solid understanding of data modeling concepts and best practices
• Familiarity with data governance metadata management and data quality frameworks
• Knowledge of HIPAA and healthcare data compliance standards
• Experience with CI/CD practices and proficiency with version control systems like Git
• Familiarity with data pipeline orchestration and monitoring tools Like Airflow Azure Data Factory or similar frameworks
• Understanding of biomedical informatics methodologies is a plus
• Strong analytical problem-solving and critical-thinking abilities
• Excellent communication and collaboration skills across diverse teams
• Ability to adapt quickly to new technologies and evolving data environments
• Exposure to FHIR OMOP or similar healthcare data models (nice-to-have)

NOTE: Tasks related to this position can be performed remotely with only occasional visits to an Emory University location. Eastern (EST) time zone business hours may apply. Emory reserves the right to change this status with notice to employee. Emory does not approve as a primary work location in the following states; NJ AK and HI any U.S. Territories or outside of the United States.

Emory is an equal opportunity employer and qualified applicants will receive consideration for employment without regard to race color religion sex national origin disability protected veteran status or other characteristics protected by state or federal law. Emory University does not discriminate in admissions educational programs or employment including recruitment hiring promotions transfers discipline terminations wage and salary administration benefits and training. Students faculty and staff are assured of participation in university programs and in the use of facilities without such discrimination. Emory University complies with Section 503 of the Rehabilitation Act of 1973 the Vietnam Era Veterans Readjustment Assistance Act and applicable executive orders federal and state regulations regarding nondiscrimination equal opportunity and affirmative action (for protected veterans and individuals with disabilities). Inquiries regarding this policy should be directed to the Emory University Department of Equity and Civil Rights Compliance 201 Dowman Drive Administration Building Atlanta GA 30322. Telephone: (V) (TDD).

Emory University is committed to ensuring equal access and providing reasonable accommodations to qualified individuals with disabilities upon request. To request this document in an alternate format or to seek a reasonable accommodation please contact the Department of Accessibility Services at or call (Voice) (TDD). We kindly ask that requests be made at least seven business days in advance to allow adequate time for coordination.