Data Scientist

Title/Role: IT : DATSCIV1 : Data Scientist: DSBB
Location:Indianapolis IN
Duration: 12 months
Employment Type : Contract on W2 (Need US Citizens or GC Holders or GC EAD or OPT or EAD or CPT)

Note:
Pay Rate: $43.35/hr

Job Short Description:

• Designs builds and maintains secure scalable data solutions across Azure Databricks and Azure Synapse Analytics.

Job Complete Description:

Local candidates only will be considered for this position.

• This role supports data integration pipeline development data modeling governance and documentation while collaborating with cross-functional teams to deliver highquality reliable data products for the agency.
• Plays a key role in implementing the agencys data strategy by developing batch and streaming pipelines optimizing data warehouse workloads and enforcing data governance standards.
• The position assists with requirements gathering solution reviews and technical support and works closely with data architects scientists analysts and project managers to translate business needs into technical designs.
• This role supports operational processes contributes to policy and procedure improvements and may provide mentorship or supervision based on agency needs.

Job Duties

• Assists with the data technology strategy including resolving strategic data integration issues such as data quality and stewardshipreal-time
• Event-based data integration and crafting a universal vision for data integration
• Provides direction and oversight for the design and implementation of data solutions for integration and synthesis across multiple secure systems
• Assists with the definition implementation and continued adherence of new data management policies and design standards for data engineering
• Provides advice training and technical support for various projects
• Participates in project activities such as requirements gathering solution reviews and explaining technical complexities to non-technical personnel
• Work closely with the solution leads project managers data architects data scientists and data analysts on solution design architecture and implementation
• Assists with extraction transformation and loading of data from a wide variety of data sources using various data engineering tools and methods.
• Designs builds and maintains batch and streaming data pipelines in Azure Databricks using Py Spark Spark SQL and Delta Lake following a medallion (bronze/silver/gold) lake house architecture
• Develops loads and tunes dimensional data models and SQL workloads in Azure Synapse Analytics (dedicated and serverless SQL pools) to support reporting and dashboard needs
• Orchestrates and monitors data workflows using Databricks Workflows/Jobs and Synapse or Azure Data Factory pipelines including scheduling alerting and failure recovery
• Applies data governance and security controls (e.g. Unity Catalog permissions role-based access row- and column-level security data masking) to protect sensitive health data (PHI/PII) in accordance with HIPAA and agency policy
• Assists with documenting and demonstrating solutions by developing documentation flowcharts layouts diagrams charts code comments and clear code.
• Assists in operations by conducting systems analysis; recommending changes in policies and procedures.
• The job description is not designed to cover or contain a comprehensive listing of activities duties or responsibilities that are required of the employee. Other duties responsibilities and activities may change or be assigned at any time.

Skills Required

• Masters degree in data science analytics informatics computer science mathematics engineering or related field; OR.... - 4 Years
• Bachelors degree in a related field plus 2 years of relevant experience; or 4 years of hands-on experience in a similar data engineering role; OR.. - 4 Years
• Relevant certifications and coursework may substitute for formal education/experience requirements - 4 Years
• Developing with Databricks notebooks in PySpark Spark SQL and R including transformations joins aggregations and window functions - 4 Years
• ACID transactions MERGE upserts schema enforcement/evolution time travel and table maintenance (OPTIMIZE Z-order VACUUM) - 4 Years
• Working with medallion (bronze/silver/gold) architecture and incremental ingestion via Auto Loader Structured Streaming or Lakeflow/DLT - 4 Years
• Using Unity Catalog for Workspace Catalogs schemas permissions lineage external locations and storage credentials - 4 Years
• Right-sizing clusters and SQL warehouses and diagnose partitioning shuffle skew and caching issues via Spark UI - 4 Years
• Scheduling and managing dependencies with ADF Pipelines Databricks Workflows (Jobs) and using Git folders (Repos) for source control - 4 Years
• Designing MPP warehouse tables including distribution (hash round-robin replicated) columnstore indexes and partitioning - 4 years
• Querying Parquet Delta and CSV files in ADLS Gen2 using OPENROWSET external tables and views - 4 years
• Building star-schema facts dimensions stored procedures and views including SCD Type 1 and Type 2 4 Years
• Developing with COPY INTO PolyBase and CETAS and building parameterized Synapse/ADF pipelines (copy data flows triggers) - 4 years
• Workload management resource classes statistics materialized views result-set caching and pool scaling/pausing - 4 Years
• ADLS Gen2 folder/file-format design (Parquet/Delta) Entra ID access control managed identities and Key Vault - 4 Years
• Git and CI/CD in Azure DevOps or GitHub including branching PRs and deploying Synapse artifacts and Databricks Asset Bundles - 4 Years
• Monitoring pipelines and workloads with Azure Monitor/Log Analytics and alerting and troubleshooting failed jobs - 4 Years
• Strong written and verbal communication skills
• Using Microsoft lakehouse patterns including interoperability and migration paths between Synapse and Databricks - 4 Years

Skills Highly Desired:

• Databricks Certified Data Engineer Associate (Databricks Certified Data Engineer Professional a plus)
• Microsoft Certified: Azure Data Engineer Associate (DP-203) if already held
• Power BI connectivity to Synapse (Import vs. Direct Query) and how model design affects dashboard performance - 4 Years