Data Scientist

TxDOT work to be accomplished

Resource with strong skills in Data architecture and expertise in dimensional data modeling preferably in a bigdata or an enterprise data warehouse environment. Experience with Informatica Power Exchange for CDC based replication or similar tool is a plus. Expertise in development of data ingestion to Hadoop based data lake (HDFS Hive) and S3 storage complex data transformations and data security. Programming expertise in a major programming/scripting language including but not limited to Java C Scala python Go. Hands on delivery capability working in a team across the full lifecycle Data Source Analysis Design Development Testing and Implementation. This includes providing input into requirements platform development technical design of the project level technical architecture for ETL big data application design and development testing and deployment of the proposed solution. Proficient in Hadoop ecosystem components such as Hive Yarn Knox Ranger Sqoop Oozie HiveQL Spark SQL/data access. Experience in ETL development using Informatica BDM Kafka or comparable software. Understanding and skills in data governance security architecture load balancing and troubleshooting. Well-versed and deep development experience in Big Data Platform clusters on AWS HDFS storage optimization Data Lake/Data Warehouse foundation capabilities.

The Data architect will work with a team of developers analysts project managers and solution architects. It is expected that this person shall have a good understanding of Snowflake cloud data warehouse architectures including loading to dimensional data structures and ETL development for many types of data such as RDBMS sources API/streaming structured and unstructured. Design develop support optimize and provide technical oversight on all phases of the development life cycle on data warehouse project engagements including but not limited to participation in data sourcing analysis requirements gathering design development testing deployment transition and support. Help monitor and manage the technical scope on engagements to prevent scope creep and deviation from the contract. Follow good practices and project guidelines in all cases including the use of source code control automated testing and deployment approaches test-case-based development and proper documentation practices.

Minimum Yrs of Experience Skills and Qualifications

6 years of Cloud Data Warehousing with specialization in Snowflake and Python

Preferred Skills and Qualifications

1 year of prior experience with State Agencies