Overview:

TekWissen is a global workforce management provider headquartered in Ann Arbor, Michigan that offers strategic talent solutions to our clients world-wide. Our client is an American organization that operates Texas's electrical grid, the Texas Interconnection, which supplies power to more than 25 million Texas customers and represents 90 percent of the state's electric load. Our client is the first independent system operator (ISO) in the United States and one of nine ISOs in North America. Our client works with the Texas Reliability Entity (TRE), one of eight regional entities within the North American Electric Reliability Corporation (NERC) that coordinate to improve reliability of the bulk power grid. Our client is a membership-based 501(c) nonprofit corporation, and its members include consumers, electric cooperatives, generators, power marketers, retail electric providers, investor-owned electric utilities (transmission and distribution providers), and municipally owned electric utilities.

Duration: 11+ Months

• Client is seeking an experienced Azure Databricks Architect to lead the design, implementation, and optimization of our enterprise-wide data platform.
• The ideal candidate will be responsible for creating a scalable, secure, and high-performance data ecosystem that enables advanced analytics, machine learning, and business intelligence capabilities.
• Aligns solutions following Client operating procedures and Architecture principles and works closely with internal technical teams.
• Performs proof of concepts to evaluate and prove out non-functional requirements and options.
• Uses toolsets that adhere to the frameworks of the technical stack complying with the organization's standards.
• Provides mentorship and technical guidance to transition work to other Client operational teams.

• Design and architect comprehensive data solutions using Azure Databricks, Azure Data Lake, Azure Event Hub, and related Azure data services
• Develop and implement data Lakehouse architecture patterns that support both batch and real-time data processing
• Create robust data ingestion pipelines leveraging Azure Databricks, Azure Data Factory, and other ETL technologies.
• Establish best practices for data governance, security, and compliance across the data platform.
• Design and implement advanced analytics and machine learning workflows using Databricks.
• Develop performance optimization strategies for large-scale data processing.
• Create and maintain infrastructure-as-code (IaC) templates for reproducible and scalable data platform deployments.
• Provide architectural guidance and mentorship to data engineering and analytics teams.
• Ensure alignment of data platform architecture with overall enterprise architecture and business objectives
• Develops prototypes to prove out designs and delivery patterns.
• Fosters collaboration with cross-functional teams including Development, Operations, Compliance, Security, etc.
• Understands and applies principles, theories and concepts related to the profession and Client's culture.
• Provides mentoring and guidance to other team members.

• Conveys team goals effectively through strong written and verbal communication skills
• Ability to execute end-to-end testing for various non-functional requirements
• Has the ability to resolve design and development issues in creative and effective ways
• Researches, evaluates, and incorporates new technology and tools to enhance the product development and platform operations
• Gives advice and recommendations to management based on experience
• Expands technical proficiency in key technical areas

• Requires minimum 5 years related work experience in excess of degree requirements

• Advanced proficiency with Azure Databricks and ability to provide best practices for a large organization.
• Expert-level knowledge of:
  • Python
  • SQL
  • Spark (PySpark) and Delta Lake
  • Azure Data Lake Storage
  • Azure Databricks
• Strong understanding of data Lakehouse architecture principles, and cloud-native data processing and analytics solutions
• Proficiency in infrastructure-as-code tools (Terraform, Github Actions) and integration with Github Enterprise for Code and Infrastructure deployments.
• Advanced knowledge of data modeling, performance tuning, and optimization techniques
• Experience with securing Azure Databricks, especially for regulated industries.
• Design and validate best practices for creating separate workspaces, catalogs, schemas, workflows.
• Experience with design and implementation of ETL pipelines with Azure Databricks
• Experience with design of Observability and Cost Management for the platform.
• Experience with design and implementation of Batch and Real time streaming data pipelines.
• Experience with design and implementation of MLOps patterns for machine learning use cases.
• Experience with integration of Databricks with external Governance tools like Purview.
• Experience migrating data and code to use the new architecture and operationalizing the platform.
• Experience with design and implementation of DR strategy for Databricks across multiple regions.

• Bachelor's Degree: Computer Science or related field (Required) OR
• A combination of education and experience that provides equivalent knowledge to a major in such fields is required

TekWissen Group is an equal opportunity employer supporting workforce diversity.