Technology Lead

Overview: Tek Ninjas is a global software consulting and digital services company built for speed, scale, and innovation, headquartered in Carrollton, Texas.

Job Title: Senior Technology Architect

Location: Raleigh, NC 27601

Duration: 12 months

Job Type: Contract

Work Type: Hybrid

Job Description:

We are looking for an experienced AI & Intelligent Automation Lead Data Engineering to drive the adoption of AI-assisted engineering and intelligent automation across our enterprise data platform.
The role will be responsible for defining and executing an AI-first Data Engineering strategy focused on accelerating development, improving data quality and platform reliability, automating operational activities, reducing technical debt, and optimizing cloud and data-platform costs.
The successful candidate will combine strong Data Engineering and Data Platform architecture expertise with practical experience applying AI capabilities such as GitHub Copilot, Snowflake Cortex and Databricks Genie across the Software Development Lifecycle.

Key Responsibilities

• AI & Intelligent Automation Strategy
• Define and execute the AI-enabled Data Engineering strategy to improve engineering velocity, quality, reliability, scalability and operational efficiency.
• Identify, assess and prioritize high-value AI and automation use cases across Data Engineering, Data Warehouse and BI teams.
• Develop reusable patterns and accelerators for introducing AI into existing Data Engineering and platform operations.
• Build an AI-first engineering culture by adopting driving, enablement, governance and measurable engineering outcomes.
• AI-Assisted Data Engineering
• Lead adoption of GitHub Copilot, Snowflake Cortex and Databricks Genie to accelerate engineering activities and improve developer productivity.
• Implement AI-assisted approaches for generating SQL, Python, PySpark, pipeline code, unit tests, documentation and deployment artifacts.
• Automate generation of standardized, reusable data pipelines and engineering components.
• Target automation of approximately 30 50% of repetitive pipeline development activities, where technically and operationally appropriate.
• Build reusable engineering templates, prompts, coding standards and AI-assisted development patterns.
• Intelligent Data Operations & Self-Healing Pipelines
• Implement AI-driven monitoring and anomaly detection to proactively identify:
• Data quality issues
• Pipeline failures
• Schema changes and drift
• Processing anomalies
• Performance degradation
• Design self-healing data pipelines capable of detecting and automatically resolving known operational issues.
• Apply intelligent automation to incident triage, troubleshooting and root-cause analysis.
• Reduce manual L1/L2 operational effort through automated diagnosis, remediation and preventive monitoring.
• Establish appropriate human-in-the-loop controls for scenarios where automated remediation could create downstream risk.
• Snowflake & Databricks Optimization
• Leverage Snowflake AI and Cortex capabilities to improve engineering productivity and platform intelligence.
• Identify expensive or inefficient queries and workloads and implement appropriate optimization strategies.
• Improve warehouse utilization, workload management, query performance and overall platform cost efficiency.
• Utilize Databricks Genie and Databricks AI capabilities to improve data discovery, analytics accessibility and engineering productivity.
• Work with platform teams to establish measurable FinOps outcomes around compute consumption and cloud cost optimization.
• AI-Enabled SDLC
• Champion the use of AI across the complete engineering lifecycle, including:
• Requirements analysis and refinement
• Solution and pipeline design
• Code generation and refactoring
• Unit and integration testing
• SQL and PySpark optimization
• Code reviews
• Technical documentation
• CI/CD and deployment automation
• Production monitoring
• Incident analysis
• Root-cause analysis
• Technical debt identification and remediation
• Ensure AI-generated outputs are appropriately reviewed, tested and integrated into existing engineering standards.
• Responsible AI & Governance
• Define standards and guardrails for responsible use of Generative AI within Data Engineering.
• Establish controls covering:
• Data privacy
• Information security
• Regulatory compliance
• Intellectual property
• Code quality
• AI-generated code validation
• Accuracy and hallucination risk
• Maintainability and auditability
• Ensure sensitive enterprise information is protected when engineers use AI-assisted development tools.
• Work with architecture, security, risk and governance teams to establish approved AI engineering patterns.

Metrics & Value Realization

• Define and maintain measurable KPIs for AI adoption and engineering effectiveness, such as:
• AI tool adoption across engineering teams
• Developer productivity improvements
• Percentage of repetitive activities automated
• Development cycle-time reduction
• Pipeline failure ####
• Mean Time to Detect (MTTD)
• Mean Time to Resolve (MTTR)
• Data-quality incident reduction
• Technical-debt reduction
• Cloud and platform cost savings
• Reuse of standardized engineering components
• The role will be expected to demonstrate measurable business value from AI adoption rather than focusing only on tool deployment.

Required Skills & Experience

• 10+ years of experience in Data Engineering, Data Platforms, Data Warehousing or related technology roles.
• Strong hands-on experience with enterprise-scale Data Engineering and Data Warehouse platforms.
• Strong expertise in Snowflake and/or Databricks.
• Advanced knowledge of SQL, Python and/or PySpark.
• Experience designing and optimizing ETL/ELT pipelines and large-scale data-processing solutions.
• Experience with AI-assisted engineering tools such as:
• GitHub Copilot
• Snowflake Cortex
• Databricks Genie
• Other enterprise Generative AI/code-assistance capabilities
• Strong understanding of Snowflake performance optimization, workload management and cost optimization.
• Experience with CI/CD, DevOps, automated testing and modern Data Engineering SDLC practices.
• Experience implementing data observability, monitoring, anomaly detection or automated remediation.
• Understanding of Generative AI, LLMs, prompt engineering and enterprise AI governance.
• Ability to translate AI capabilities into practical engineering solutions and measurable business outcomes.
• Strong stakeholder management and communication skills, with the ability to influence engineering, architecture and leadership teams.

Preferred Skills

• Experience working with Azure/AWS enterprise data platforms.
• Experience with Azure Data Factory, dbt, Airflow or similar orchestration technologies.
• Knowledge of data observability and data-quality frameworks.
• Experience building reusable Data Engineering accelerators or frameworks.
• Understanding of MLOps/LLMOps, RAG and AI agent/agentic workflow concepts.
• Experience implementing enterprise AI governance and secure Generative AI adoption.
• Exposure to DataOps, DevOps, FinOps and Platform Engineering practices.

Minimum Years of Experience:

• 8 to 10+ years

About TekNinjas

TekNinjas is a global IT staffing partner placing skilled professionals with leading enterprises and system integrators across the US, Canada, UK, India, and the EU. We focus on the right fit - not just the fast fill - and stay engaged with our consultants well beyond the start date.

We're the right partners in your success.

TekNinjas is an Equal Opportunity Employer.