Observability Engineer

Enterprise Observability Program | Dynatrace APM on OpenShift, VM and COTS workloads

Engagement length: up to 20 weeks Delivery: remote, with travel limited to the kickoff Team: Delivery Lead, Technical Lead, Observability Engineer, Technical Principal

The role

Help a large U.S. energy utility see how its most important applications actually perform. You will be the hands-on engineer who gets Dynatrace running in production, instruments three Tier 1 applications, and builds the automation that lets the client's own teams onboard the rest without us.

You will work under the Principal Dynatrace Architect and the Technical Lead, in two-week sprints, alongside the client's platform, application and NOC teams.

What you will do

Instrument three application patterns. Deploy OneAgent on a custom OpenShift application, a COTS application on OpenShift (IBM Maximo / PAMS), and a COTS application on traditional VMs (). Resolve instrumentation issues with the client's teams.

Configure collectors and agents. Set up Dynatrace Operator DynaKube resources, host-based agents and Grafana Alloy collectors to the deployment specifications from Planning and Design.

Build deployment automation. Create and validate OneAgent rollout for the OpenShift Operator and VM-hosted patterns, delivered to the client's Azure DevOps environment and integrated with ArgoCD where it applies. Confirm automated deployments match manual ones.

Validate signal quality. Confirm traces, metrics, logs and events are accurate, complete and consistent with the telemetry output specification across all three applications before handoff.

Support the observability layer. Help configure dashboards, alert policies and Real User Monitoring, and support alert tuning and SLO baselines.

Document and hand over. Contribute to the runbook library and automation documentation, with enough README and operating guidance for the client to maintain it alone. Support the two knowledge transfer sessions.

Experience and minimum years

The SOW sets the responsibilities of this role but not years of experience. The minimums below are our recommended screening thresholds, based on what each responsibility requires.

Skill area

Minimum years

Where the SOW needs it

Observability / monitoring engineering (overall)

2+

Hands-on delivery across APM, infrastructure and log monitoring in enterprise environments; owns instrumentation work across all three application patterns.

Dynatrace SaaS platform

2+

OneAgent, ActiveGate, host groups, process group detection, tagging, management zones or segments and IAM, dashboards, alert policies, anomaly detection and SLOs. Production tenant promotion.

Kubernetes / Red Hat OpenShift

2+

Namespaces, security context constraints, admission control, operators. Needed to run the OCP-native and COTS-on-OpenShift (IBM Maximo / PAMS) instrumentation.

Dynatrace Operator and DynaKube

2+

Configure DynaKube resources; choose between classicFullStack and cloudNativeFullStack; validate agent injection and rollout.

Deployment automation and CI/CD

2+

Azure DevOps pipelines and ArgoCD / GitOps; build repeatable OneAgent rollout for OpenShift and VM patterns with rollback and validation.

Scripting and infrastructure as code

2+

Python, PowerShell or Bash, plus Ansible, Helm or Terraform for agent lifecycle management.

Linux and Windows host administration

2+

Host-based OneAgent install, OS compatibility, version selection, and process and log troubleshooting on VMs ( pattern).

Enterprise networking for agents

2+

Firewall rules, proxies, SSL inspection bypass and connectivity validation from on-premises hosts to a SaaS cluster.

Log ingestion and management

2+

Log sources, parsing rules, indexing, storage and retention design.

Real User Monitoring and synthetics

2+

Browser agent injection, session replay, user action naming and real-user performance baselines.

OpenTelemetry and collectors (Grafana Alloy)

2+ / 1+

2+ years OpenTelemetry concepts and telemetry pipelines; 1+ year configuring Grafana Alloy collectors.

SLOs, alert tuning and signal validation

2+

Establish SLO baselines, reduce alert noise to agreed targets, and validate trace, metric, log and event quality.

COTS application instrumentation

2+

Instrument vendor stacks with limited access (Java application servers, packaged enterprise apps); document constraints and workarounds.

Technical documentation and knowledge transfer

2+

Runbooks, READMEs, instrumentation standards and live KT sessions for engineering and NOC teams.

Nice-to-haves

Dynatrace certification (Associate or Professional), Certified Kubernetes Administrator, or Red Hat OpenShift administration certification.

Exposure to Cribl Stream, BigPanda or ServiceNow event integration, and to Azure or AWS observability. The program plans to integrate these later.

Experience with SSO integration (SAML or OIDC) for Dynatrace SaaS, and with Dynatrace licensing and cost governance (host units and DEM units).

Experience in utilities, energy or other regulated industries.

How you work

You are comfortable in agile delivery: two-week sprints, a Definition of Ready and Done, and demos with testing evidence.

You explain technical trade-offs plainly to engineers, application owners and NOC staff.

You document as you build, so the client can run what you hand over.

Requirements

Must be a US citizen and reside in the US

Ability to pass the client's background check and remote-access onboarding.

Availability during the client's business hours, Monday to Friday, 8:00 a.m. to 5:00 p.m. local time.

Bachelor's degree in computer science, information systems or equivalent practical experience. d Job Description