Likeremote

Subscribe to the latest remote jobs:

  • Likeremote jobs on https://LinkedIn.com/
  • Likeremote jobs on https://telegram.org/
  • Likeremote jobs on Reddit.com
PL

AWS Observability Architect

PeopleNTech LLC
🇺🇸 United States
On-site
3 months ago
  • AWS
  • Grafana
  • OpenTelemetry
  • EC2
  • EKS
  • ECS
  • AIOps
  • FinOps
  • RBAC
  • CloudWatch
  • Mimir
  • Prometheus
  • Loki
  • OpenSearch
  • Elasticsearch
  • Incident Response
  • PagerDuty
  • Opsgenie
  • ServiceNow
  • Incident Management
  • YAML
  • JSON
  • Terraform
  • GitOps
  • Jaeger
  • Zipkin
  • X-ray
  • Fluent Bit
  • Java
  • Python
  • Node.js
  • Microservices
  • CloudTrail
  • VPC
  • RDS
  • API Gateway
  • Datadog
  • New Relic
  • Splunk
  • Nagios
  • Zabbix
  • Devops
  • Kubernetes
3 months ago
Auto apply with Likeremote
Not scoredNo CV on file. Upload one and this job gets a score out of 100.Upload CV
AWS Observability Architect
Location: Warren, NJ (Onsite)
Fulltime ($160 - $170K PA) Don't stop profiles on Salary
Indent: TBC (will share tomorrow)

We are seeking a highly skilledAWS Observability Architect with deep, hands-on expertise in designing and implementing enterprise-grade observability platforms on AWS — withGrafana as the primary observability tool andOpenTelemetry as the instrumentation standard. This is a technical specialist role requiring genuine implementation experience, not platform familiarity.
The ideal candidate has personally architected and delivered large-scale observability solutions for production AWS environments — building telemetry pipelines, designing dashboards that operations teams actually use, and creating alerting frameworks that reduce MTTR rather than add noise. You understand the full observability stack: from application instrumentation with OpenTelemetry SDKs through to Grafana dashboards consumed by SREs, on-call engineers, and engineering leadership.
This role sits at the intersection of cloud infrastructure, software engineering discipline, and operational excellence — requiring someone who can design an enterprise observability architecture in the morning, write a Grafana dashboard query in the afternoon, and advise a development team on OpenTelemetry instrumentation strategy the next day.

Key Responsibilities
Observability Architecture & Strategy
  • Define and own theenterprise observability architecture for AWS environments — establishing the target-state design across the four pillars of observability: metrics, logs, traces, and events.
  • Designend-to-end telemetry pipelines — from instrumentation at the application and infrastructure layer through collection, processing, storage, and visualisation — with Grafana as the enterprise observability platform.
  • Developobservability standards and reference architectures — defining how AWS workloads across compute (EC2, EKS, ECS, Lambda), storage, networking, and managed services should be instrumented, collected, and visualised consistently across the organisation.
  • Establishsignal-to-noise discipline across the observability platform — designing alerting frameworks that surface actionable signals, eliminate false positives, and ensure on-call engineers are alerted only when human intervention is genuinely required.
  • Defineobservability maturity roadmaps for client environments — assessing current-state coverage, identifying gaps, and building a phased improvement plan from reactive monitoring to proactive, AIOps-ready observability.
  • DriveFinOps for observability — optimising telemetry data volumes, retention policies, and Grafana Enterprise licensing costs to ensure the observability platform itself does not become a significant cost centre.

Grafana Enterprise Implementation
  • Architect, deploy, and operateGrafana Enterprise orGrafana SaaS as the primary observability platform — including high-availability Grafana deployment on AWS (EKS-based or managed via Grafana Cloud), data source federation, RBAC configuration, and enterprise plugin management.
  • Design and implementGrafana data source integrations across the AWS observability ecosystem:
    • Amazon CloudWatch — metrics, logs, and alarms as a core AWS data source
    • Grafana Mimir — for scalable, long-term Prometheus-compatible metrics storage
    • Grafana Loki — for cost-efficient, label-based log aggregation at scale
    • Grafana Tempo — for distributed tracing storage and trace-to-log-to-metric correlation
    • Amazon Managed Service for Prometheus (AMP) — for AWS-native Prometheus metrics
    • Amazon OpenSearch — for log analytics and full-text search use cases
    • Elasticsearch / OpenSearch — for existing log infrastructure integration
  • Build and maintain aGrafana dashboard library — covering infrastructure health, application performance, SLO/SLA tracking, capacity planning, cost visibility, incident response, and executive reporting — using reusable, variable-driven, and consistently styled templates.
  • ImplementGrafana alerting at enterprise scale — including alert routing, notification policies, silence management, and integration with PagerDuty, OpsGenie, ServiceNow, and Slack for multi-channel incident notification.
  • ConfigureGrafana RBAC and team structures — designing role hierarchies, folder permissions, and data source access controls that enable self-service dashboarding for development teams while protecting sensitive operational data.
  • Deploy and manageGrafana Oncall for on-call scheduling and alert routing, or integrate Grafana alerting with existing incident management platforms.
  • ImplementGrafana SLO (Service Level Objectives) — defining, tracking, and reporting error budgets across production services, enabling data-driven reliability decisions.
  • ManageGrafana as code — using Grafana's provisioning capabilities (YAML/JSON), Terraform provider, and Grizzly/Grafonnet for dashboard version control, environment promotion, and GitOps-based dashboard management.

OpenTelemetry Implementation
  • Define and lead the organisation'sOpenTelemetry (OTel) instrumentation strategy — establishing standards for automatic and manual instrumentation across application stacks running on AWS.
  • Design and deploy theOpenTelemetry Collector as the central telemetry processing layer — including:
    • Collector deployment patterns: agent (DaemonSet on EKS), gateway (centralised), and sidecar configurations
    • Receiver configuration — OTLP, Prometheus, Jaeger, Zipkin, AWS X-Ray, CloudWatch, Fluent Bit
    • Processor pipeline design — batch processing, memory limiting, attribute enrichment, tail-based sampling, and resource detection processors
    • Exporter configuration — routing telemetry to Grafana Mimir (metrics), Grafana Loki (logs), Grafana Tempo (traces), AMP, and CloudWatch
  • InstrumentAWS workloads with OpenTelemetry SDKs across languages (Java, Python, Node.js, Go) — including auto-instrumentation for containerised EKS workloads, Lambda instrumentation using OTel Lambda layers, and ECS task definition instrumentation.
  • Implementdistributed tracing using OpenTelemetry — establishing trace propagation standards across microservices, configuring context propagation (W3C TraceContext, B3), and ensuring end-to-end trace visibility from frontend to backend to database.
  • DesignOTel-based log correlation — enriching logs with trace IDs and span IDs to enable trace-to-log navigation in Grafana, supporting faster RCA during incidents.
  • ImplementOTel-based metric instrumentation — defining custom business and application metrics alongside system metrics, following OTel semantic conventions for consistent metric naming and attribute tagging across services.
  • Definesampling strategies for distributed traces — including head-based sampling for development environments and tail-based sampling (via OTel Collector) for production environments, balancing observability coverage with storage cost.
  • ManageOTel Collector as infrastructure — including horizontal scaling, resource limits, high-availability deployment, collector health monitoring, and pipeline performance optimisation.

AWS Observability Services Integration
  • Design theintegration architecture between AWS-native observability services and Grafana — positioning Grafana as the unified observability plane while leveraging AWS-native services as data sources:
    • Amazon CloudWatch — metrics, logs, alarms, dashboards, Contributor Insights, and Synthetics
    • Amazon Managed Grafana (AMG) — evaluating and advising on AMG vs self-managed Grafana deployment decisions
    • Amazon Managed Service for Prometheus (AMP) — remote write from OTel Collector and Prometheus agents, recording rules, and alert manager integration
    • AWS X-Ray — ingesting X-Ray traces into Grafana Tempo or directly via Grafana X-Ray data source
    • AWS CloudTrail — audit log integration for security and compliance observability
    • VPC Flow Logs — network observability integration for security monitoring and traffic analysis
  • Implementinfrastructure-level observability for core AWS services — EC2 (CloudWatch agent, Node Exporter via OTel), EKS (kube-state-metrics, cAdvisor, OTel DaemonSet), RDS (Enhanced Monitoring, Performance Insights), Lambda (OTel Lambda layer, custom metrics), and API Gateway (access logs, CloudWatch metrics).
  • Designbusiness and synthetic monitoring — implementing Grafana Synthetic Monitoring or CloudWatch Synthetics for endpoint availability, API health, and user journey monitoring with Grafana alerting integration.

Delivery & Enablement
  • Leadobservability implementation projects end-to-end — from requirements gathering and architecture design through deployment, dashboard development, alert tuning, and team enablement.
  • Conductobservability maturity assessments for client environments — evaluating current monitoring coverage, tool sprawl, alert quality, and SLO definition maturity, and producing prioritised remediation roadmaps.
  • Develop and deliverobservability enablement workshops for engineering and operations teams — covering OTel instrumentation, Grafana dashboard development, alert design, and on-call best practices.
  • Produceobservability architecture documentation — reference architectures, runbooks, onboarding guides, and dashboard documentation that enable teams to self-serve and maintain the platform.
  • Advise onobservability tool consolidation — helping organisations rationalise fragmented monitoring estates (Datadog, New Relic, Splunk, Nagios, Zabbix) toward a unified Grafana + OTel platform, including migration planning and cost impact analysis.

Experience
  • 10+ years of overall experience in cloud infrastructure, platform engineering, or DevOps.
  • 5+ years of hands-on AWS experience in production environments — not advisory or oversight roles.
  • 3+ years of hands-on Grafana Enterprise or SaaS implementation experience — designing, deploying, and operating Grafana at enterprise scale, including Mimir, Loki, Tempo, and the LGTM stack.
  • Proven experience implementingOpenTelemetry in production environments — including OTel Collector deployment, SDK-based instrumentation, and distributed tracing implementation.
  • Demonstrated experience buildingproduction-grade observability pipelines — from instrumentation through collection, processing, storage, and visualisation.
  • Hands-on experience withPromQL for metrics querying and alerting — including complex queries, recording rules, and alert expression design.
  • Experience withLogQL (Grafana Loki) for log querying and log-based alerting.
  • Hands-on experience deployingobservability infrastructure on Kubernetes (EKS) — including Prometheus Operator, OTel DaemonSets, Grafana deployment, and persistent storage configuration.
  • Experience withGrafana as code — provisioning dashboards, data sources, and alert rules via YAML, Terraform, or Grafonnet.

AWS Observability Architect · PeopleNTech LLC

Auto apply with Likeremote