<!-- source: https://deepwiki.com/qhkm/zeptoclaw/11.4-health-monitoring -->

# Health Monitoring

- src/api/middleware.rs
- src/api/mod.rs
- src/api/routes/mod.rs
- src/api/server.rs
- src/cli/serve.rs
- src/health.rs
- src/utils/metrics.rs
- src/utils/mod.rs
- src/utils/slo.rs

## Purpose and Scope

This document describes ZeptoClaw's health monitoring infrastructure, which provides real-time observability into the running system. It covers:

- HealthRegistry — Named component health checks with status tracking.
- UsageMetrics — Lock-free request/token counters for telemetry.
- Health HTTP endpoints — /health and /ready for liveness and readiness probes, plus Panel API integration.
- Session Metrics & SLOs — Per-session performance tracking and Service Level Objective evaluation.
- Platform memory tracking — RSS measurement on macOS and Linux.

For cron-based scheduled tasks and background job management, see [11.2](https://github.com/qhkm/zeptoclaw/blob/2c743d31/11.2) For channel supervision and restart logic, see [7.1](https://github.com/qhkm/zeptoclaw/blob/2c743d31/7.1)

## Architecture Overview

ZeptoClaw implements a multi-tier health monitoring system: a **HealthRegistry-based** server for detailed component checks, **UsageMetrics** for global counters, and **Panel API** routes for remote monitoring.

### System Data Flow

```

```

**Sources:** [src/health.rs203-207](https://github.com/qhkm/zeptoclaw/blob/2c743d31/src/health.rs#L203-L207) [src/health.rs356-384](https://github.com/qhkm/zeptoclaw/blob/2c743d31/src/health.rs#L356-L384) [src/utils/metrics.rs51-56](https://github.com/qhkm/zeptoclaw/blob/2c743d31/src/utils/metrics.rs#L51-L56) [src/utils/slo.rs39-56](https://github.com/qhkm/zeptoclaw/blob/2c743d31/src/utils/slo.rs#L39-L56) [src/api/server.rs43-48](https://github.com/qhkm/zeptoclaw/blob/2c743d31/src/api/server.rs#L43-L48)

## HealthRegistry System

The `HealthRegistry` maintains a registry of named component health checks, enabling fine-grained observability into subsystem status.

### Core Types

**Defined in:** [src/health.rs136-143](https://github.com/qhkm/zeptoclaw/blob/2c743d31/src/health.rs#L136-L143) [src/health.rs161-172](https://github.com/qhkm/zeptoclaw/blob/2c743d31/src/health.rs#L161-L172) [src/health.rs203-207](https://github.com/qhkm/zeptoclaw/blob/2c743d31/src/health.rs#L203-L207)

### Component Lifecycle

Components register themselves at startup and update their status throughout the process lifetime. The registry drives the `/ready` response; if any check is in `Down` status, the system is considered unready.

**Sources:** [src/health.rs254-263](https://github.com/qhkm/zeptoclaw/blob/2c743d31/src/health.rs#L254-L263)

## UsageMetrics Tracking

`UsageMetrics` provides **lock-free atomic counters** for gateway-level telemetry. This is distinct from per-session metrics and tracks the lifetime of the gateway process.

### Counter Fields

**Defined in:** [src/health.rs356-384](https://github.com/qhkm/zeptoclaw/blob/2c743d31/src/health.rs#L356-L384)

### Periodic Flush

The system includes `start_periodic_usage_flush`, which emits a `usage_summary` log event every 60 seconds (default).

**Sources:** [src/health.rs28](https://github.com/qhkm/zeptoclaw/blob/2c743d31/src/health.rs#L28-L28) [src/health.rs584-615](https://github.com/qhkm/zeptoclaw/blob/2c743d31/src/health.rs#L584-L615)

## Session Metrics & SLOs

While `UsageMetrics` tracks the global state, ZeptoClaw uses `MetricsCollector` and `SessionSLO` for granular session-level monitoring.

### MetricsCollector

A thread-safe collector used within an `AgentLoop` session to track:

- Tool Performance: call_count , error_count , and total_duration per tool via ToolMetrics . src/utils/metrics.rs 13-24
- Latency: min_duration , max_duration , and computed average_duration . src/utils/metrics.rs 28-33
- Token Usage: Session-specific input and output totals. src/utils/metrics.rs 95-98

**Defined in:** [src/utils/metrics.rs51-56](https://github.com/qhkm/zeptoclaw/blob/2c743d31/src/utils/metrics.rs#L51-L56)

### Service Level Objectives (SLO)

At the end of a session, the `SessionSLO` evaluates the session against predefined targets:

1. Tool Success Rate: Target $\ge 95%$ ( SLO_TOOL_SUCCESS_RATE ). src/utils/slo.rs 15
2. p95 Tool Latency: Target $< 10s$ ( SLO_P95_LATENCY_SECS ). src/utils/slo.rs 16
3. Agent Completion: Did the agent produce a final response?

**Sources:** [src/utils/slo.rs20-35](https://github.com/qhkm/zeptoclaw/blob/2c743d31/src/utils/slo.rs#L20-L35) [src/utils/slo.rs39-56](https://github.com/qhkm/zeptoclaw/blob/2c743d31/src/utils/slo.rs#L39-L56)

```

```

**Sources:** [src/utils/metrics.rs73-92](https://github.com/qhkm/zeptoclaw/blob/2c743d31/src/utils/metrics.rs#L73-L92) [src/utils/slo.rs39-56](https://github.com/qhkm/zeptoclaw/blob/2c743d31/src/utils/slo.rs#L39-L56) [src/utils/slo.rs82-95](https://github.com/qhkm/zeptoclaw/blob/2c743d31/src/utils/slo.rs#L82-L95)

## Health Server Endpoints

ZeptoClaw exposes health endpoints via **raw TCP** (manual HTTP) to preserve the binary footprint (4MB design goal) [src/health.rs11-12](https://github.com/qhkm/zeptoclaw/blob/2c743d31/src/health.rs#L11-L12) and via the **Panel API** (Axum) [src/api/server.rs125-126](https://github.com/qhkm/zeptoclaw/blob/2c743d31/src/api/server.rs#L125-L126)

### Endpoint Specification

**Sources:** [src/health.rs437-517](https://github.com/qhkm/zeptoclaw/blob/2c743d31/src/health.rs#L437-L517) [src/api/server.rs125-126](https://github.com/qhkm/zeptoclaw/blob/2c743d31/src/api/server.rs#L125-L126) [src/api/middleware.rs111-171](https://github.com/qhkm/zeptoclaw/blob/2c743d31/src/api/middleware.rs#L111-L171)

### Health Response Snapshot

The `/health` endpoint aggregates:

- Process Info: Uptime and Version.
- Memory: RSS bytes via get_rss_bytes() . src/health.rs 44
- Global Usage: Values from UsageMetrics .
- Component Checks: Current status of all registered HealthCheck entries.

**Rendering logic:** [src/health.rs283-343](https://github.com/qhkm/zeptoclaw/blob/2c743d31/src/health.rs#L283-L343)

## Platform Memory Tracking

The `get_rss_bytes()` function provides platform-specific Resident Set Size (RSS) measurement without external dependencies.

### macOS Implementation

Uses `mach_task_self_` and `task_info` FFI to read `resident_size` from `MACH_TASK_BASIC_INFO` (flavor 20).

**Sources:** [src/health.rs45-102](https://github.com/qhkm/zeptoclaw/blob/2c743d31/src/health.rs#L45-L102)

### Linux Implementation

Reads `/proc/self/statm`, parses the 2nd field (resident pages), and multiplies by the kernel page size retrieved via `sysconf(_SC_PAGESIZE)`.

**Sources:** [src/health.rs104-122](https://github.com/qhkm/zeptoclaw/blob/2c743d31/src/health.rs#L104-L122)

## API & Panel Integration

The `AppState` used by the Axum-based Panel API holds optional references to health components, allowing the dashboard to visualize system health.

### AppState Health Fields

- health_registry : Option<Arc<HealthRegistry>> for component status. src/api/server.rs 44
- usage_metrics : Option<Arc<UsageMetrics>> for global counters. src/api/server.rs 46
- metrics_collector : Option<Arc<MetricsCollector>> for session-specific stats. src/api/server.rs 48

### Security

The `/api/health` and `/api/metrics` routes are protected by `auth_middleware`, requiring either a static API token or a valid HS256 JWT.

**Sources:** [src/api/middleware.rs111-171](https://github.com/qhkm/zeptoclaw/blob/2c743d31/src/api/middleware.rs#L111-L171) [src/api/server.rs125-126](https://github.com/qhkm/zeptoclaw/blob/2c743d31/src/api/server.rs#L125-L126)
