<!-- source: https://deepwiki.com/openclaw/openclaw/3.4.5-web-search-and-fetch -->

# Web Search & Fetch

- docs/help/environment.md
- docs/tools/exec.md
- scripts/deadcode-unused-files.allowlist.mjs
- src/agents/acp-spawn-parent-stream.test.ts
- src/agents/acp-spawn-parent-stream.ts
- src/agents/bash-process-registry.ts
- src/agents/bash-tools.exec-host-gateway.test.ts
- src/agents/bash-tools.exec-host-gateway.ts
- src/agents/bash-tools.exec-host-node-phases.ts
- src/agents/bash-tools.exec-host-node.test.ts
- src/agents/bash-tools.exec-host-node.ts
- src/agents/bash-tools.exec-runtime.test.ts
- src/agents/bash-tools.exec-runtime.ts
- src/agents/bash-tools.exec-types.ts
- src/agents/bash-tools.exec.path.test.ts
- src/agents/bash-tools.exec.script-preflight.test.ts
- src/agents/bash-tools.exec.security-floor.test.ts
- src/agents/bash-tools.exec.ts
- src/agents/bash-tools.schemas.ts
- src/agents/bash-tools.test.ts
- src/agents/command/attempt-execution.error-propagation.test.ts
- src/agents/command/attempt-execution.runtime.ts
- src/agents/exec-approval-result.test.ts
- src/agents/exec-approval-result.ts
- src/agents/sessions/bash-executor.ts
- src/agents/sessions/resolve-config-value.ts
- src/agents/sessions/tools/bash.test.ts
- src/agents/sessions/tools/bash.ts
- src/agents/sessions/tools/render-utils.ts
- src/agents/shell-snapshot.test.ts
- src/agents/shell-snapshot.ts
- src/agents/shell-utils.test.ts
- src/agents/shell-utils.ts
- src/auto-reply/reply/agent-runner-payloads.test.ts
- src/auto-reply/reply/agent-runner-payloads.ts
- src/auto-reply/reply/agent-runner.runreplyagent.e2e.test.ts
- src/auto-reply/reply/reply-delivery.test.ts
- src/auto-reply/reply/reply-delivery.ts
- src/auto-reply/reply/reply-media-paths.test.ts
- src/auto-reply/reply/reply-media-paths.ts
- src/gateway/managed-image-attachments.ts
- src/gateway/server-node-events.test.ts
- src/gateway/server-node-events.ts
- src/infra/exec-safe-builtins.test.ts
- src/infra/exec-safe-builtins.ts
- src/infra/heartbeat-runner.subagent-session-guard.test.ts
- src/media/image-ops.rastermill-adapter.test.ts
- src/media/image-ops.ts
- src/media/web-media.test.ts
- src/media/web-media.ts

This section documents the implementation and integration of web search and fetch tools within the OpenClaw agent runtime. It covers the supported web search providers, the HTTP fetch tool with SSRF (Server-Side Request Forgery) protection, and the underlying transport mechanisms that ensure secure and reliable web access.

## Overview

OpenClaw provides built-in web tools that extend agent capabilities with external web data access:

- web_search : Executes web queries across multiple providers offering structured search results or AI-grounded answers with citations.
- web_fetch : Fetches raw HTTP content. It includes safeguards to prevent SSRF attacks and enforces resource limits. src/media/web-media.ts 46-56
- x_search : A specialized tool for searching X (formerly Twitter) content, primarily integrated through the xAI provider.

These tools are managed through a provider-based architecture where capabilities are resolved at runtime based on configuration and available API keys or Auth Profiles. [src/media/web-media.ts97-114](https://github.com/openclaw/openclaw/blob/fbdf5937/src/media/web-media.ts#L97-L114)

Sources:

- src/media/web-media.ts 46-114

## Web Search Tool Integration

### Supported Providers & Runtime Resolution

OpenClaw supports various web search providers integrated as plugins. The system selects the active provider based on available credentials and configuration.

### Failover and Profile Rotation

Web search operations are subject to rate limits. The system manages these failures by rotating through available `AuthProfile` entries. When an agent execution fails due to provider-specific errors, the `runWithModelFallback` utility handles retries and provider transitions. [src/auto-reply/reply/agent-runner.runreplyagent.e2e.test.ts76-90](https://github.com/openclaw/openclaw/blob/fbdf5937/src/auto-reply/reply/agent-runner.runreplyagent.e2e.test.ts#L76-L90)

Title: Web Search Failover and Credential Rotation

```

```

Sources:

- src/auto-reply/reply/agent-runner.runreplyagent.e2e.test.ts 76-90

## Web Fetch & SSRF Protection

The fetch system is designed for secure retrieval of web content, focusing on protection against Server-Side Request Forgery (SSRF).

### SSRF Guarding

The `SsrFPolicy` is enforced during web media and content fetching. This policy validates target URLs to prevent access to sensitive internal infrastructure or restricted local paths. [src/media/web-media.ts18](https://github.com/openclaw/openclaw/blob/fbdf5937/src/media/web-media.ts#L18-L18)

- Private Network Blocking : Requests to loopback or internal IP ranges are intercepted by the SSRF guard.
- Local Media Access : The system explicitly checks if a URL refers to local files via resolveMediaStoreUriToPath or resolveInboundMediaReference . src/media/web-media.ts 83-95

### Media Fetching and Sanitization

The `loadWebMedia` function handles fetching of images, PDFs, and other media types. [src/media/web-media.ts52-54](https://github.com/openclaw/openclaw/blob/fbdf5937/src/media/web-media.ts#L52-L54)

- Host Read Capability : Restricts local file reads to validated plain-text (CSV/Markdown) and trusted documents via HOST_READ_ALLOWED_DOCUMENT_MIMES . src/media/web-media.ts 140-157
- Content-Type Validation : Validates the content-type of fetched data against an allowlist of supported media mimes. It rejects HTML from local host-read paths to prevent security boundary escapes. src/media/web-media.ts 168-174
- Image Optimization : Fetched images can be automatically processed (resized/compressed) using optimizeImageToJpeg or createImageProcessor . src/media/web-media.ts 31-34 src/media/web-media.ts 131-134

### Implementation Details

- Fetch Implementation : Tools can provide a custom fetchImpl to override default behavior while still adhering to the ssrfPolicy . src/media/web-media.ts 50-52
- Idle Timeouts : The system enforces readIdleTimeoutMs to prevent stalled connections from consuming agent resources. src/media/web-media.ts 54

Title: Secure Web Fetch Pipeline

```

```

Sources:

- src/media/web-media.ts 18-174
- src/media/web-media.test.ts 91-203

## Transport and Provider Specifics

OpenClaw implements transport logic to handle provider-specific response formats and ensure stability.

### Payload Normalization

When agents return web content (including media), the system performs a final normalization in `buildReplyPayloads`. This includes stripping duplicate media already sent by messaging tools and ensuring that media paths are relative to the safe workspace. [src/auto-reply/reply/agent-runner-payloads.test.ts146-169](https://github.com/openclaw/openclaw/blob/fbdf5937/src/auto-reply/reply/agent-runner-payloads.test.ts#L146-L169)

### Request Timeouts and Retries

Web operations use configurable timeouts. For instance, the `web_fetch` operations within `loadWebMedia` respect a `readIdleTimeoutMs` to ensure the agent does not hang indefinitely on a slow response. [src/media/web-media.test.ts186-203](https://github.com/openclaw/openclaw/blob/fbdf5937/src/media/web-media.test.ts#L186-L203)

Sources:

- src/auto-reply/reply/agent-runner-payloads.test.ts 146-169
- src/media/web-media.test.ts 186-203
