io.github.jgador/websharp

Search the web and extract article text for LLMs.

1MITdevtools

Install

Config snippet generator goes here (5 client tabs)

README

# WebSharp MCP HTTP Server

WebSharp is a Model Context Protocol (MCP) server written in C# (.NET 9) that exposes web search and readable page extraction tools over the streamable HTTP transport. It pairs a SearXNG metasearch instance with the SmartReader content extraction library and ships with Docker assets so you can run everything locally with one command.

## Highlights
- Streamable HTTP MCP endpoint served by ASP.NET Core via `ModelContextProtocol.AspNetCore`.
- SearXNG-backed metasearch tool that returns both a readable summary and structured JSON metadata.
- SmartReader-powered page reader tools for single URLs or batches, ideal for grounding and fact checking workflows.
- Container-first development story with Docker Compose and an inspector-ready `.mcp.json` profile.

## Available MCP tools

### `web_search`
| Parameter | Type | Default | Notes |
| --- | --- | --- | --- |
| `query` | string | _required_ | Free-form search string submitted to the configured search backend (defaults to SearXNG). |
| `limit` | integer | 10 | Maximum number of results to surface in the text summary. When set to `0` or negative, the parameter is omitted and the backend decides the result count. |
| `categories` | string | `"general"` | Comma-separated categories forwarded to the backend (for SearXNG, examples include `general,it,science`). |
| `language` | string | `"en"` | Language tag forwarded to the backend. When using the default SearXNG instance, see its language list. |

**Response**
- Text summary showing the top results, related suggestions, and any unresponsive engines.
- `structuredContent` contains a JSON object with:
  - `query`: echo of the evaluated query string.
  - `items`: array of result objects (`title`, `url`, `content`, `engine/engines`, `category`, `positions`, `parsed_url`).
  - `suggestions`: related queries returned by the backend (SearXNG surfaces them when available).
  - `unresponsive_engines`: list of engines that timed out or failed, with optional messages.
  - `error`: present when the search could not be completed.

### `web_page_read`
| Parameter | Type | Default | Notes |
| --- | --- | --- | --- |
| `urls` | array<string> | _required_ | One or more absolute HTTP/HTTPS URLs to fetch and parse. |

**Response**
- When a single URL is provided, returns the article title, excerpt, a preview of the readable text (~800 characters), and footer metadata (source URL, detected language).
- When multiple URLs are provided, the summary includes overall success/failure counts, per-page details (title, URL, full text, language), and error messages for any failures. Requests run concurrently and honor the MCP cancellation token.
- `structuredContent` is not populated; all data is returned in the text block.


### Run everything with Docker Compose
1. Install Docker Desktop (or another OCI-compatible runtime) and ensure it is running.
2. From the repository root, start both the SearXNG dependency and the MCP server:  
   `docker compose -f docker-compose.yaml --project-name websharp up -d` (add `--build` on the first run or whenever the Dockerfile changes).
3. Wait until the `websharp-mcp` container reports `Now listening on: http://0.0.0.0:8080`.
4. The MCP server is now reachable at `http://localhost:8081` (mapped from container port 8080). SearXNG is available at `http://localhost:8080/` for inspection.

### Build the Docker image manually
Run the server image build without Docker Compose from the repository root:
```powershell
docker build -f .\src\WebSharp.Mcp.Http\Dockerfile -t jessegador/websharp-mcp:v0.99.0-rc2 --no-cache src
```
Update `-t` to match the image registry and tag you plan to publish.

### Connect with the MCP Inspector
1. Start (or keep running) the compose stack above.
2. Launch the Inspector with the included client profile:  
   `npx -y @modelcontextprotocol/inspector --config .\.mcp.json --server websharp-mcp`
3. Copy the printed token into the Inspector UI by opening `http://localhost:6274/?MCP_PROXY_AUTH_TOKEN=<token>` in your browser.
4. In the Inspector UI, pick the `websharp-mcp` server to explore the available tools and try calls right away.

## Running from source
- Install the [.NET 9 SDK](https://dotnet.microsoft.com/download/dotnet/9.0).
- Ensure you have access to a SearXNG endpoint. The easiest option is to keep using the Docker Compose service and export `Searxng__BaseUrl` to `http://localhost:8080` before starting the app.
- Launch the server:  
  `dotnet run --project src/WebSharp.Mcp.Http/WebSharp.Mcp.Http.csproj --urls http://localhost:8081`
- The MCP endpoint listens on `/`. In `DEBUG` builds you can hit `/__routes` to inspect the configured ASP.NET endpoints.

## Configuration
| Environment variable | Default | Description |
| --- | --- | --- |
| `ASPNETCORE_URLS` | `http://0.0.0.0:8080` (Docker) | Listening address(es) for the HTTP server. Override to change the exposed MCP port. |
| `Searxng__BaseUrl` | `http://searxng:8080` | Base URL of the SearXNG ins