Pyunto Diary

pyunto-agent

Make an external agent the partner of a Pyunto exchange diary. The agent is an ordinary member: its own Pyunto account, its own X25519 identity key, invited with the normal invite link. The server is unchanged and never sees plaintext; decryption happens in this process.

Pyunto for iPhone and iPad · Pyunto for Android · pyunto-robotics — the same idea, with a robot at the other end (watch it, 100 s)

A real run with Claude Code as the model: a strength-coach persona in a Markdown file, and a client logging sessions from their phone. The second reply compares with the day before. Watch the full 90-second video.

---

Two kinds of agent

The same package, the same account, the same keys. What differs is who starts the conversation.

1. Agent — it writes to your users

You write; it replies, unprompted, in the same thread.

     your phone                     your computer
 ┌───────────────┐            ┌──────────────────────┐
 │  Pyunto app   │            │  pyunto-agent run    │
 │               │            │                      │
 │  "tough day   │  ───────▶  │  reads the entry     │
 │   at work"    │  encrypted │  decrypts it HERE    │
 │               │            │        │             │
 │               │            │        ▼             │
 │  "that sounds │  ◀───────  │  Claude API, or      │
 │   exhausting" │  encrypted │  Claude Code, or     │
 └───────────────┘            │  your own HTTP URL   │
                              └──────────────────────┘
        │                                │
        └────────  api.pyunto.com  ──────┘
              (ciphertext only, never plaintext)

How you invite it: run pyunto-agent pair, scan the QR code with the app, choose a diary. The agent starts answering as soon as you approve.

What it is for: running a service that reaches people where they already are.

A general-purpose chatbot is a website somebody has to remember to visit, in a tab with no memory of them. This is a named contact in a messaging app on their phone, who has read everything they wrote before, and who answers in character because you wrote the character.

That character is a file. --persona coach.md is the whole difference between a polite assistant and a service worth paying for:

You are a strength coach. Your client logs every session here.

- Hold them to the programme. If they skipped legs again, say so plainly.
- Always ask for the numbers: weight, sets, reps. A session without numbers is not logged.
- Compare against last week before you praise anything.
- No pep talk. One sentence of encouragement, only when it is earned.

pyunto-agent run --backend claude-api --persona coach.md

Some shapes this takes:

Service	The persona does what a chatbot will not
🏋️ Strength coach	Demands the numbers, remembers last week's, refuses to praise a skipped session
🗣️ Language tutor	Corrects every message, keeps a running list of the learner's own mistakes, escalates difficulty
🏥 Clinic follow-up	Asks the post-operative questions in order, every day, and flags the answers a nurse should see
🥗 Nutritionist	Reads the meal photographs, keeps the week's running total, notices the pattern rather than the meal
📚 Study supervisor	Holds a student to a revision schedule, asks what was actually covered, will not accept "I studied"
🔧 Property manager	Tenants report a problem in the same thread each time; the agent triages, asks for a photograph, and escalates
📐 Field inspection	An engineer photographs a site; the agent records it against the job and asks for what is missing
📅 Sobriety or habit support	Checks in at the hour that matters, keeps the streak, responds to a relapse the way you told it to

What makes these work here rather than in a chat window:

• The persona holds. It is a file you control, not a prompt the user can talk their way out of.
• It remembers. --history gives every reply the recent thread, so "the same as last Tuesday" means something.
• It is on their phone. A notification arrives; they reply in a messaging app they already have. No login, no tab, no app to learn.
• You see nothing. The diary is end-to-end encrypted and decrypted only in the process you run. That is a real claim to make to a client talking about their body, their health or their finances.
• One process, many clients. With no --space, pyunto-agent run answers every diary the account has been invited into, so onboarding a client is them scanning a QR code. Use --space to pin one agent to one client.

2. MCP — you ask Claude about a diary

Nothing runs in the background. Claude Code or Claude Desktop reaches into the diary when you ask it to.

     your computer
 ┌────────────────────────┐
 │  Claude Code / Desktop │
 │            │           │        ┌──────────────────┐
 │            ▼           │        │  your phone      │
 │  "what did I write     │        │  Pyunto app      │
 │   about the garden?"   │        │                  │
 │            │           │        │  the same diary, │
 │            ▼           │        │  read and written│
 │  ┌──────────────────┐  │        │  from either end │
 │  │ pyunto-agent mcp │──┼──────▶ │                  │
 │  │  read_thread     │  │encrypt │                  │
 │  │  post_entry      │  │        └──────────────────┘
 │  │  ...12 tools     │  │
 │  └──────────────────┘  │
 └────────────────────────┘

How you invite it: add the server to your MCP client (below), then ask Claude to pair: the pair tool shows a QR code to scan in the Pyunto app. No terminal step is needed.

claude mcp add pyunto-diary -- uvx pyunto-agent mcp

What it is for: using your diary as memory. Searching months of entries, summarising a week, writing an entry from the desktop, letting Claude check what you recorded before it answers. You start every exchange; it never speaks unasked.

Which one?

	Agent (run)	MCP (mcp)
Who speaks first	the agent	you
Runs in the background	yes, continuously	no, only when asked
Where the person talks to it	the Pyunto app, on their phone	Claude Code / Desktop
Who it is for	a service and its users	one person and their own diary
Typical use	a coach, a tutor, a desk that answers	searching and summarising your entries

Both can be paired into the same diary at once — they are the same account, and nothing stops run answering on your phone while mcp reads the same entries from your desk.

A third kind lives elsewhere. pyunto-robotics puts a robot at the other end instead of a language model: you write "go and find some sunlight" and a simulated — or real — machine does it and reports back with photographs. It is built on this package, and pairs the same way.

Quick start: a personal trainer your clients message

Building a real service, from nothing to a client's phone.

1. Install

Python 3.11 or newer. Install into a virtual environment: Homebrew's Python refuses a global pip install (externally-managed-environment).

python3.12 -m venv ~/pyunto-env
source ~/pyunto-env/bin/activate
pip install 'pyunto-agent[qr]'

Then choose where replies come from:

export ANTHROPIC_API_KEY=sk-ant-...    # the Claude API (used below), or
claude --version                       # Claude Code, if installed: no API key needed

With Claude Code, replace --backend claude-api below with --backend command --command 'claude -p --output-format json'. If neither is set up, pair and run say so and list the options before showing a QR code.

2. Write the trainer

This file is the service. Everything the trainer is — strict or gentle, what it insists on, what it refuses to let slide — is here, and your clients cannot talk it out of any of it.

cat > trainer.md <<'EOF'
You are a strength coach. Each client logs their sessions in this diary.

- Always ask for the numbers: exercise, weight, sets, reps. "I trained today" is not a log --
  ask what they lifted.
- Compare against their recent sessions before responding. If the weight has not moved in
  three weeks, say so.
- If they skipped a session, ask what happened. Once. Then move on.
- No motivational speeches. One line of encouragement, only when the numbers earn it.
- Never give medical advice. Pain goes to a doctor, and say so plainly.
- Reply in the language they wrote in. Two to four sentences.
EOF

3. Make a QR code to hand out

pyunto-agent pair --operator "Sano Fitness" --image trainer-qr.png

Written to trainer-qr.png — send this to whoever should be able to reach the
agent. Each person who scans it lets 🤖 Claude into their own diary; the code
names the account asking and nothing else.

Put that image on your booking page, in the welcome email, or printed on a card at the desk. It is not a secret and it does not expire: the same image works for every client. Scanning it only lets them ask — each client approves it into their own diary, on their own phone, and sees who is running it before they do.

Use .svg instead of .png for print, or when Pillow is not installed.

4. Start answering

pyunto-agent run --backend claude-api --persona trainer.md

It starts by naming every diary it answers in, so you can see which ones you were let into:

Answering in 2 diaries:
  Sano Fitness - Aiko  [2 members, answers every entry]  1552f3dc
  Sano Fitness - Ken   [2 members, answers every entry]  3124f1b8  (cannot read it yet: open this space in the Pyunto app once)

One process serves every client who has scanned the code. A client writes:

Bench 80kg 5x5, felt heavy on the last set

and the trainer replies in their diary, having read what they lifted last week — as a notification on their phone, in an app they already have.

Each client's diary is separate and end-to-end encrypted. Decryption happens only in the process you are running; Pyunto's servers never see any of it.

Just trying it yourself?

Skip the persona and pair without an image — the QR code appears in the terminal, and the agent starts answering as soon as you scan it:

pyunto-agent pair --operator "your name"                       # with ANTHROPIC_API_KEY set
pyunto-agent pair --operator "your name" \
  --backend command --command 'claude -p --output-format json'  # or with Claude Code

Then open that space in the app once (that hands the agent the key) and write an entry.

Who it answers, and what members see

• In a diary with one person and the agent, it answers every entry.
• With three or more members (the agent counts), it answers only when it is mentioned by name, or when an entry is sent to everyone. An entry addressed to someone else does not wake it.
• An entry written by another agent or a robot is answered only when it names this agent (@Claude). A robot's reports and another agent's check-ins are not replied to, so two programs in one diary never talk over the people in it or to each other in a loop.
• A robot (pyunto-robotics) is stricter: it acts only on entries from people, and only when addressed by name.
• Every member sees, in the app's participant list, who runs the agent (--operator) and where the diary is decrypted (--runtime, self_hosted by default).

Details

If the agent is already in a diary, pair says so and waits: scan the code to add it to another diary, or press Enter to keep the one it is in.

Add --no-run to draw the QR code and exit, if you would rather start it yourself later with pyunto-agent run. The QR code holds no secret: it names the account asking, and the decision stays with whoever holds the phone.

Without PYUNTO_EMAIL the agent uses an anonymous account named PYUNTO_AGENT_NAME (default "Claude"); the device id and identity key live in ~/.pyunto-agent/.

Installing without cloning

If you only want to run the agent, not work on it — one line, no clone:

pip install 'pyunto-agent[qr]'
pyunto-agent pair --operator "your name"

To run the latest unreleased code instead:

pip install 'pyunto-agent[qr] @ git+https://github.com/utagoeinc/pyunto-agent'

Longer guides

• GUIDE.md — the whole setup, step by step, for somebody who does not live in a terminal. Uses Claude Code (claude -p) rather than an API key.
• DEPLOY.md — running pyunto-agent serve in a container alongside the server.

Pairing from the app instead

If someone else set the agent up for you, go the other way:

1. In the Pyunto app, create or open a shared space and generate an invite link.
2. pyunto-agent join 'pyunto://invite/…'
3. Open the space once in the app, so the space key is shared with the agent's identity key.
4. pyunto-agent whoami should now show key=yes for that space.

Photos, videos and documents

The agent reads what people attach, not only what they type. Each photo, video or document is downloaded, decrypted on your machine into ~/.pyunto-agent/attachments/, and shown to the model:

Attached	Claude API (claude-api)	Claude Code (command)	http
Photo	the image itself	the file path; Claude Code opens it	path + data_base64
Video	4 evenly spaced frames (needs ffmpeg)	the file and its frames	path + data_base64
PDF	the PDF itself	the file path	path + data_base64
Text, Markdown, CSV	its text	the file path	path + data_base64
Word, Excel, PowerPoint	its text (pip install 'pyunto-agent[docs]')	the file path	path + data_base64

So "can you check the grammar in this?" with a document attached, or "what do you think of the garden?" with three photos, works as you would expect. A few details:

• A caption is its own entry right after the photo. The agent waits a few seconds after a photo and answers the photo and its caption together, once.
• Only the newest four attachments in a thread are shown to the model on each reply.
• With Claude Code, the attachment folder is passed with --add-dir automatically, so it may open the files.
• Whatever the model is shown is sent to that model's provider, as with text. Say so to the people in the diary.

Agent: Claude API replies

pyunto-agent run --backend claude-api --persona persona.md   # replies, phone notification
pyunto-agent run --backend claude-api --silent               # replies, no notification
pyunto-agent run --backend claude-api --dry-run                     # log replies, do not post

Agent: Claude Code as the partner

pyunto-agent run --backend command --command 'claude -p --output-format json'

The command gets the prompt on stdin (JSON with the persona, the thread so far, and prompt), and its stdout is used as the reply ({"result": …}, {"reply": …}, or plain text). Add {prompt} to the command to pass the prompt as an argument instead.

MCP: Pyunto as tools for Claude

Listed in the official MCP Registry as com.pyunto/diary (Pyunto Diary). It runs with uv; nothing else needs installing.

Claude Code

claude mcp add pyunto-diary -- uvx pyunto-agent mcp

Claude Desktop (claude_desktop_config.json) and Cursor (~/.cursor/mcp.json)

{
  "mcpServers": {
    "pyunto-diary": { "command": "uvx", "args": ["pyunto-agent", "mcp"] }
  }
}

VS Code (.vscode/mcp.json)

{
  "servers": {
    "pyunto-diary": { "type": "stdio", "command": "uvx", "args": ["pyunto-agent", "mcp"] }
  }
}

Then say "pair with my Pyunto diary". Claude calls pair and shows a QR code; scan it in the app, choose a diary, approve, and open that diary in the app once so the key is shared.

Tool	What it does
pair	QR code that lets a person add this account to one of their diaries
whoami, list_spaces, list_members	the account, its diaries, and who is in each (and who runs any agent)
list_threads, read_thread	entries, decrypted on this machine
read_attachment	a photo (as an image), a video (as frames) or a document (as text)
wait_for_message	block until someone writes
post_entry, react, post_sticker, post_list_item	write into the diary
quick_list_stats	how often each quick-list item was logged over a period
join_space	join from an invite link or code made in the app

A minimal autonomous loop in Claude Code:

> Use wait_for_message, then reply with post_entry in the same thread. Repeat.

Two of these are why a diary partner can say things a chat model cannot.

quick_list_stats counts the repeated things a diary tracks — medicines taken, books read to a child, meals, walks — over a period. It is what lets an agent say "that is the third time this week" instead of asking. The tally is assembled on your machine from decrypted entries: the server stores these posts as ciphertext, so no endpoint could answer it.

list_members says who else is in the space, and for each agent who runs it and where it runs. Call it before writing anything sensitive — it tells you who reads what you post.

Pair it with @pyunto/tm-mcp and the partner can also read and book the human's schedule.

Reference

Command	What it does	Options
pair	Shows a QR code; the person who scans it lets the agent into a diary. Then answers entries	--operator NAME (shown to every member), --runtime self_hosted|hosted|endpoint, --image FILE.png|.svg (write the code to a file and exit), --big (larger terminal QR), --no-run (exit after pairing), plus the run options
run	Answers entries in every diary the agent is in	--backend claude-api|command|http, --command CMD, --url URL, --model ID, --persona FILE.md, --space ID (repeatable; only these diaries), --history N (entries of context, default 12), --silent (post without a notification), --dry-run (log replies, do not post)
mcp	MCP server over stdio for Claude Code / Claude Desktop
whoami	The agent's account, and each space with key=yes/no
join LINK	Join through an invite link made in the app
send SPACE_ID TEXT	Post one entry	--thread ID, --silent
serve	HTTP service for running agents on behalf of others (see DEPLOY.md)	--listen HOST:PORT (default 127.0.0.1:8788), plus the run options

Backends: claude-api needs ANTHROPIC_API_KEY. command runs any program with the prompt as JSON on stdin and uses its stdout as the reply (claude -p --output-format json is Claude Code). http POSTs the same JSON to --url.

Configuration, from the environment or a .env file in the working directory:

Variable	Meaning
ANTHROPIC_API_KEY	for --backend claude-api
PYUNTO_EMAIL, PYUNTO_PASSWORD	run as a registered account instead of an anonymous one
PYUNTO_AGENT_NAME	display name of the anonymous account (default Claude)
PYUNTO_AGENT_DIR	where the device id, identity key and space keys live (default ~/.pyunto-agent)
PYUNTO_BACKEND, PYUNTO_COMMAND, PYUNTO_BACKEND_URL, PYUNTO_MODEL	defaults for --backend, --command, --url, --model
PYUNTO_PERSONA	default for --persona
PYUNTO_HISTORY	serve only: entries of context (default 8)
PYUNTO_AGENT_LISTEN, PYUNTO_AGENT_SECRET	serve only
PYUNTO_BASE_URL	API server (default https://api.pyunto.com)

Changes in each version are in Releases.

Notes

• Diary text is sent to the backend you choose. With claude-api that is Anthropic's API; say so to the people in the diary.
• Rate limit: 60 replies per hour by default (Bridge(max_replies_per_hour=…)).
• The agent can read a diary only after a member has shared the space key with it, which the app does when a member opens the space. If whoami shows key=no, open the space in the app.
• How the encryption works, including test vectors: https://pyunto.com/encryption.html.
• Tests: pip install -e '.[dev]' then pytest (includes the sealed-box test vector).

Licence

Apache-2.0. See LICENSE.