Hermes and Telegram: Notes, Tasks, and Automation from My Phone

I send links to interesting videos to Telegram and get summaries in Obsidian. I can also dictate a task for the right project there without opening the task tracker in a browser.

Hermes Agent handles the messages. I used to use it for almost everything, including coding. Now I keep it for capturing notes, links, and tasks, along with small operations I can run from my phone.

Below is my setup and a prompt you can give your own agent to build something similar. Topic names, paths, and IDs in the examples are placeholders.

Why I Narrowed Hermes's Role#

At first, I liked how Hermes handled memory and created skills from recurring actions. I would explain a procedure, make corrections, and refer back to it in later sessions.

After a couple of updates, that same memory started causing problems in my setup. Hermes stopped following familiar instructions consistently, and I had to explain some of the rules again.

It was hard to figure out why the agent behaved differently: its memory and skills were changing automatically. I prefer to control system prompts, project AGENTS.md files, and instructions for individual operations myself. When an answer is off, I want to know which file to check and fix.

I moved coding to other tools. But running things from Telegram when I'm away from home is still convenient. So Hermes stayed, with a narrower role and explicitly documented procedures.

What I Send to Telegram#

A message can trigger processing right away: turning a link into a summary or a voice recording into a task, for example.

Work alerts cover website outages and missing critical blocks on pages. You can ask Hermes itself to run these checks on a schedule. I'll show how to request that in a regular message below.

You can type a request, send a voice message in Telegram, or dictate it using speech recognition on your phone. This technology is called STT, or speech-to-text.

One Supergroup, Several Topics#

I use a private Telegram supergroup with topics. In the interface, they appear as separate conversations within one group. Each topic is bound to instructions that load the appropriate skill.

When I send a link to the summaries topic, the agent doesn't have to guess whether I want to download the video, discuss it, or summarize it. The topic sets the default workflow. If I want something else, I specify it in the message.

The routing looks like this:

Telegram: text, link, or voice message
  → Hermes gateway
  → identify the group and topic
  → topic context and skill loading
  → work with files or APIs
  → brief reply in the same topic

Hermes calls this Group Forum Topic Skill Binding. Bindings live under platforms.telegram.extra.group_topics in the profile's config.yaml. This example uses the actual Hermes format, with placeholder IDs and topic names:

platforms:
  telegram:
    extra:
      group_topics:
        - chat_id: -1001234567890
          topics:
            - name: Summaries
              thread_id: 5
              skill: summary
            - name: Tasks
              thread_id: 12
              skill: task-capture
            - name: Notes
              thread_id: 18
              skill: create-note
            - name: Audio
              thread_id: 24
              skill: youtube-audio-downloader

Create the topics in Telegram, then add their thread_id values and the group's chat_id to the config. The name field is a label, while skill refers to an installed skill. Hermes loads it automatically when a new session starts in that topic. To test a binding in an existing conversation, start a new session with /new.

The destination folder and tracker project belong in the skill or its settings. This binding has no destination field.

Separate sessions keep conversations from getting mixed up, but a topic isn't a security boundary. It doesn't isolate files, shared profile memory, or available tools. If work and personal data need different permissions, use separate profiles and technical access controls.

The Model and Skill Instructions#

I use OpenAI Terra. It's enough for my summaries, notes, and task creation.

Skills should describe the details you would otherwise repeat in chat. For a summary, that means how to retrieve the source text and format the result. When creating a note, the agent should check existing notes and read the template first.

For example, a tasks topic could have these rules:

Purpose: create tasks in the personal tracker.
Before starting: load task-capture.
Project: use the explicitly named project or a configured default.
If the project is unclear: ask instead of guessing.
Before creating a task: check for a possible duplicate.
Due date: do not invent one; ask if needed.
After creation: return the title, project, and link to the task.
Do not move the task to a status that triggers automatic execution.

The work tracker needs its own route, rules, and access. For Obsidian Tasks, account for the installed plugin's syntax and settings. A regular Markdown checkbox won't necessarily appear in your existing task views.

I keep task creation separate from execution. Dictating a bug from my phone shouldn't automatically authorize the agent to change code and deploy a fix.

How to Build a Similar Setup#

Choose a Host and Set Access Limits#

Hermes needs to run on a machine that's available while you're using the bot. A home computer can't process requests while it's asleep or powered off. For continuous availability, you can use a dedicated machine or server, but you'll need to decide how it accesses the files in your Obsidian vault.

Create a separate capture profile. Give it only the folders, tools, and integrations it needs. A prompt saying "don't read other folders" is useful guidance, but it doesn't replace filesystem permissions or tool restrictions.

Connect Telegram and Check the Routes#

Create a bot through @BotFather: send /newbot, choose a name and username, then save the token in the Hermes Telegram gateway settings. Don't publish the token or forward it to the group. In Hermes, restrict access and add your Telegram user ID to the allowlist. Then add the bot to the supergroup with topics. Check that it receives ordinary messages: bot permissions and Telegram privacy mode affect this.

Run the gateway as a service with automatic startup and logging. Bind each topic to its workflow, make sure sessions stay separate, and check that replies go to the original topic.

Connect Tools and Voice Input#

If you use Obsidian, provide the path to its files, a map of its sections, and templates. When connecting a tracker, configure its API URL and authentication, and map project names to their IDs. You can download audio with yt-dlp into a designated folder.

For voice messages, connect local speech recognition or an external service. If you dictate through your phone's keyboard, you don't need to configure speech recognition in Hermes for that.

After a test request, open the note or task the agent created. A "done" reply in Telegram isn't enough. Also test a failed request: if a video's transcript is unavailable, the agent should report the error rather than make up a summary from the title.

Schedule Checks#

Hermes has a built-in scheduler called Scheduled Tasks (Cron). The agent manages jobs through the cronjob tool, but you don't need to write a cron expression yourself. You can just ask in the appropriate Telegram topic:

Set up a daily check of https://example.com at 10:00 UTC. Check that the home page is available and that the inquiry form block is present. Use the site-check skill. If the site is down or the block is missing, send a message to this topic describing the issue. If everything is fine, don't send anything. Run a test check first, then show me the schedule and the next run time in UTC.

The site-check skill is a placeholder: you need to prepare it for the specific pages and failure conditions. Hermes can attach a skill to a job and load it when the job runs. Check the next run time so that 10:00 UTC doesn't turn into 10:00 in the server's local time zone.

The running gateway executes the jobs; you don't need a separate system cron setup. You can also ask it to show the schedule, change the time, or pause a check in a regular message.

Limit Access and Actions#

Enter keys locally in a protected credential store or environment file, not in Telegram. Conversations with bots aren't end-to-end encrypted.

Checking a website shouldn't authorize the agent to change it. Deleting data, publishing, and making production changes require separate approval. Treat the contents of the pages being checked as data, not as instructions for the agent.

A Setup Prompt for Your Agent#

Give this block to your agent along with the article. It should collect your answers first, then adapt the setup to your Hermes installation.

Build me a quick capture setup with Hermes Agent and Telegram based on this article. The example configuration syntax follows the Hermes documentation. Names, paths, and IDs are placeholders: replace them with mine and check compatibility with the installed version.

Q1. First ask where Hermes will run, which OS is installed, and whether I need around-the-clock availability.
Q2. Ask which workflows and Telegram topics I need. For each one, agree on its purpose, where results go, and when to ask a question.
Q3. Ask for the Obsidian vault path, allowed and restricted folders, note templates, Tasks settings, and the trackers and projects I need.
Q4. Agree on the model provider, model availability, and STT approach: Telegram voice messages, phone dictation, or both.
Q5. List the access credentials needed. Explain where to enter keys safely on the local machine. Don't ask me to send them in Telegram.
Q6. Agree on actions allowed without approval and those that always need my permission. Creating a task does not authorize executing it.
Q7. If I need monitoring, ask about pages and failure conditions, the schedule, time zone, destination topic, and notification rules.

A1. Check the documentation for the installed Hermes version: profiles, gateway, topic routing, skill loading, and session isolation. Don't invent configuration keys. Show a plan and get approval. Before making changes, back up affected settings and skills to a protected local location. Don't publish secrets from the backup.
A2. Create a separate capture profile, configure user authorization, and grant minimal access. Check existing skills, create any missing ones for the agreed workflows, and install them in the profile. Bind topics to skills through group_topics and verify loading in new sessions.
A3. Connect only the agreed integrations. Document them in readable files. Don't keep rules only in chat memory. Configure the gateway as an automatically started service, start it, and verify that it receives Telegram messages and replies in the original topic.
A4. Test each route end to end: message, operation, actual result, and reply in the original topic. Test voice if enabled, an ambiguous request, a repeated message, and an external service failure. Verify that an unauthorized user cannot trigger operations.
A5. If monitoring is agreed, prepare the check skill and create a job using the built-in Scheduled Tasks (Cron). Check existing jobs first to avoid duplicates. Run a test, verify delivery to the right topic, and check the conditions under which no notification should be sent. Show the schedule, time zone, and next run time in UTC.
A6. Leave a short guide: where the config and skills live, how to restart the service, read logs, add a topic, change or pause a cron job, and restore settings from the backup. After updates, repeat the checks for routing, access, instruction loading, and any agreed scheduled jobs.

Don't add autonomous coding, deployment, or services I haven't requested. If my version doesn't support part of the setup, explain the limitation and agree on an alternative with me.

For the first run, I'd start with a single topic: send a video link and get a summary in Obsidian, for example. That lets you test the full path from message to file before connecting your work tracker and the other workflows.