Running A Privacy-First LLM Agent Workflow

I know what I write here is not a complete private setup or a guarantee of true privacy, But I have kept the definition loose to privacy that I can afford.

There are a few LLM agent use-cases I have in mind that I want complete privacy for, like financial analysis (maybe an extension of hdfc-analytics), my emails, health records management, etc.

I planned to start with setting up this workflow for emails as it acts like a data store for a lot of things like weekly credit card bills, invoices, travel tickets (that helped me build my travel analytics once), etc.

I would like to use truly locally hosted LLMs but, since I am constrained by the lack of hardware, I am at the mercy of cloud self-hosteable LLM models and their guarantees of not sharing and accessing my data in their data & privacy policy.

Starting with my email use-case, I have set up an on demand Qwen3-4B-Instruct-2507, served by vLLM on an L4 instance on Modal that reads my emails, triages it, and posts a digest into a private channel on my self-hosted Mattermost.

I am still working on how periodic this check should be. Since email is an async mode of communication, some degree of asyncness or TAT for responses should be expected. But now that people have their complementary or co-pilots, we are kinda questioning traditional functioning alltogether. For now, it runs every 3 hours but a lower frequency can help me with costs and probably running a larger model. In case I want to run it on-demand, I can ask my Hermes Agent to trigger this workflow for me which acts just as a trigger and has no access to other data.

Point of doing all this was to set up some kind of kinda privacy first agent setup that helps me as a groundwork for some of my other use-cases.

As of now a warm run is around 60 seconds of L4 time which roughly comes around $0.015 per run.