Where the Answers Come From

A beta tester asked me a fair question: when ThAD hands me a draft, or a suggestion, or a list of my own sermons - where is that actually coming from? It doesn’t have a one-line answer, but it has an understandable one. Here it is.

Three Things Are Working at Once

When you work with ThAD, three separate things are in play. Most of the confusion about AI comes from mistaking one for another - so it’s worth knowing which is which.

1. The model - the part people mean when they say “AI”

Claude (from Anthropic) and ChatGPT (from OpenAI) are large language models - LLMs, which is how you’ll usually see it written. A model is trained on an enormous body of writing - at this point, a substantial portion of the internet - wrapped in code that handles conversations and remembers things about you. Each vendor offers several models, which differ in how much they were trained on and in the instructions that shape how they engage with you.

Here’s the important limit: a model only knows what it was trained on. It knows nothing about anything outside that - and nothing at all about events after its training ended. Three years ago that was a real ceiling. The vendors added internet access, which helped. But a great deal of useful writing isn’t on the internet at all. Your sermons, for instance.

2. Your archive - this is the part ThAD does

The way you give a model access to writing it was never trained on is called retrieval-augmented generation, or RAG. The model asks a separate system for the material it needs, at the moment it needs it, instead of having to have been trained on it. A substantial portion of ThAD is exactly that, for your library.

ThAD reads and indexes every document you give it, gets your archive into a form a model can query quickly, and then hands the model a set of tools for running those queries. You see it happen every time you search for a sermon or ask for a Sunday briefing. Your writing is never part of the model’s training - it’s looked up, on request, and only for you.

3. CONTEXT - and yes, I meant to capitalize that

Context is everything the model knows right now: about you, about your material, and about whatever you happen to be working on - including the entire conversation you’re in. It comes from several places at once:

  • What the model remembers about you. In Claude, look under Settings → Memory to see exactly what it has retained. ChatGPT has its own equivalent.
  • The conversation itself. Everything said so far in the current thread.
  • What its tools tell it. Including ThAD.

ThAD contributes several documents built specifically to give the model useful context about you - an author profile drawn from your own public biography, an overall writing style guide, and a set of style guides for individual document types. Each one ships with instructions for how to use it, which the model reads the first time ThAD is called in a conversation.


Why This Matters: Context

Take two things you’d actually ask for.

You ask for help editing. Part of that is the model - it was trained on proper English and it’s good at prose. The rest is the model asking ThAD how you write, specifically, and often how you write a sermon, specifically.

You ask for a theologian or writer worth reading on a text. Now it’s drawing on its training data (there is a great deal of religious writing on the internet), what it remembers about you from past conversations, and whatever it has pulled from ThAD - your author profile, your style. If you’ve been in the same thread for a while, it uses that too, and the suggestions bend toward what you’re actually working on. Then it takes all of that and goes looking - a web search, or a ThAD call for the scripture text.

Same model, same archive, different context - different answer. That’s the whole mechanism. The more the model knows about where you are and what you’re doing, the more the answer sounds like it was meant for you.


Three Questions Worth Asking

How do I know I can trust the answers?

Ask for citations. Insist on a link for every quote - and then follow the link. I do this more or less rigorously depending on how much getting it right matters, but when it matters, I follow every single one. Sometimes the model is wrong. It was trained on writing produced by people, and we are a fallible bunch.

This is the single most useful habit you can build. It costs you thirty seconds and it is the difference between a tool you can preach from and a tool you can’t.

What is it all working from?

Everything the model was trained on, everything it can reach by searching the web, and - through ThAD - everything you’ve written. Only that last part is ours. The first two belong to the model and its built-in tools, and they’d be there with or without us.

What if I don’t want it offering to write things for me?

Tell it. Ask Claude to remember that you don’t want it writing on your behalf, and that should land in its memory and stick. The same goes for most preferences about how it engages with you - tone, length, how much it volunteers. I’ve asked mine to be just a bit snarky.

On the ThAD side, I’m adding stronger guardrails around unsolicited drafting - one of the more useful things I’m learning from the beta. In the meantime, saying so directly works.


This page began as an email to a ThAD beta tester who asked where the answers come from. If you have a question it doesn’t answer, ask me - those emails are how this gets written.

- Jim Marlow
Marlow Technology Advisors

See It With Your Own Archive

All of the above gets a lot more concrete when the archive being searched is yours.