Building First LLM

All Experiments with Large Language models begin with chat windows

You type a question, the model produces an answer, and yada yada. This serves the purpose, but is only useful in a world without engineers

Real LLM applications need to retrieve information, call external systems, handle failures, validate outputs, preserve state and explain where their answers came from.

To demonstrate these concepts, I built Pocket Powered Analyser, a local-first research agent written in Python.

The application accepts a research question such as:

What are the most important developments in AI agent safety?

Instead of immediately asking an LLM to answer, it executes a research workflow:

Question
  → plan searches
  → search several sources
  → fetch documents
  → extract evidence
  → remove duplicates
  → score source reliability
  → synthesise conclusions
  → generate a sourced report

The complete example is available in the Pocket Powered Analyser repository.

The difference between an LLM, a tool and an agent

Before walking through the project, let us clarify three commonly mixed-up terms.

LLM

An LLM receives tokens and generates more tokens (predictive tokens more on them in subsequent blogs) .

It does not inherently browse the web, read your database or remember previous requests.

Tool

A tool is a function the application can execute.

Examples include:

search_web(query)
fetch_url(url)
save_report(report)

Tools perform actions that the language model cannot reliably perform by itself.

Agent

An agent combines a model, tools, state and control flow (traditionally ReAct model)

It can decide or be instructed to:

formulate a query;
invoke a search tool;
examine the result;
perform another step;
stop when an objective has been reached.

Pocket Powered Analyser is closer to a workflow-oriented agent than an unconstrained autonomous agent. Its possible steps and transitions are defined in advance.

That makes it easier to understand, test and operate

So what are we actually building today?

Pocket Powered Analyser is not intended to be an all-knowing autonomous agent.

It is a bounded research workflow.

That distinction matters.

A basic chatbot follows an interaction similar to this:

response = model.invoke("Explain LangGraph") 
print(response)

A research agent has to perform several different kinds of work:

queries = plan_research(question) 
search_results = search_sources(queries) 
documents = fetch_documents(search_results) 
evidence = extract_claims(documents) 
report = synthesise(question, evidence)

The LLM is only one component in the system.

Search APIs retrieve information. Python code manages control flow. Pydantic validates data. LangGraph coordinates the workflow. Ollama allows the model to run locally.

This is a healthier mental model for learning agentic development:

The LLM provides probabilistic reasoning inside a deterministic software system.

Building blocks

Before examining the code, let us look at the main building blocks.

Configuration

The project uses configuration to decide which model and external services should be used.

A local configuration might contain:

LLM_PROVIDER=ollama 
OLLAMA_BASE_URL=http://localhost:11434 
OLLAMA_MODEL=phi4-mini:latest

Search-provider keys can also be supplied through environment variables.

LLM client

The LLM client is the application boundary around language-model providers.

Instead of letting every node create its own Ollama or hosted-model connection, the application creates a common client.

Conceptually:

model = get_model_from_configuration() 
response = await model.ainvoke(messages)

Research state

The workflow needs a shared object that records what has happened.

This is the research state.

At the beginning, only the user input is present:

As the graph executes, nodes return updates:

Planning node → adds query_plan
 
Search nodes → add raw_search_results
 
Fetch node → adds raw_documents
 
Extraction node → adds extracted_evidence
 
Synthesis node → adds synthesis
 
Report node → adds report_markdown

The state acts as the memory of one research run.

LangGraph workflow

LangGraph coordinates the execution of the research process.

The workflow is represented as nodes and edges.

A node performs work:

Conditional edges allow execution to depend on the current state:

Search adapters

Different information sources have different APIs and response formats.

Wikipedia is not queried in the same way as Reddit. A web-search provider does not return the same fields as a news provider.

Search adapters hide those differences.

Conceptually, they provide a common interface:

Document fetcher

A search result is not the same as a document.

Search APIs often return only:

Title URL Short snippet Source name

The snippet may not provide enough context for reliable evidence extraction.

The document-fetching stage attempts to retrieve the actual source content:

Search result ↓ Fetch URL ↓ Validate content ↓ Extract readable text ↓ Create RawDocument

Deduplication

Multiple sources may repeat the same information.

A news article may summarise a company announcement. Several blogs may repeat the same benchmark. Search results may also contain syndicated copies of one article.

Without deduplication, one repeated claim can appear more widely supported than it really is.

Report generation

The final stage converts structured synthesis into a human-readable report.

The report may include:

Research question Executive summary Key findings Supporting evidence Contradictory evidence Source list Warnings Research limitations

(Full architectural flow can be found here)

Running an LLM locally with Ollama

Many LLM agents start by asking for a paid API key. That can discourage experimentation and makes it harder to understand what is happening locally. This ~~elite~~ barrier is quashed by ollama

Ollama exposes locally installed models through an HTTP service. Pocket Powered Analyser supports Ollama as one of its model providers and documents a local configuration using OLLAMA_BASE_URL and OLLAMA_MODEL.

A typical local setup looks like this:

brew install ollama
ollama pull phi4-mini:latest

Now let's get our hands dirty

Clone the repository

git clone [email protected]:uncle-voh-max/pocket-powered-analyser.git
cd pocket-powered-analyser

Create the repository configuration:

cp .env.example .env
## Add your own tavily keys (https://app.tavily.com/home)
    ### Reddit keys and other

Then configure the model:

LLM_PROVIDER=ollama
OLLAMA_BASE_URL=http://localhost:11434
OLLAMA_MODEL=phi4-mini:latest

Install the Python project:

uv sync --extra dev

Run a research request from the command line:

uv run python -m research_agent.cli run \
  "What are the practical limitations of local LLM agents?"

Alternatively, run the FastAPI server:

uv run uvicorn research_agent.main:app \
  --host 0.0.0.0 \
  --port 8000 \
  --reload

Then submit a request:

curl -X POST http://localhost:8000/research \
  -H "Content-Type: application/json" \
  -d '{
    "question": "What are the practical limitations of local LLM agents?",
    "max_results_per_source": 3,
    "include_sources": ["web", "reddit", "wikipedia"]
  }'

The project also includes mock search adapters, allowing the workflow to execute without every external API key being configured.

Project quirks

The LLM client: keeping providers behind one boundary

One of the first useful abstractions in the repository is the LLM client.

The client builds a model from configuration rather than letting every part of the codebase initialise its own provider:

def _get_model() -> BaseChatModel:
    kwargs = {
        "model": settings.effective_llm_model,
        "temperature": settings.llm_temperature,
    }
 
    if settings.llm_provider == "ollama":
        kwargs["model_provider"] = "ollama"
        kwargs["base_url"] = settings.ollama_base_url
    else:
        kwargs["model_provider"] = settings.llm_provider
 
    return init_chat_model(**kwargs)

This boundary gives the application an important property: the research workflow does not need to know whether the model is running through Ollama or a hosted provider.

Structured output

Suppose we ask a model to produce a search plan.

A model may return:

I think you should search the web for recent research and perhaps Reddit for opinions.

That is readable, but difficult for code to use.

The application needs something closer to:

{
  "original_question": "What are the limitations of local LLM agents?",
  "research_objective": "Identify technical and operational limitations",
  "search_plan": [
    {
      "source": "web",
      "query": "local LLM agent limitations"
    }
  ]
}

Pocket Powered Analyser uses Pydantic response models for structured LLM output. Providers with native structured-output support can return validated objects directly; local providers can fall back to parsing model-generated JSON.

Search adapters as tools

The project defines source-specific adapters for news, general web search, Reddit, Hacker News-style social content and Wikipedia. The adapters share a common interface for searching and fetching.

The architectural idea is simple:

class BaseSearchAdapter:
    async def search(self, plan):
        ...
 
    async def fetch(self, result):
        ...

The research workflow does not need to understand the details of Reddit OAuth, MediaWiki requests or a web-search provider.

It asks an adapter to search:

adapter = _get_adapter(source_type)
results = await adapter.search(query_plan.search_plan)

This is ordinary software engineering applied to an LLM application.

Agentic development does not replace interfaces, dependency inversion or test doubles. It makes them more important.

What this project teaches a beginner

A small but complete project teaches more than isolated prompt examples.

By walking through Pocket Powered Analyser, we encounter:

local model execution with Ollama;
structured output with Pydantic;
external tools and adapters;
retrieval and extraction;
API and CLI interfaces;

Most importantly, it teaches that an LLM application is still an application.

It needs contracts, state, observability and tests. The model is powerful, but it is not the architecture.

Exercises

After running the repository, try these changes in order.

Exercise 1: Add a source

Create an arXiv adapter that searches papers and fetches their abstracts.

Exercise 2: Compare models

Run the same question through two Ollama models and compare:

planning quality;
valid JSON rate;
evidence extraction accuracy;
total response time.

Exercise 3: Save intermediate artefacts

Persist the query plan, raw search results and extracted evidence, not only the final report.

Exercise 4: Introduce one controlled failure

Make a search adapter time out and observe whether the final report clearly communicates that the source was unavailable.

These exercises move learning away from “Which prompt should I use?” and towards the more important question:

How should I design a system when one of its components is probabilistic?

In the next article, we will examine the mechanism coordinating the entire pipeline: LangGraph.

Building LLM Router

Beating The Black Terminal