School of Data Analysis. Agent intensive

School of Data Analysis. Agent intensive

There is an informational storm in the channels I read about agentic programming. The School of Data Analysis released a free course on agentic programming, and a lot of people are discussing it. Mostly, they are trying to find gaps in it.

For example:
🐞 Polyakov, tools and MCP or
🐞 Kovalsky, on Polyakov

The idea of this channel is to share things from my perspective, so I want to share my conspect of the course. These are the things that made a difference for me personally, given my current background in this area.

Let's start.

Agent

It's a totally unstable concept. Of course, you need an LLM (large language model), some prompts to guide it, tools to call, memory, guardrails (for safety), and planning skills (to fill the gap in the LLM's ability to make plans).

For me, guardrails and planning skills are the most interesting things to hear about.

LLM

The lecture said that an LLM is basically two files. One huge file with parameters, and one small program that runs it. For me, this statement is important because it demystifies the technology. Just two files. That's it.

Karpaty's LLM OS

I had read about this idea several times before. But now it really clicked.

In an agent, the LLM works like a CPU on a motherboard. It processes data in different modalities, acts through tools, and performs Input/Output operations. This "OS" thing sounds wrong to me. An OS is the first program that starts on your computer when you turn it on. A CPU on a motherboard suits this analogy much better.

Special tokens

You know, an LLM can't see letters. It sees tokens. Each token is a group of letters. This helps optimize both training and inference.
I already knew that. But for me there was still a gap between the JSON I send to the LLM API and the array of tokens actually fed into the model. There is one element that makes this gap narrower: special tokens.
I knew there were special tokens to start and stop generation. But it turned out that there are also tokens for roles in conversations and for actions, like text translation.