When not to create a skill: processes that needed automating, not packaging

We expected skills to save us work on the company’s internal processes, and in quite a few cases they did the opposite: they added another layer to something that was already confusing. Many skills that looked like a good idea at first ended up as unnecessary complexity on top of processes that actually needed to be defined or automated.

It wasn’t a technical failure. The skills triggered, the agent followed the instructions and the result was reasonable. The problem came earlier: we had picked the wrong tool for the problem.

Why a skill always looks like a good idea

Creating a skill is cheap. You write a SKILL.md, describe the process, install it and the agent knows how to do it. When a team discovers this, every repetitive task looks like a candidate, and the temptation is reinforced because you see the result right away: you ask the agent for something and it does it.

What you don’t see as quickly is what you’ve stopped doing. If a process had never been written down clearly, packaging it in a skill doesn’t define it; it moves the ambiguity to a place where the model resolves it, every time, in its own way. And if a process is always the same, putting it in a skill turns something deterministic into something probabilistic.

Three different tools for three different problems

Over time I’ve ended up separating three things we used to mix up at the start.

A procedure is a process written for people: who does what, in what order, with what criteria and what to do when something goes wrong. Many company processes fail because nobody has written that down, and no tool fixes it. First you have to decide how it’s done.

An automation is a script, a pipeline or a job that always does the same thing. If the steps don’t change and the decisions can be expressed as rules, a script is faster, cheaper and more reliable than an agent. It can be tested, versioned and reviewed in a PR. At work we use n8n less and less for this reason: with AI a script takes minutes and, once dockerized, it deploys like any other service. AI helps write the automation, but it doesn’t have to run it.

A skill makes sense when the task requires judgment the agent doesn’t have: interpreting an ambiguous request, choosing between options depending on context, working in a domain the model doesn’t know. The Illuminate skill is a good example: every SQL query a user asks for is different, and what the skill brings is the schema and the rules for not inventing tables. That can’t be reduced to a script.

Signs that a skill is the wrong answer

I don’t have a foolproof rule, but I do have a few questions I now ask myself before proposing a new skill.

Are the steps always the same? If the skill describes a fixed sequence with no decisions, what you’re asking is for the agent to act as a script, at a higher cost and with fewer guarantees. Write the script; if needed, let the agent call it.

Could someone explain the process in five minutes? If nobody can, the skill is going to encode the current confusion. For example, a skill that says “check with the person in charge whether the customer has special terms” in a process where nobody knows who that person is doesn’t help the agent; it documents a gap.

Is the skill making up for a problem somewhere else? Sometimes the skill exists because an internal tool is awkward or a format is badly designed, and the agent acts as an interpreter. Fixing the source is more useful.

How many people would use it? A skill that only solves a case that comes up twice a year takes up space in the context of every session in exchange for very little.

The hidden cost: skills step on each other

Every installed skill adds its description to the agent’s context, and each description competes with the others to be triggered. With a few skills you don’t notice. When a team keeps adding them without criteria, you get similar descriptions claiming the same request, activations where they don’t belong and heavier sessions without anyone having changed anything specific.

In our case, the marketplace we use in the department goes through continuous improvement: we add skills and commands as needs come up. What we’ve learned is that the hard part isn’t updating, it’s keeping things in order. Someone has to lead what goes in and what doesn’t, there has to be a guide on when something deserves to be a skill, and new ones have to be checked for overlap with the ones that already exist. Without that, the marketplace grows and the agent works worse.

Activation evals help detect overlaps: if one request triggers two skills, one of the two descriptions is badly scoped or one of them is redundant. But the decision not to create a skill comes before any eval, and it’s an organizational one.

What I would do from the start

Before creating a skill, I’d write the procedure in a normal document. If, while writing it, I find the steps are fixed, I’d turn it into a script. If I find nobody knows how it’s done, the pending work is deciding that, not packaging it. Only if there’s a part left that requires judgment over knowledge the model doesn’t have would that part become a skill, and probably a smaller one than I would have written at first.

Skills are a good tool, and this series has several articles on how to build them well. But the criteria for agent skills in a team include knowing when the answer is something else, and in company processes that has happened more often than I expected. If you’re torn between a skill and a script for a specific process, write the script first: if it falls short, you’ll know exactly which part needs judgment.