In the plugins level you installed them. Now the other side: releasing your own in a way people can trust. There are two kinds of checks, and publishing turns out simpler than it seems.
Two checks
claude plugin validate reads files: manifests and frontmatter. With --strict, even a warning fails the check.
claude plugin validate --strict ./plugins/commit-helper
claude plugin validate --strict .
Check both the plugin and the marketplace: validating the marketplace looks at its file and each plugin’s plugin.json, but does not look inside skill files.
claude plugin eval measures behaviour. Each case is a prompt plus graders — checks that score the run. A case runs in a fresh, isolated session where only your is loaded: by default three times with it (the WITH arm) and as many times without it (W/OUT). The difference between their scores is Δ, the plugin’s own contribution.
The first case
cd plugins/commit-helper
claude plugin eval init --bare names-the-rename
--bare runs nothing: it creates prompt.md and a graders/criteria.md draft — a rubric for a judge model. It is usually replaced with free checks.
prompt.md is the prompt, with frontmatter:
---
max_turns: 10
allowed_tools: [Skill]
---
Write me a commit message for this change: I renamed getUser to fetchUser and updated the three call sites.
A run gets only the tools listed, so allowed_tools: [Skill] is what lets Claude run the skill. Two checks in graders/:
---
type: tool_used
tool: Skill
input_match: 'commit-message'
---
---
type: regex
pattern: 'fetchUser'
---
The first means “the skill ran”, the second “the answer contains a fact from the prompt”. Claude words things differently each time, so the regex looks for what the prompt supplied, not a specific phrase.
Which graders are free
| Grader | Cost |
|---|---|
regex, tool_used, tool_order, file_exists | free |
llm, baseline | a judge model call in every run |
A good rule: one check on the result and one on the steps. Keep the judge for what free checks cannot see.
Running it
A cheap trial run — once, with the plugin only, the report stays with you:
claude plugin eval . --runs 1 --ablation none --no-publish
A full run with a threshold:
claude plugin eval . --threshold 0.8 --no-publish
Without --threshold, any case below a perfect score counts as failed. Results land in evals/results/ — keep those out of git, and commit the cases themselves.
One trap. The check “the skill did NOT fire on an unrelated request” is tool_used on Skill with min: 0 and max: 0. But in a run with two arms, tool_used graders on Skill do not count by default: without the plugin there is no skill. For the check to count, add arm: both.
Publishing and releasing
To publish is to put .claude-plugin/marketplace.json in a git repository and push. No application forms: whoever can clone the repository can install from it.
claude plugin marketplace add owner/claude-plugins
claude plugin install commit-helper@owner-plugins
The marketplace’s name comes from marketplace.json, not from the repository’s name. Some marketplace names are reserved — for example, anything starting with claudeai-.
A new release: bump version in plugin.json (and do not set a version of your own in the marketplace entry), push, and users update with:
claude plugin update commit-helper@owner-plugins
It ends with the line Restart to apply changes. If it says the version is already the latest, the new version has not been pushed yet.
Treat the plugin’s name as permanent: it is installed and configured as name@marketplace, and a renamed plugin is a different plugin for everyone. displayName exists for a new label.
Quick check
The eval table shows WITH 0.9, W/OUT 0.9, Δ 0. What does that mean?
Careful
claude plugin eval runs the plugin’s skills, hooks and agents on your machine, in your name, and a green run says nothing about safety. And a hook or MCP server you add in the next version will run for everyone who updates, with their permissions. Before making a marketplace repository public, read every file in it. And remember: --max-cost-usd caps an estimate based on list prices, not your plan’s limit.
Practice in the terminal