Plugin-Probe

About

Plugin-Probe — Scan LLM Plugins for Security Risks

Plugin-Probe automatically scans GitHub repositories for LLM plugin security risks and shows a trust badge directly on the page — no setup, no servers, no data collection.

--- THE PROBLEM ---

We scanned 100 publicly available LLM plugins and found that 54% had at least one security issue. These risks range from subtle prompt injection vulnerabilities to malicious code patterns and outdated dependencies with known exploits. Most developers and users have no visibility into these risks before installing or integrating a plugin.

--- HOW IT WORKS ---

1. Visit any GitHub repository containing an LLM plugin.
2. Plugin-Probe automatically fetches and analyzes the code using the GitHub API.
3. A color-coded badge appears in the top-right corner of the page:
   - Green  = SAFE — no significant issues detected
   - Yellow = SUSPICIOUS — medium-risk patterns worth reviewing
   - Red    = SUSPICIOUS — high-risk patterns found, review before installing

Click the badge to rescan. For detailed findings, run the CLI command shown below the badge.

--- WHAT IT DETECTS ---

Plugin-Probe scans for three categories of security risk:

1. Prompt Injection Risks
   Patterns that could allow malicious instructions to hijack LLM behavior — system prompt leaks, jailbreak vectors, and unsafe instruction handling.

2. Code Malware Patterns
   Suspicious code constructs like obfuscated payloads, unexpected network calls, dangerous eval usage, and exfiltration-style patterns.

3. Dependency Risks
   Outdated or flagged packages in requirements.txt, package.json, and other dependency files that have known vulnerabilities.

--- TRUST SCORE ---

Every scan produces a clear verdict:
- Safe: The plugin passed all checks with no significant findings.
- Suspicious (yellow): Medium-risk patterns detected — worth reviewing before use.
- Suspicious (red): High-risk patterns detected — review carefully before installing.

Smart detection recognizes security tools, developer tools, and test files to reduce false positives. Real-time scan progress shows exactly how many files have been analyzed.

--- PRIVACY FIRST ---

Everything runs locally in your browser. Plugin-Probe does not send your data anywhere, does not track you, and does not require an account. Scan results are stored only in your local browser storage.

Optionally add a GitHub token in settings to avoid API rate limits — your token stays on your device.

--- FREE AND OPEN SOURCE ---

Plugin-Probe is free to use and fully open source. Contributions welcome.
https://github.com/prateekparshwa/llm-plugin-malware-scanner