# How shadow AI detection tools are scored

URL: https://shadowaidetectiontools.com/method/
Updated: 2026-11-16

One research plan was run for all 16 vendors on 16 Nov 2026, producing 1,469 facts across 32 vendor files, each quote checked against its page. A scorer worked blind from those files, a second blind reviewer checked every score, and the weights were committed before any evidence or score existed.

## The weights

The weights were fixed on 2026-11-16, before any score existed, and have not changed since.

**Criteria and weights (out of 100)**

| Criterion | What it measures | Weight |
|---|---|---|
| What it discovers | The kinds of unapproved AI it finds, as documented: AI web apps, desktop AI apps, coding assistants and command-line agents, browser extensions, IDE plugins, MCP servers, AI features embedded in SaaS, and AI apps granted OAuth access | 25 |
| Where it looks | The vantage points it reads from, as documented: the endpoint, the browser, network or proxy traffic, the identity provider, SaaS APIs and logs, email or billing records; more vantage points and named integrations score higher | 20 |
| Time to a first inventory | How quickly a first AI inventory appears after setup, as published, and how light the setup is (agentless, API connection, extension or agent), with operating systems and browsers covered | 15 |
| What it can do about a finding | Actions on a discovered tool, as documented: block, warn or coach the user in the moment, require approval, revoke OAuth grants, redact or stop data sent to the tool, route to a ticket | 15 |
| Risk context and reporting | A risk rating per AI tool found (data use, training on inputs, vendor posture), who uses it and how much, and an exportable inventory or audit report | 10 |
| Transparency | Public documentation readable without sign-in, a published price, published research | 10 |
| Proof | Named customers, funding, independent analyst coverage | 5 |

## Weights by situation

**Weights used for each situation view**

| Situation | What it discovers | Where it looks | Time to a first inventory | What it can do about a finding | Risk context and reporting | Transparency | Proof |
|---|---|---|---|---|---|---|---|
| [A regulated company that must show auditors every AI tool touching customer data](/best-for/regulated-inventory/) | 25 | 20 | 5 | 15 | 25 | 5 | 5 |
| [A CISO who must hand the board an inventory of AI in use within 30 days](/best-for/board-30-days/) | 30 | 20 | 30 | 5 | 5 | 5 | 5 |
| [An engineering organization where developers add coding agents, IDE plugins and MCP servers on their own](/best-for/developer-heavy/) | 35 | 25 | 10 | 15 | 5 | 5 | 5 |
| [A SaaS-heavy company where AI features switch on inside approved apps and AI apps get OAuth access](/best-for/saas-embedded/) | 25 | 30 | 10 | 15 | 10 | 5 | 5 |
| [A company that wants staff to keep using AI and be guided to approved tools rather than blocked](/best-for/coach-not-block/) | 20 | 15 | 10 | 35 | 10 | 5 | 5 |

## Score anchors

- **0**: none found on the read date
- **1-3**: mentioned, partial, or limited to one surface, one layer or one action type
- **4-6**: documented and working, with gaps the documentation shows
- **7**: the most a vendor claim or vendor documentation alone can earn: detailed, specific, covering the whole criterion
- **8-10**: detailed and confirmed by a second source (third-party write-up, or observed directly in public documentation, code, a marketplace listing or a release record)

## Rules

- Every vendor is scored 0 to 10 on every criterion, whole numbers, from the evidence file only, by a scorer that works from coded packets with every vendor name removed.
- A vendor claim scores at most 7 until a second source confirms it (kind third-party or observed).
- None found scores 0 and is written "none found on the read date".
- Breadth facts (module counts, integration counts, headcount, funding) inform Proof only, which weighs 5, so they carry at most 10 percent of any ranking.
- The total is the weighted sum of the scores divided by 10 (0 to 100 scale, published to one decimal).
- Equal totals print as a joint rank (for example joint 2nd); inside a joint rank vendors are listed alphabetically for display only.
- Once the first score exists, no weight and no criterion changes. A score changes only when a sourced fact changes, logged with the fact.

## How were the facts gathered?

One research plan was applied to every vendor on 16 Nov 2026: the vendor's own pages, documentation, pricing, release notes and at least two third-party hosts, such as news outlets and analyst or marketplace listings. Each fact records its claim, a verbatim quote, a source URL, the read date and a kind (vendor, third-party, observed or none found).

The run produced 1,469 facts across 32 vendor files (758 of them in this site's 16 files), and each quote was checked against its source. Where nothing was found, the file says none found on 16 Nov 2026 instead of leaving a gap, so an absence is visible and sourced.

## How are shadow AI detection tools scored?

Each vendor receives a whole-number score from 0 to 10 on seven criteria: what it discovers, where it looks, time to a first inventory, what it can do about a finding, risk context and reporting, transparency, and proof. The total is the weighted sum of the scores divided by 10, published to one decimal on a 0 to 100 scale.

Scoring was blind. One scorer worked only from the evidence files with vendor names replaced by codes, then one blind reviewer checked each score against the same files. Every score carries the fact IDs it relies on and a one-sentence reason, both printed on the vendor page.

## Why were the weights fixed before any score?

The weights and criteria were committed on 16 Nov 2026 before any vendor evidence was gathered and before any score existed, so the scale could not be adjusted to favor a result. They are: discovers 25, where 20, time to a first inventory 15, acts 15, context 10, transparency 10, proof 5.

Once the first score existed, no weight and no criterion changed. A score changes only when a sourced fact changes, and the change is logged with that fact in the changelog. Five situations reweight the same scores for different buyers, and their weights were fixed in the same commit.

## What counts as evidence?

Evidence is a sourced fact in a vendor's file. A vendor claim scores at most 7 on a criterion until a second source, either third-party coverage or something observed directly in public documentation, a marketplace listing or a release record, confirms it. A score of 8 to 10 needs that confirmation.

None found scores 0 and is written as none found on the read date. Breadth facts such as module counts, integration counts, headcount and funding inform Proof only, which weighs 5, so they carry at most 10 percent of any ranking. The anchors table below lists the meaning of each score band.

## How do I request a correction?

Email editor@shadowaidetectiontools.com with the page, the sentence or number, and a source URL that shows the fact. A vendor may also send a primary source that supports a higher score.

A corrected fact is re-read, the score is revised only if the fact changes, and the change is recorded with the date in the changelog and on the corrections page.

## Changelog

- 2026-11-16: weights and criteria fixed.
- 2026-11-16: evidence read for every vendor by one research plan.
- 2026-11-16: scores published (blind: one scorer, then one blind reviewer).
- 2026-11-16: this site built from the committed scores.

## Related pages

- [The ranking these weights produce](/ranking/)
- [Vendor profiles with evidence per score](/vendors/)
- [Editorial policy](/editorial-policy/)
- [Corrections](/corrections/)
- [Glossary](/glossary/)
- [Vantage Point Matrix](/vantage-point-matrix/)
- [Finding-to-Action Ladder](/finding-to-action-ladder/)
