Loading report…
← nittim
← nittimshared report
README badgeBadge for your project's readme fileREADME badge
[](https://nittim.com)[](https://nittim.com)Critical issues block production use. Do not ship as-is.Serious problems block using this for real users. Don't make it live as-is.Fix these serious problems before shipping.
This verdict covers the 48% of your repository we read — see what we didn’t read below.This result is based on the 48% of your project’s code we actually read — see what we didn’t read below.Covers 48% of your repo — see what we didn’t read below.
This audit read 220 of your 450 files — the ones most likely to hurt you first. The snapshot was truncated — this audit's size limit was reached.
The scores and findings above cover only those files. The rest of the repository hasn’t been audited yet. The denominator counts every file in the repository — images, generated output, vendored and build directories included.
.github/workflows/validate-corpus.ymlsets python-version: '3.x', and runs scripts/validate_corpus.py, run_safe_harnesses.py, run_cve_packet_suite.py.cases/python/src/reachable_import_time_callback.pycalls urllib.request.urlopen(...) at module top level; cases/python/src/reachable_setup_lifecycle_network.py calls run_setup_hook() at module scope; cases/ruby/src/reachable_gem_lifecycle_network.rb calls post_install at load.220 of 450 files were audited. This audit's size or file-count limit was reached, so some source and configuration files were left unread. The denominator counts every file in the repository — images, generated output, vendored and build directories included.
This audit's size limit was reached before it read the whole repository. The verdict and every score on this report were computed from what it did read — read them as evidence about that sample, not a clean bill of health for the rest.
The AI-generated likelihood estimate is based on 48% of repository files (220/450) — read it cautiously. The audited files were selected by priority, but a large portion of the repository was not seen.
security-defect-corpus is a PUBLIC benchmark/corpus repository whose declared and self-evident purpose (see SECURITY.md) is to hold deliberately vulnerable toy fixtures for defensive security evaluation. It contains ~80 paired 'reachable' (vulnerable) and 'unreachable/safe' fixtures across many languages, plus evaluation metadata (case splits, scoring rubric, surplus-capability schema), expected remediations, fixed-version fixtures, and negative-control evidence packets. Every 'vulnerability' the deterministic scanner and manual review surface — including the flagged hardcoded secret (a fake 'sk_live_51HARD_CODED_TEST_SECRET' placeholder) — is an intentional fixture, not an operational defect, and is therefore recorded under excluded_intentional_vulnerabilities rather than counted against the scores/verdict. The repository is not a deployable service; it is corpus + harness. Judged as a corpus, it is clean, consistent, and well-organized. The genuine (non-fixture) findings are minor: unpinned CI Python version, harness fixtures with import-time side effects, and the fact that a novel-ish evaluation methodology is exposed in a public repo and spelled out in public docs. No real credentials, PII, or live infrastructure were found (all hosts use .invalid/.test/.example). The audited snapshot was truncated (220/450 files); the actual harness scripts under scripts/ were not present, so operational maturity could not be fully verified.
docs/surplus-capability-metadata.schema.json(external capability-envelope ground truth, repo_local_claims_authoritative:false), docs/ai-agent-prompt-pack-run-rubric.json (weighted dimensions + score-cap gates), evaluation-splits/case-splits.json (deterministic contamination-aware hash-bucket splits with mandatory mutation axes for held-out). The consistent reachable/unreachable/fixed triad is a coherent evaluation design.docs/surplus-capability-metadata.schema.jsondescribes the mechanism: comparing behavior against an 'external capability envelope', enumerating allowed/disallowed capabilities, evidence_sources with trust tiers, and const repo_local_claims_authoritative:false; the rubric gate 'accepts_repo_local_authority' (score_cap 49) spells out the exact detection rule.Weighted blend of 4 dimensions → ~87/100. Dimensions are weighted by importance (Security and Privacy weigh heaviest). Published figures are approximate; nittim's exact weighting and precise scoring are proprietary, while the verdict is computed on exact internal values.4 categories combine into ~87/100. Some categories count for more than others (Security and Privacy count the most). The published numbers are approximate; nittim's exact math is kept private, while the verdict itself is worked out from the exact internal numbers.Weighted blend of 4 dimensions → ~87/100 (Security and Privacy weigh heaviest). Published figures are approximate; nittim's exact weighting is proprietary. The verdict runs on exact internal values.
IP Protection is an independent IP-defensibility lens and is excluded from this blend.IP Protection (checking whether your ideas are exposed) is judged separately and isn’t
part of this combined number.IP Protection is judged separately, excluded from this blend.
Scores are a judgement, not a measurement: a re-audit of the same commit can land a few
points either side of this figure. The evidence underneath it is deterministic — the same
commit always yields the same committed-secret and vulnerable-dependency findings.Scores are a judgment call, not an exact measurement: checking the same saved snapshot of
your code twice can land a few points either side of this number. The evidence underneath
it is not — the same snapshot always turns up the same passwords-in-code and known-security-bug
findings.Scores are a judgment, not a measurement — a re-audit can land a few points either side. The evidence underneath is deterministic: the same commit always yields the same secret and vulnerable-dependency findings.
Each score is the model's overall assessment of the dimensions it covers, across everything it reviewed — not a tally of the findings in this report. Findings are the subset worth acting on, so a score can sit above or below what the listed findings alone would suggest.Each score is the AI's overall judgment of everything it reviewed in that category — not a
count of the problems listed in this report. The listed problems are the ones worth acting
on, so a score can sit above or below what they alone would suggest.Each score is the model's overall assessment across everything it reviewed — not a tally of this report's findings. A score can sit above or below what the findings alone suggest.
nittim guardrail — nittim's safety check — nittim guardrail — The model proposed “ready_with_conditions”; nittim's deterministic layer set the final verdict to “not_safe” because of a committed production secret. The written rationale below reflects the model's proposal — the verdict above is the one that governs.
Dependency manifests: none found — the dependency scan had nothing to read.
Verdict rationale — Why this verdict — Verdict rationale — This repository is a deliberately-vulnerable security benchmark corpus, not a deployable application — SECURITY.md declares the intent explicitly and every 'vulnerability' (including the scanner-flagged secret, which is an obvious fake placeholder) is a paired, documented, non-deployable fixture using only non-resolvable .invalid/.test hosts. Those intentional defects are excluded from scoring per the corpus-purpose rule. Judged on what it actually is — a corpus plus validation/harness tooling — it is clean, consistent, well-organized, and contains no real credentials, PII, or live infrastructure. It falls short of unconditional 'production ready' only on minor operational items: an unpinned CI Python version that threatens benchmark determinism, reachable fixtures with import-time side effects that a careless harness could trigger, and the fact that the core harness scripts (scripts/) were outside the audited snapshot and so could not be verified. The IP findings are informational and do not affect the verdict. Address the CI pinning and confirm the harness treats fixtures as inert, and this is production-safe for its intended use.