class Detector

Direct Known Subclasses

Defined in:

models/detector.cr

Constant Summary

CONTENT_MATCH_OPTIONS = Noir::TextFile::MATCH_OPTIONS

Detector content always arrives from Noir::TextFile.read, so the subject is known-valid UTF-8 and PCRE2's per-call revalidation is skippable. See Noir::TextFile::MATCH_OPTIONS.

Constructors

Macro Summary

Instance Method Summary

Constructor Detail

def self.new(options : Hash(String, YAML::Any)) #

[View source]

Macro Detail

macro detector_for(tech, extensions = nil, basenames = nil, path_segments = nil, idempotent = nil) #

Declares the detector's tech name and the cheap filename gate that lets the detect loop skip #detect on files this detector cannot match.

class Gin < Detector
  detector_for "go_gin", extensions: %w[.go], basenames: %w[go.mod]
end

Expands to the same short-circuiting chain the detectors used to write by hand — one ends_with? / == per declared term, not a runtime loop over an array — so the hot path is unchanged. Terms are emitted extension, then path segment, then basename: extensions reject the most files for the least work, and only a surviving candidate pays for File.basename.

A path_segments term that names a directory ("/metadata/") implies #path_sensitive?, and that link is the whole point. #applicable? is memoized by basename, so a detector that consults directory segments without declaring itself path-sensitive has its gate silently deleted — that is how the Hasura metadata/** gate was lost. Declaring the segment is declaring the sensitivity, so the two can no longer drift apart.

A separator-free term ("go.mod") is a plain substring test on the whole path and deliberately does not imply sensitivity, because Noir::Detection.path_sensitive? does not flag those today: the probe compares #applicable?("a/b/c/go.mod") with #applicable?("go.mod"), which agree. Declaring them sensitive would drop those detectors out of the basename memo and slow the hot loop for no correctness gain here.

Pass idempotent: false for a detector whose #detect has side effects (registering spec paths in CodeLocator), so the pass keeps calling it after its first match.

A gate that is more than a term list — one that normalises separators, checks a parent directory, or excludes .d.ts — keeps its hand-written #applicable?. Pass just the tech name and define the method below; with no terms the macro emits no gate to collide with.


[View source]

Instance Method Detail

def applicable?(filename : String) : Bool #

Cheap filename-only filter the detector pass uses to skip #detect on files the detector cannot possibly match. The default true preserves prior behavior (every detector runs on every file). Override with the same predicate the body of #detect starts with — e.g., filename.ends_with?(".py") for a Python framework detector — so the detector loop avoids the #detect dispatch on files outside the detector's language.

On large codebases (saleor's 4255 .py files) this lifts ~100 virtual #detect calls per file out of the hot loop because most detectors' inner first-line is exactly this kind of cheap filename check.


[View source]
def base_relative_path(filename : String) : String #

filename relative to the scan base that owns it, /-separated and rooted with a leading /.

A detector that keys off a DIRECTORY (wp-content/, vendor/yiisoft/, routes/) rather than a filename must match on this: on the absolute path, a checkout that merely sat under a same-named directory made every file in the project look like the framework's. Filename markers (composer.json, Cargo.toml) are unaffected either way — an ancestor directory contributes no filename.

Reads the roots from CodeLocator, which NoirRunner#detect publishes before the walk; with none registered (detector unit specs) the path is returned unchanged.


[View source]
def content_matches?(file_contents : String, markers : Regex) : Bool #

Whether any alternative of a precompiled Regex.union appears in file_contents. With a union of plain literals this is exactly OR-ing String#includes? over the same literals — Regex.union escapes every String argument — but it costs one pass instead of N.

String#includes? runs Rabin-Karp: a rolling hash over every byte position, restarted for each marker. PCRE2 JIT-compiles the union into a program that skips ahead on a start-byte bitmap. Measured on a 467 KB locale file against four non-matching markers: 2.16 ms of chained includes? versus 26 µs here (82x). Even a single marker over 300 small Ruby files is 13x, so the win is not a big-file artifact.

Use this for the marker sweep at the top of #detect. A union is not a substitute for a real pattern: keep purpose-built regexes as they are, and keep a single includes? that merely gates an expensive parse (there the substring check is the cheap half, not the cost).


[View source]
def detect(filename : String, file_contents : String) : Bool #

[View source]
def gemfile_dependency?(file_contents : String, gem_name : String) : Bool #

Tolerant matcher for a Gemfile gem "<name>" line. Accepts both the bare and parenthesized call forms with arbitrary spacing — gem 'x', gem "x", gem('x'), gem( "x" ) — and a trailing version constraint, while still requiring the closing quote right after the name so gem 'sinatra' never matches gem 'sinatra-contrib'.


[View source]
def gemspec_dependency?(file_contents : String, gem_name : String) : Bool #

Tolerant matcher for a gemspec runtime dependency on <name>, in either the space or parenthesized call form — gems routinely write s.add_dependency('sinatra', "~> 4.0") (geminabox) or spec.add_runtime_dependency "railties", neither of which the old "add_dependency 'sinatra'" substring markers matched.


[View source]
def idempotent? : Bool #

Whether the detector can be skipped on subsequent files once it has matched. Defaults to true (idempotent — the detector only signals tech presence). Detectors that perform side effects in #detect (e.g., the C# ASP.NET ones populate the CodeLocator with route-config paths, the OAS/RAML detectors register spec paths) must override to false so the detector pass keeps invoking them on every file.


[View source]
def logger : NoirLogger #

[View source]
def name : String #

[View source]
def path_sensitive? : Bool #

Whether #applicable? inspects more than the basename — directory segments (/metadata/, /.kamal/) or root placement.

The detect loop memoizes #applicable? by basename, so a detector that consults the path but does not declare it here has its path gate silently deleted: #applicable? is only ever asked about the bare filename. That is a false-negative, not a crash, so nothing fails loudly. Noir::Detection.path_sensitive? also probes for this, but the probe is fail-open — a probe whose basename independently matches masks the directory gate behind it (this is exactly how the Hasura metadata/** gate was lost). Declare it explicitly.

Guarded by spec/unit_test/detector/applicable_lookup_fidelity_spec.cr.


[View source]
def record_unparsable_document(filename : String, error : Exception) : Nil #

A document that matched this format's content marker but could not be parsed at all.

The spec detectors wrap "parse, then check the root key" in one rescue, and the two halves fail for very different reasons. data["openapi"].as_s raising KeyError / TypeCastError means "this JSON is simply not my format" — the gate doing its job, and reporting it would flag every document that merely mentions the word. A JSON::ParseException / YAML::ParseException is the other case: the file is not readable as JSON/YAML by anything, so it is never registered, no analyzer ever opens it, and every endpoint it declares is lost.

That half used to leave a --debug line and nothing else. An OpenAPI document nested deeper than Crystal's 512-level JSON.parse ceiling — or one truncated by a failed download — reported zero endpoints, "errors": [], and exit 0 under --strict, which is indistinguishable from a repository that declares no API at all.


[View source]