class Analyzer::AI::Unified

Overview

Unified AI analyzer that uses a provider-agnostic LLM adapter. Supports both OpenAI-compatible APIs and Ollama.

Defined in:

analyzer/analyzers/llm_analyzers/unified_ai.cr

Constant Summary

AGENT_CONTEXT_MAX_CHARS = 100 * 1024
AGENT_CONTEXT_MAX_DYNAMIC_MESSAGES = 16
AGENT_DEFAULT_FILE_PATTERN = "*.{go,py,js,ts,java,rb,php,cs,cr,kt,rs,swift,scala,graphql}"
AGENT_GREP_SNIPPET_MAX_CHARS = 220
AGENT_MAX_DEPTH = 6
AGENT_MAX_READ_BYTES = 10 * 1024
AGENT_TOOL_CACHE_MAX_ENTRIES = 96
AGENT_TOOL_MAX_LINES = 300
AGENT_TOOL_MAX_MATCHES = 200
AGENT_TOOL_RESULT_MAX_CHARS = 16 * 1024
IGNORE_EXTENSIONS = [".css", ".xml", ".json", ".yml", ".yaml", ".md", ".jpg", ".jpeg", ".png", ".gif", ".svg", ".ico", ".eot", ".ttf", ".woff", ".woff2", ".otf", ".mp3", ".mp4", ".avi", ".mov", ".webm", ".zip", ".tar", ".gz", ".7z", ".rar", ".pdf", ".doc", ".docx", ".xls", ".xlsx", ".ppt", ".pptx", ".txt", ".csv", ".log", ".sql", ".bak", ".swp", ".jar"] of String
LLM_NO_RESPONSE_REASON = "the AI provider returned no usable response (see the WARNING lines above for the cause)"

Every adapter maps a call it could not complete — an HTTP error the retries did not clear, a provider error body, a dead ACP agent, a reply that would not parse — to an empty string, and each one already prints its own WARNING naming the cause. The empty string is what reaches here, so this is the reason the analyzer can attach to the lost coverage.

MAX_BUNDLE_WORKERS = 4

Ceiling on simultaneous bundle requests, independent of a high --concurrency: past a handful of in-flight calls a metered provider answers with 429s rather than faster.

MAX_ENDPOINT_URL_LENGTH = 2048
MAX_PARAM_NAME_LENGTH = 128
PLACEHOLDER_URLS = Set {"url", "uri", "endpoint", "n/a", "na", "none", "null", "nil", "undefined", "your_endpoint", "your-endpoint", "path/to/endpoint", "...", "<url>", "<endpoint>", "<path>"}

Bare URL tokens the LLM emits when it has nothing real to report (schema echoes, "no endpoint" stand-ins). Compared case-folded against the whole path with a single leading slash stripped, so a legitimate nested route like /example/users is never rejected — only a URL that IS one of these placeholders is dropped.

VALID_METHODS = ["GET", "POST", "PUT", "DELETE", "PATCH", "OPTIONS", "HEAD", "QUERY"]
VALID_PARAM_TYPES = ["query", "json", "form", "header", "cookie", "path"]

Constructors

Class Method Summary

Instance Method Summary

Instance methods inherited from class Analyzer

analyze analyze, base_path : String base_path, base_paths : Array(String) base_paths, base_relative_path(path : String) : String base_relative_path, callees_needed? : Bool callees_needed?, content_matches?(content : String, markers : Regex) : Bool content_matches?, http_header_name(name : String) : String | Nil http_header_name, line_number_for_index(content : String, char_index : Int32) : Int32 line_number_for_index, logger : NoirLogger logger, parallel_analyze(files : Array(String), &block : String -> Nil) parallel_analyze, read_file_content(path : String) : String read_file_content, result : Array(Endpoint) result, tech : String tech, unique_params(params : Array(Param)) : Array(Param) unique_params, url : String url, web_root_path(path : String, markers : Array(String)) : String web_root_path

Constructor methods inherited from class Analyzer

new(options : Hash(String, YAML::Any)) new

Macros inherited from class Analyzer

analyzer_for(tech) analyzer_for

Instance methods inherited from module FileHelper

all_files : Array(String) all_files, get_files_by_basename(basename : String) : Array(String) get_files_by_basename, get_files_by_extension(extension : String) : Array(String) get_files_by_extension, get_files_by_extensions(extensions : Array(String)) : Array(String) get_files_by_extensions, get_files_by_prefix(prefix : String) : Array(String) get_files_by_prefix, get_files_by_prefix_and_extension(prefix : String, extension : String) : Array(String) get_files_by_prefix_and_extension, get_files_by_relative_path(relative_path : String, root : String = "") : Array(String) get_files_by_relative_path, get_public_dir_files(base_path : String, folder : String) : Array(String) get_public_dir_files, get_public_files(base_path : String, anchors : Array(String) = ["shard.yml", "Gemfile"]) : Array(String) get_public_files, walked_path(expanded : String) : String walked_path

Constructor Detail

def self.new(options : Hash(String, YAML::Any)) #

[View source]

Class Method Detail

def self.tech_name : String #

[View source]

Instance Method Detail

def analyze #

[View source]
def ignore_extensions #

[View source]
def max_tokens : Int32 #

[View source]
def path_within_base?(path : String) : Bool #

Containment gate for every agent file tool. The agent picks these paths from LLM output, and the LLM is steered by the source tree being scanned, so "stay inside the scan base" is a boundary an untrusted repo gets to push on.

File.expand_path is purely lexical — it folds .. and makes the path absolute but never follows a link. A checked-out repo containing notes -> /home/user/.ssh/id_rsa therefore passed as an in-base path, and read_file shipped the target's contents to the LLM provider. (The tree walk, the grep and the bundling path all skip symlinks already; these two entry points were the gap.)

So resolve both sides and require containment of the real path. A base that is itself reached through a link (/tmp -> /private/tmp on macOS) still matches, because @real_base_paths went through the same resolution. The lexical check stays as the first gate: it rejects the plain ../../etc/passwd case without touching the filesystem.

Public rather than private so the rule can be asserted directly: reaching it through a live LLM round trip is not a test.


[View source]
def tech : String #

Instance-side view of the same declaration. The per-file rescues live on this base class, which has no way to name the analyzer that is running inside them, so a skipped file could not be attributed to a tech. Deriving it from analyzer_for keeps the name written exactly once.


[View source]