class Analyzer::Python::Cli

Overview

Surfaces the command-line attack surface of Python programs as cli:// endpoints: one endpoint per (sub)command, with named options (param_type "flag"), positional arguments ("argument"), and consumed environment variables ("env"). Covers stdlib argparse / getopt / sys.argv plus click, typer, fire, docopt, Abseil (absl-py) flags, and Cleo commands.

Line-scan analyzer (the house style for non-tree-sitter Python adapters, e.g. bottle). Endpoints are merged by URL so options registered across decorators/functions collect onto a single command.

Included Modules

Defined in:

analyzer/analyzers/python/cli.cr

Constant Summary

ABSL_DEFINE_CALL_RE = /\bflags\.DEFINE_(?:string|integer|bool|boolean|float|enum|list|multi_string|multi_integer|multi_enum)\s*\(/

Abseil (absl-py): flags.DEFINE_* calls register a single flat flag namespace consumed via FLAGS. after app.run(main) — there is no subcommand concept, so every flag attaches to the binary's root URL.

ABSL_DEFINE_NAME_RE = /\bflags\.DEFINE_(?:string|integer|bool|boolean|float|enum|list|multi_string|multi_integer|multi_enum)\s*\(\s*[rf]?["']([^"']+)["']/
ADD_ARGUMENT_RE = /(\w+)\s*\.\s*add_argument\s*\(([^)]*)/
ADD_PARSER_RE = /(\w+)\s*=\s*(\w+)\.add_parser\s*\(\s*[rf]?["']([^"']+)["']/
ARGPARSE_NEW_RE = /(\w+)\s*=\s*argparse\.ArgumentParser\s*\(/

argparse

ARGPARSE_PROG_RE = /ArgumentParser\([^)]*\bprog\s*=\s*[rf]?["']([^"']+)["']/
CLEO_ARGUMENT_CALL_RE = /\bself\.argument\s*\(\s*[rf]?["']([^"']+)["']/
CLEO_COMMAND_CLASS_ANYWHERE_RE = /(?:^|\n)class\s+\w+\s*\(\s*Command\b/
CLEO_COMMAND_CLASS_RE = /^class\s+(\w+)\s*\(\s*Command\b/

^ in Crystal/PCRE without the multiline flag anchors to the start of the whole subject, not each line — fine for the per-line line.match use in scan_cleo below, but a separate (?:^|\n)-anchored variant is needed to test this against the full multi-line file content. Both variants are column-0 anchored (no \s* before class) so the entrypoint gate below matches exactly what scan_cleo can extract — a class Foo(Command) nested inside a function/method (indented) is never resolved by scan_cleo, so it must not satisfy the gate either, or the file gets treated as a CLI entrypoint with no real command found (scan_stdlib then fires unconditionally on an unrelated env/argv read).

CLEO_IMPORT_RE = /(?:^|\n)\s*(?:import\s+cleo\b|from\s+cleo(?:\.\w+)*\s+import\b)/

Cleo: a Command subclass declares its subcommand name as a class attribute, then reads its own arguments/options back out by name inside its methods (self.argument("x") / self.option("y")) — that readback call is also the most reliable textual anchor for the option/argument name itself.

CLEO_NAME_ATTR_RE = /^\s*name\s*=\s*[rf]?["']([^"']+)["']/
CLEO_OPTION_CALL_RE = /\bself\.option\s*\(\s*[rf]?["']([^"']+)["']/
CLI_LITERAL_MARKERS_RE = Regex.union("argparse.ArgumentParser", "click.command", "click.group", "typer.Typer", "fire.Fire", "getopt.getopt", "docopt")

The seven literal markers below were seven separate Rabin-Karp passes over the whole file, run for every .py in the tree. One precompiled union is the same predicate in a single pass. The remaining tests go through content_matches? so PCRE2 skips its per-call UTF-8 revalidation of the buffer. The gate is a pure OR, so collapsing and reordering its terms cannot change the answer.

CLICK_ARGUMENT_RE = /^@\w+\.argument\s*\(/
CLICK_OPTION_RE = /^@\w+\.option\s*\(/
DECORATOR_CMD_RE = /^@(\w+)\.(command|group|callback)\s*\(/

click / typer decorators

DECORATOR_NAME_RE = /\(\s*[rf]?["']([^"']+)["']/
DEF_RE = /^def\s+(\w+)\s*\(/
ENVVAR_RE = /\benvvar\s*=\s*[rf]?["']([^"']+)["']/
FIRE_RE = /\bfire\.Fire\s*\(\s*(\w+)/
GETOPT_RE = /getopt\.getopt\s*\([^,]+,\s*[rf]?["']([^"']*)["']\s*(?:,\s*\[([^\]]*)\])?/
MAIN_GUARD_RE = /if\s+__name__\s*==\s*[rf]?["']__main__["']/
OS_ENVIRON_GET_RE = /\bos\.environ\.get\s*\(\s*[rf]?["']([^"']+)["']/
OS_ENVIRON_RE = /\bos\.environ\s*\[\s*[rf]?["']([^"']+)["']\s*\]/
OS_GETENV_RE = /\bos\.getenv\s*\(\s*[rf]?["']([^"']+)["']/
SYS_ARGV_RE = /\bsys\.argv\s*\[\s*(\d+)\s*\]/

stdlib argv / env / getopt

TYPER_ARGUMENT_PARAM_RE = /(\w+)\s*:[^,=]+=\s*typer\.Argument/
TYPER_NEW_RE = /(\w+)\s*=\s*typer\.Typer\s*\(/

typer

TYPER_OPTION_PARAM_RE = /(\w+)\s*:[^,=]+=\s*typer\.Option/
WEB_FRAMEWORK_RE = /(?:^|\n)\s*(?:import|from)\s+(?:flask|django|fastapi|starlette|sanic|aiohttp|bottle|tornado|falcon|pyramid|quart|litestar|robyn)\b/

Web frameworks: their os.environ/os.getenv reads are config, not a CLI surface, so raw env is suppressed when one is present (framework-bound env via click/typer is still emitted).

Class Method Summary

Instance Method Summary

Instance methods inherited from class Analyzer::Python::PythonEngine

build_callees_from(body : String, body_start_line : Int32, path : String, *, definition_base_path : String | Nil = nil, source : String | Nil = nil) : Array(Callee) build_callees_from, find_def_line(lines : Array(String), decorator_line : Int32) : Int32 | Nil find_def_line, find_imported_modules(app_base_path : String, file_path : String, content : String | Nil = nil) : Hash(String, Tuple(String, Int32)) find_imported_modules, find_imported_package(package_path : String, dotted_as_names : String) : Array(Tuple(String, String, Int32)) find_imported_package, find_json_params(codeblock_lines : Array(String), json_var_names : Array(String)) : Array(Param) find_json_params, join_until_python_call_closes(lines : Array(String), index : Int32, line : String) : String join_until_python_call_closes, parse_code_block(data : String | Array(String), after : Regex | Nil = nil) : String | Nil parse_code_block, parse_function_def(source_lines : Array(String), start_index : Int32) : FunctionDefinition | Nil parse_function_def, push_callees_from(endpoint : Endpoint, body : String, body_start_line : Int32, path : String, *, definition_base_path : String | Nil = nil, source : String | Nil = nil) : Nil push_callees_from, python_bracket_delta(line : String) : Int32 python_bracket_delta, python_paren_delta(line : String) : Int32 python_paren_delta, python_signature_line_span(lines : Array(String)) : Int32 python_signature_line_span, return_literal_value(data : String) : String return_literal_value

Class methods inherited from class Analyzer::Python::PythonEngine

python_test_path?(path : String, base_path : String | Nil = nil) : Bool python_test_path?

Instance methods inherited from class Analyzer

analyze analyze, base_path : String base_path, base_paths : Array(String) base_paths, base_relative_path(path : String) : String base_relative_path, callees_needed? : Bool callees_needed?, content_matches?(content : String, markers : Regex) : Bool content_matches?, http_header_name(name : String) : String | Nil http_header_name, line_number_for_index(content : String, char_index : Int32) : Int32 line_number_for_index, logger : NoirLogger logger, parallel_analyze(files : Array(String), &block : String -> Nil) parallel_analyze, read_file_content(path : String) : String read_file_content, result : Array(Endpoint) result, tech : String tech, unique_params(params : Array(Param)) : Array(Param) unique_params, url : String url, web_root_path(path : String, markers : Array(String)) : String web_root_path

Constructor methods inherited from class Analyzer

new(options : Hash(String, YAML::Any)) new

Macros inherited from class Analyzer

analyzer_for(tech) analyzer_for

Instance methods inherited from module FileHelper

all_files : Array(String) all_files, get_files_by_basename(basename : String) : Array(String) get_files_by_basename, get_files_by_extension(extension : String) : Array(String) get_files_by_extension, get_files_by_extensions(extensions : Array(String)) : Array(String) get_files_by_extensions, get_files_by_prefix(prefix : String) : Array(String) get_files_by_prefix, get_files_by_prefix_and_extension(prefix : String, extension : String) : Array(String) get_files_by_prefix_and_extension, get_files_by_relative_path(relative_path : String, root : String = "") : Array(String) get_files_by_relative_path, get_public_dir_files(base_path : String, folder : String) : Array(String) get_public_dir_files, get_public_files(base_path : String, anchors : Array(String) = ["shard.yml", "Gemfile"]) : Array(String) get_public_files, walked_path(expanded : String) : String walked_path

Class Method Detail

def self.tech_name : String #

[View source]

Instance Method Detail

def analyze #

[View source]
def tech : String #

Instance-side view of the same declaration. The per-file rescues live on this base class, which has no way to name the analyzer that is running inside them, so a skipped file could not be attributed to a tech. Deriving it from analyzer_for keeps the name written exactly once.


[View source]