class Analyzer::Go::Cli

Overview

Surfaces the command-line attack surface of Go programs as cli:// endpoints: one endpoint per (sub)command, with named flags/options (param_type "flag"), positional arguments ("argument"), and consumed environment variables ("env"). Covers the stdlib flag package + os.Args + os.Getenv/os.LookupEnv, plus the cobra, urfave/cli, pflag, go-arg, go-flags, kong, kingpin and mitchellh/cli ecosystems.

This is a line-scan analyzer (the house style for non-tree-sitter Go adapters, e.g. fasthttp). Endpoints are merged by URL across files so a root command whose flags are registered in a separate init() block collects them all under a single cli://<binary> entry.

Included Modules

Defined in:

analyzer/analyzers/go/cli.cr

Constant Summary

BUILTIN_ARG_RE = /\bflag\.Arg\s*\(\s*(\d+)\s*\)/
BUILTIN_ARGS_RE = /\bflag\.Args\s*\(\s*\)/
BUILTIN_FLAG_RE = /\bflag\.(?:String|Int|Int64|Uint|Uint64|Bool|Float64|Duration)(?:Var)?\s*\(\s*(?:&?[\w.]+\s*,\s*)?"([^"]+)"/

--- builtin flag / argv ------------------------------------------------- The flag name is always the FIRST quoted string in the call: the *Var forms put the destination pointer (unquoted) first, then the name.

BUILTIN_FLAG_VAR_RE = /\bflag\.Var\s*\(\s*&?[\w.]+\s*,\s*"([^"]+)"/
CLI_PARSE_POINT_RE = Regex.union("github.com/spf13/cobra", "github.com/urfave/cli", "github.com/alexflint/go-arg", "github.com/jessevdk/go-flags", "flag.Parse(")

Five markers OR-ed as a standalone boolean gate for cli_parse_point? below — one precompiled Regex.union (PCRE2 JIT) replaces the separate content.includes? scan each used to make over the same buffer. kong/kingpin/mitchellh are deliberately excluded (see the comment in cli_parse_point?).

COBRA_CMD_DECL_RE = /(\w+)\s*[:=]\s*&?cobra\.Command\s*\{/

--- cobra ---------------------------------------------------------------

COBRA_FLAG_RE = /(\w+)\s*\.\s*(?:Persistent)?Flags\s*\(\s*\)\s*\.\s*[A-Za-z0-9]+\s*\(\s*(?:&?[\w.]+\s*,\s*)?"([^"]+)"/
COBRA_USE_RE = /\bUse\s*:\s*"([^"\s]+)/
FLAG_IMPORT_RE = /"flag"/
FLAG_PARSE_RE = /flag\.Parse\(/
FRAMEWORK_IMPORTS = ["github.com/spf13/cobra", "github.com/urfave/cli", "github.com/alexflint/go-arg", "github.com/jessevdk/go-flags", "github.com/spf13/pflag", "github.com/alecthomas/kong", "github.com/mitchellh/cli", "github.com/alecthomas/kingpin", "gopkg.in/alecthomas/kingpin.v2"]

Any of these in a file means it participates in the CLI surface.

FRAMEWORK_IMPORTS_RE = Regex.union(FRAMEWORK_IMPORTS)

FRAMEWORK_IMPORTS.any? { |m| content.includes?(m) } re-scans the whole file once per marker (used at two call sites below); one precompiled Regex.union (PCRE2 JIT) checks all nine in a single pass.

GOARG_TAG_RE = /`[^`]*\barg:"([^"]*)"[^`]*`/

--- go-arg / go-flags struct tags ---------------------------------------

GOFLAGS_ENV_RE = /`[^`]*\benv:"([^"]+)"/
GOFLAGS_LONG_RE = /`[^`]*\blong:"([^"]+)"/
HTTP_LISTEN_RE = /\b(?:http|fasthttp)\.ListenAndServe(?:TLS)?\s*\(|\.(?:ListenAndServe|RunTLS)\s*\(/

An HTTP listen call in the same file means env reads are most likely server config, not a CLI surface — suppress raw env there.

KINGPIN_ARG_RE = /(\w+)\s*\.\s*Arg\s*\(\s*"([^"]+)"\s*,/
KINGPIN_COMMAND_RE = /(\w+)\s*:?=\s*(\w+)\s*\.\s*Command\s*\(\s*"([^"]+)"/
KINGPIN_ENVAR_RE = /\.\s*Envar\s*\(\s*"([^"]+)"\s*\)/
KINGPIN_FLAG_RE = /(\w+)\s*\.\s*Flag\s*\(\s*"([^"]+)"\s*,/
KINGPIN_IMPORT_RE = Regex.union("github.com/alecthomas/kingpin", "gopkg.in/alecthomas/kingpin.v2")
KINGPIN_NEW_RE = /(\w+)\s*:?=\s*kingpin\.New\s*\(/

--- kingpin (fluent Flag/Arg/Command builder) --------------------------- app := kingpin.New(...) seeds the root receiver; cmd := app.Command(...) (or sub := cmd.Command(...) for nesting) maps a new receiver var onto a command URL, and .Flag(...)/.Arg(...) calls on a known receiver attribute a param to that command — never to a sticky "current" command.

KONG_ARG_TAG_RE = /\barg\s*:\s*"[^"]*"/
KONG_CMD_TAG_RE = /\bcmd\s*:\s*"[^"]*"/
KONG_ENV_TAG_RE = /\benv\s*:\s*"([^"]+)"/
KONG_FIELD_RE = /^\s*([A-Z]\w*)\s+([\w.\[\]*]+)\s+`([^`]*)`/
KONG_IMPORT_RE = Regex.union("github.com/alecthomas/kong")

Per-framework matchers for the scan dispatch below, which asks the same questions again once cli_evidence? has let a file through.

KONG_NAME_TAG_RE = /\bname\s*:\s*"([^"]+)"/
KONG_TAG_KEY_RE = /\b(?:cmd|arg|env|default|help)\s*:\s*"/
KONG_TYPE_DECL_RE = /^type\s+(\w+)\s+struct\s*\{/

--- kong (struct-tag CLI) ------------------------------------------------ Root/subcommand fields are declared as struct fields; a cmd:"" tag marks a field as a subcommand whose own flags/args live in that field's named struct type, an arg:"" tag marks a positional, and an env:"" tag additionally binds the field to an environment variable.

MITCHELLH_FLAGVAR_RE = /\b\w+\.(?:StringVar|IntVar|Int64Var|BoolVar|Float64Var|DurationVar)\s*\(\s*&?[\w.]+\s*,\s*"([^"]+)"/
MITCHELLH_IMPORT_RE = Regex.union("github.com/mitchellh/cli")
MITCHELLH_KEY_RE = /^\s*"([^"]+)"\s*:\s*func\s*\(\s*\)\s*\(cli\.Command,\s*error\)\s*\{/

--- mitchellh/cli (Commands map of factories) ---------------------------- c.Commands = map[string]cli.CommandFactory{"name": func() (cli.Command, error) { return &FooCommand{}, nil }} maps each map key to the concrete command type it instantiates; that type's own Run method is then scanned (scoped to its own body) for flag.FlagSet-style *Var registrations and raw env reads.

MITCHELLH_RETURN_RE = /return\s+&(\w+)\{/
MITCHELLH_RETURN_VAR_RE = /return\s+(\w+)\s*,/
MITCHELLH_RUN_FUNC_RE = /^func\s*\(\s*\w+\s+\*?(\w+)\)\s*Run\s*\(/
MITCHELLH_VAR_ASSIGN_RE = /(\w+)\s*:=\s*&(\w+)\{/

cmd := &DeployCommand{} followed later (within the SAME closure) by return cmd, nil — the idiomatic form used whenever the command needs field initialization, as common as the single-expression return &X{}.

OS_ARG_INDEX_RE = /\bos\.Args\s*\[\s*(\d+)\s*\]/
OS_ARGS_RE = /os\.Args/
OS_GETENV_RE = /\bos\.(?:Getenv|LookupEnv)\s*\(\s*"([^"]+)"\s*\)/

--- env -----------------------------------------------------------------

PACKAGE_MAIN_RE = /^\s*package\s+main\b/m

A Go module can hold many commands — cmd/foo, services/bar/cmd — and each package main directory is a separate binary. Naming every one of them after the module merged unrelated programs' flags into a single cli://<module> endpoint. Library packages keep the module name, so the cross-file merge this analyzer is built around still works for the helper files a command pulls its flags from.

PFLAG_RE = /\bpflag\.(?:String|Int|Int64|Uint|Bool|Float64|Duration|StringSlice|StringArray)(?:Var)?(?:P)?\s*\(\s*(?:&?[\w.]+\s*,\s*)?"([^"]+)"/

--- pflag (when used without cobra) -------------------------------------

RAW_CLI_EVIDENCE_RE = Regex.union(FRAMEWORK_IMPORTS + ["\"flag\"", "os.Args"])

Cheap pre-gate applied to the RAW file, before the comment strip.

GoEngine.strip_comments materialises an Array(Char) of the whole file and rebuilds it character by character; running it on every .go file in a monorepo — 15k of them in kubernetes, of which a couple of hundred have any CLI surface — was the whole cost of this analyzer. Stripping only ever replaces characters with ' ' or '\n' and never changes the character count, so a literal with no whitespace in it can appear in the stripped text only if it already appears, unchanged, in the raw text. Every branch of cli_evidence? requires one of these literals verbatim (FLAG_IMPORT_RE is the literal "flag"; OS_ARG_INDEX_RE needs an adjacent os.Args), so a file that misses all of them cannot pass cli_evidence? after the strip either. Files that do match still go through the full, comment-aware gate below.

STRUCT_TAG_IMPORT_RE = Regex.union("alexflint/go-arg", "jessevdk/go-flags")
TOPLEVEL_BRACE_CLOSE_RE = /^\}\s*$/

A bare unindented } closes the top-level struct/func block it belongs to (the gofmt convention this line-scan relies on for kong's type X struct {} and mitchellh/cli's func (...) Run(...) {} scoping below).

URFAVE_APP_RE = /(?:&)?cli\.App\s*\{/

--- urfave/cli ----------------------------------------------------------

URFAVE_CMD_RE = /(?:&)?cli\.Command\s*\{/
URFAVE_ENVVAR_RE = /\bEnvVar\s*:\s*"([^"]+)"/
URFAVE_ENVVARS_RE = /\bEnvVars\s*:\s*\[\]string\s*\{([^}]*)\}/
URFAVE_FLAG_RE = /(?:&)?cli\.[A-Za-z0-9]*Flag\s*\{/
URFAVE_NAME_RE = /\bName\s*:\s*"([^"]+)"/
VIPER_BINDENV_RE = /\bviper\.BindEnv\s*\(\s*"([^"]+)"(?:\s*,\s*"([^"]+)")?/

Class Method Summary

Instance Method Summary

Instance methods inherited from class Analyzer::Go::GoEngine

add_param_to_endpoint(param : Param, endpoint : Endpoint) add_param_to_endpoint, add_static_path_if_valid(static_path : Hash(String, String), public_dirs : Array(Hash(String, String))) add_static_path_if_valid, collect_import_path_function_bodies(package_function_bodies : Hash(String, Hash(String, Noir::GoCalleeExtractor::FunctionBody))) : Hash(String, Hash(String, Noir::GoCalleeExtractor::FunctionBody)) collect_import_path_function_bodies, collect_import_path_method_bodies(package_method_bodies : Hash(String, Hash(String, Array(Noir::GoCalleeExtractor::FunctionBody)))) : Hash(String, Hash(String, Array(Noir::GoCalleeExtractor::FunctionBody))) collect_import_path_method_bodies, collect_package_controller_method_bodies(file_contents : Hash(String, String)) : Hash(String, Hash(String, Array(Noir::GoCalleeExtractor::FunctionBody))) collect_package_controller_method_bodies, collect_package_controller_methods(file_contents : Hash(String, String)) : Hash(String, Hash(String, Array(String))) collect_package_controller_methods, collect_package_function_bodies(file_contents : Hash(String, String)) : Hash(String, Hash(String, Noir::GoCalleeExtractor::FunctionBody)) collect_package_function_bodies, collect_package_groups_ts(group_method : String = "Group", import_marker : String | Nil = nil) : Tuple(Hash(String, Hash(String, String)), Hash(String, String)) collect_package_groups_ts, framework_package_dirs(file_contents : Hash(String, String), import_marker : String) : Set(String) framework_package_dirs, framework_route_source_candidate?(content : String, dir : String, framework_dirs : Set(String), import_marker : String, extra_methods : Array(String)) : Bool framework_route_source_candidate?, go_route_source_candidate?(content : String, extra_methods : Array(String)) : Bool go_route_source_candidate?, read_package_file_contents : Hash(String, String) read_package_file_contents, resolve_public_dirs(public_dirs : Array(Hash(String, String))) resolve_public_dirs, resolve_public_dirs_with_glob(public_dirs : Array(Hash(String, String))) resolve_public_dirs_with_glob, static_dir_entry(source_path : String, static_path : String, file_path : String) : Hash(String, String) static_dir_entry, ts_controller_method_bodies_for_directory(package_controller_method_bodies : Hash(String, Hash(String, Array(Noir::GoCalleeExtractor::FunctionBody))), dir : String) : Hash(String, Array(Noir::GoCalleeExtractor::FunctionBody)) ts_controller_method_bodies_for_directory, ts_controller_methods_for_directory(package_controller_methods : Hash(String, Hash(String, Array(String))), dir : String) : Hash(String, Array(String)) ts_controller_methods_for_directory, ts_function_bodies_for_directory(package_function_bodies : Hash(String, Hash(String, Noir::GoCalleeExtractor::FunctionBody)), dir : String) : Hash(String, Noir::GoCalleeExtractor::FunctionBody) ts_function_bodies_for_directory, ts_groups_for_directory(package_groups : Hash(String, Hash(String, String)), dir : String) : Hash(String, String) ts_groups_for_directory

Class methods inherited from class Analyzer::Go::GoEngine

go_test_file?(relative_path : String) : Bool go_test_file?, strip_comments(text : String) : String strip_comments

Instance methods inherited from class Analyzer

analyze analyze, base_path : String base_path, base_paths : Array(String) base_paths, base_relative_path(path : String) : String base_relative_path, callees_needed? : Bool callees_needed?, content_matches?(content : String, markers : Regex) : Bool content_matches?, http_header_name(name : String) : String | Nil http_header_name, line_number_for_index(content : String, char_index : Int32) : Int32 line_number_for_index, logger : NoirLogger logger, parallel_analyze(files : Array(String), &block : String -> Nil) parallel_analyze, read_file_content(path : String) : String read_file_content, result : Array(Endpoint) result, tech : String tech, unique_params(params : Array(Param)) : Array(Param) unique_params, url : String url, web_root_path(path : String, markers : Array(String)) : String web_root_path

Constructor methods inherited from class Analyzer

new(options : Hash(String, YAML::Any)) new

Macros inherited from class Analyzer

analyzer_for(tech) analyzer_for

Instance methods inherited from module FileHelper

all_files : Array(String) all_files, get_files_by_basename(basename : String) : Array(String) get_files_by_basename, get_files_by_extension(extension : String) : Array(String) get_files_by_extension, get_files_by_extensions(extensions : Array(String)) : Array(String) get_files_by_extensions, get_files_by_prefix(prefix : String) : Array(String) get_files_by_prefix, get_files_by_prefix_and_extension(prefix : String, extension : String) : Array(String) get_files_by_prefix_and_extension, get_files_by_relative_path(relative_path : String, root : String = "") : Array(String) get_files_by_relative_path, get_public_dir_files(base_path : String, folder : String) : Array(String) get_public_dir_files, get_public_files(base_path : String, anchors : Array(String) = ["shard.yml", "Gemfile"]) : Array(String) get_public_files, walked_path(expanded : String) : String walked_path

Class Method Detail

def self.tech_name : String #

[View source]

Instance Method Detail

def analyze #

[View source]
def tech : String #

Instance-side view of the same declaration. The per-file rescues live on this base class, which has no way to name the analyzer that is running inside them, so a skipped file could not be attributed to a tech. Deriving it from analyzer_for keeps the name written exactly once.


[View source]