class Noir::JSRouteExtractor

• Noir::JSRouteExtractor
• Reference
• Object

Overview

JSRouteExtractor provides a unified interface for extracting routes from JavaScript files

Defined in:

miniparsers/js_route_extractor.cr

Constant Summary

BRACKET_ROUTE_CALL_PATTERN = /\[\s*['"](?:get|post|put|delete|del|patch|options|head|all)['"]\s*\]\s*\(/i

Pre-filter for .extract_routes: returns false when content contains no shape the JS parser knows how to emit (any verb invocation pattern like .get(/.post(/... or Fastify/Restify .route(, plus Express-style mounts .use( which feed into the cross-file router prefix table). Matching is millions of times cheaper than tokenizing the file.

CLIENT_SIDE_FRAMEWORK_MARKER = Regex.union(CLIENT_SIDE_FRAMEWORK_MARKERS)

CLIENT_SIDE_FRAMEWORK_MARKERS = ["from \"vue\"", "from 'vue'", "from \"@vue/", "from '@vue/", "from \"vue-router\"", "from 'vue-router'", "from \"@vueuse/", "from '@vueuse/", "from \"pinia\"", "from 'pinia'", "from \"react\"", "from 'react'", "from \"react-dom", "from 'react-dom", "from \"react-router", "from 'react-router", "from \"@angular/", "from '@angular/", "from \"svelte\"", "from 'svelte'", "from \"svelte/", "from 'svelte/", "from \"solid-js", "from 'solid-js", "from \"preact\"", "from 'preact'", "from \"preact/", "from 'preact/", ".vue\"", ".vue'", ".svelte\"", ".svelte'"]

Client-side UI framework imports. A file that imports a browser UI framework (Vue, React, Angular, Svelte, Solid, Preact) and its satellite libs (pinia, vue-router, @vueuse, react-router, ...) is SPA/frontend code, not an HTTP server. Its route-shaped calls are outbound API-client requests against a configured client — e.g. directus's admin app does api.get(/users/${userId}) where api is a wrapped axios instance imported from @/api. The existing axios/got/ky markers miss these because the wrapper hides the raw client behind a local module, but the UI-framework import is an unambiguous "this is browser code" signal. directus's admin SPA alone parks ~61 phantom Express endpoints across app/src/{stores,composables,layouts,...} this way. Like the test-stub markers, this is gated by the HTTP-server-import exemption below: an SSR entrypoint that imports BOTH vue and express keeps its routes.

FASTIFY_RECEIVER_MARKER = /\bfastify\s*\.\s*(?:get|post|put|patch|delete|head|options|all|route|register)\s*\(/

A @fastify/autoload plugin module names no framework at all — it receives the instance as a parameter:

export default async function (fastify) {
  fastify.get('/status', handler)
}

so no import marker fires and the shared extractor happily reads those registrations for whichever framework asked. Express reported /status and /go out of the fastify autoload fixture, without the autoPrefix the Fastify analyzer applies. The receiver name is the only evidence in the file, and fastify is unambiguous — nothing else calls its app instance that. Consulted only when the file imports no HTTP server.

FLEXIBLE_ROUTE_CALL_PATTERN = /\.(?:\s|\n|\r)*(?:get|post|put|delete|del|patch|options|head|all|route|register|use)(?:\s|\n|\r)*\(/i

HTTP_CLIENT_CONSTRUCTOR_MARKER = Regex.union("restify-clients", "createJSONClient(", "createStringClient(", "createHttpClient(")

Constructors that build an HTTP client. client.get('/todo', cb) is the same call shape as a route registration, so a client module reads as a server to the shared extractor: the Express analyzer reported GET /todo and DELETE /todo/example out of a restify-clients module that only calls a remote API.

The restify analyzer already refused these; the check belongs here so every framework sharing the extractor gets it. It is deliberately limited to client constructors rather than client package names — a genuine route file may well require('axios') to call downstream services, and gating on that would drop its routes.

HTTP_SERVER_LIBRARY_MARKER = Regex.union(HTTP_SERVER_LIBRARY_MARKERS)

HTTP_SERVER_LIBRARY_MARKERS = ["from \"express\"", "from 'express'", "require(\"express\")", "require('express')", "from \"fastify\"", "from 'fastify'", "require(\"fastify\")", "require('fastify')", "from \"koa\"", "from 'koa'", "require(\"koa\")", "require('koa')", "from \"hono\"", "from 'hono'", "require(\"hono\")", "require('hono')", "from \"restify\"", "from 'restify'", "require(\"restify\")", "require('restify')", "from \"polka\"", "from 'polka'", "from \"h3\"", "from 'h3'", "from \"@nestjs/", "from '@nestjs/"]

Real HTTP-server library imports. When any of these is present alongside a test-stub marker, the file is doing legitimate server work (e.g., spinning up a test instance of an Express app) and we still want to extract its routes.

MINIFIED_AVG_LINE_THRESHOLD = 1000

Average bytes-per-line above which a file is considered dominated by long lines, i.e. a bundle rather than hand-written source that merely carries one fat literal (a big inline JSON seed, an embedded base64 data URI, a long regex). Real code keeps the average low because it has many short lines around any such literal.

MINIFIED_LINE_THRESHOLD = 5000

Byte length above which a single source line is considered "long". Hand-written JS/TS keeps lines well under this even in dense route tables (noir's own widest fixture line is ~150 bytes); webpack/ rollup/esbuild bundles and *.min.js assets routinely pack tens of thousands of bytes onto one line, so 5000 leaves a wide margin. NB: the metric is bytes, not characters — a dense single-line non-Latin blob (>=5000 bytes but fewer chars) can trip it, which is acceptable since real route registrations are ASCII verbs/paths.

OTHER_EXTRACTOR_MARKER = SHARED_EXTRACTOR_FRAMEWORK_MARKERS.keys.to_h do |framework| others = SHARED_EXTRACTOR_FRAMEWORK_MARKERS.reject do |other, _| other == framework end.values.flatten {framework, Regex.union(others)} end

OTHER_FRAMEWORK_MARKERS = {:nestjs => ["from \"@nestjs/", "from '@nestjs/", "require(\"@nestjs/", "require('@nestjs/"], :hapi => ["from \"@hapi/hapi\"", "from '@hapi/hapi'", "require(\"@hapi/hapi\")", "require('@hapi/hapi')", "from \"hapi\"", "from 'hapi'", "require(\"hapi\")", "require('hapi')"], :elysia => ["from \"elysia\"", "from 'elysia'", "require(\"elysia\")", "require('elysia')"], :adonisjs => ["from \"@adonisjs/", "from '@adonisjs/", "require(\"@adonisjs/", "require('@adonisjs/", "from \"@ioc:Adonis/", "from '@ioc:Adonis/"]}

Sibling JS/TS server frameworks that DON'T call .extract_routes (NestJS uses decorators, Hapi/Elysia/AdonisJS have their own tree-sitter extractors) but whose files are still walked by every JS/TS analyzer's parallel_file_scan. These markers are exclusion- only: recognizing "this file belongs to NestJS" keeps Express/ Fastify/Koa/Hono/Restify from re-extracting a route-shaped call (an inline example, a raw app.use() bridge, ...) out of it.

OWN_EXTRACTOR_MARKER = SHARED_EXTRACTOR_FRAMEWORK_MARKERS.transform_values do |markers| Regex.union(markers) end

True when content carries definitive import evidence of a different shared-extractor (or sibling) framework than framework, with no evidence of framework itself. Guards the main .extract_routes call in Express/Fastify/Koa/Hono/Restify's analyzers: a file that only imports 'hono' should never be re-attributed to js_express just because its .get()/.post() chaining looks the same as Express's (issue #2368) — whichever analyzer runs over it first no longer matters once #2367 made the dedup tiebreak deterministic, because the over-matching analyzer never produces a competing endpoint in the first place.

Files with no shared-extractor/sibling import at all (a router module that only receives app/router as a bare parameter, with no import in that file) return false here and fall through to the existing, permissive whole-tree scan — there's no import to disambiguate on, so narrowing further is left as follow-up scope (a confirmed package.json dependency or cross-file mount signal would be needed to resolve those). Per-framework precompiled unions of the two tables above: the framework's own import markers, and the markers of every other shared-extractor framework. Together with the sibling union this turns the up-to-44-literal includes? walk below into three matches. Five analyzers call this on every JS/TS file, so the old shape scanned some trees over 200 times per file.

ROUTER_PREFIX_KEY = Analyzer::Javascript::ExpressConstants::ROUTER_PREFIX_KEY

Import constants for key generation

SHARED_EXTRACTOR_FRAMEWORK_MARKERS = {:express => ["from \"express\"", "from 'express'", "require(\"express\")", "require('express')"], :fastify => ["from \"fastify\"", "from 'fastify'", "require(\"fastify\")", "require('fastify')", "from \"fastify-plugin\"", "from 'fastify-plugin'", "require(\"fastify-plugin\")", "require('fastify-plugin')", "from \"@fastify/", "from '@fastify/", "require(\"@fastify/", "require('@fastify/"], :koa => ["from \"koa\"", "from 'koa'", "require(\"koa\")", "require('koa')", "from \"koa-router\"", "from 'koa-router'", "require(\"koa-router\")", "require('koa-router')", "from \"@koa/router\"", "from '@koa/router'", "require(\"@koa/router\")", "require('@koa/router')"], :hono => ["from \"hono\"", "from 'hono'", "require(\"hono\")", "require('hono')", "from \"hono/", "from 'hono/"], :restify => ["from \"restify\"", "from 'restify'", "require(\"restify\")", "require('restify')", "from \"restify-router\"", "from 'restify-router'", "require(\"restify-router\")", "require('restify-router')"]}

Import markers for the five frameworks whose analyzers call .extract_routes directly and therefore share its framework-agnostic verb-chaining shape (.get(/.post(/...). Keyed by the same Symbol each analyzer passes to .other_shared_extractor_framework? below.

SIBLING_FRAMEWORK_MARKER = Regex.union(OTHER_FRAMEWORK_MARKERS.values.flatten)

STATIC_MOUNT_MARKER = Regex.union(".use(", ".use (", ".register(", ".register (", "ServeStaticModule.forRoot", "serveStatic")

Extract static path declarations from JavaScript content Returns array of hashes with static_path (URL prefix) and file_path (directory) framework scopes the scan to one framework's static-mount idiom so a framework analyzer running over a sibling project's file (every JS analyzer walks all .js/.ts files) doesn't pick up another framework's static declaration and re-emit it under the wrong tech. nil runs every pattern (back-compat for un-scoped callers).

STRICT_TEST_PATH_MARKER = Regex.union(STRICT_TEST_PATH_MARKERS)

STRICT_TEST_PATH_MARKERS = ["/e2e/", "/cypress/", "/playwright/", "/__mocks__/", "/__tests__/", "/e2e-tests/", "/mirage/"]

True when the file's route-shaped calls are almost certainly mock-server stubs (Ember pretender, MSW, nock, ...) rather than real route registrations. Two routes:

Path markers strict enough that the HTTP-server-import exemption shouldn't apply: /e2e/, /cypress/, /playwright/, /__mocks__/, /__tests__/, /e2e-tests/, /mirage/. Real apps never park production handlers under any of these — even when the harness file imports express to spin up a faked service (Ghost's e2e/helpers/services/stripe/fake-stripe-server.ts is the canonical example). Keeping the exemption out of these paths catches the harness fakes without affecting legit backend code.

TEST_STUB_FILENAME_MARKER = Regex.union(TEST_STUB_FILENAME_MARKERS)

• Filename markers fire unconditionally — foo.test.ts is a test no matter what it imports.
  • Strict path markers also fire unconditionally — e2e/, cypress/, etc. are dedicated test/mock trees that never contain production handlers, even when the harness file imports a server lib.
  • Library + the remaining directory markers honor an exemption — if the file also imports a real HTTP server lib (express, fastify, ...), keep it so legit test-server harnesses (e.g. mattermost's webhook_serve.js) keep their routes. include_client_frameworks controls whether a client-side UI framework import (Vue/React/...) counts as a skip signal. It must be ON for the verb-DSL extractor (a React/Vue file calling api.get(...) is an outbound client call, not a route), but OFF for analyzers whose OWN route definitions live in client-side files — TanStack Router (createFileRoute) and tRPC route modules routinely import { ... } from 'react', and skipping them on that basis dropped every such route. The test-stub library markers (msw/supertest/...) and path/ filename markers still apply in both modes. Precompiled unions of the marker lists above. Regex.union escapes every String argument, so each is exactly the any? includes? it replaces — but the content lists are long (72 test-stub libraries, 32 client frameworks, 26 server libraries), and every JS/TS file in the tree used to be walked once per literal.

TEST_STUB_FILENAME_MARKERS = [".test.", ".spec.", "-spec.", "-test.", ".test-d."]

Hard test-file markers: when the filename itself follows a ubiquitous test convention, the file practically never defines real routes. Skip these even when the file imports a real HTTP server lib — NestJS e2e tests routinely import @nestjs/platform-express for type-only references, and supertest harnesses import the same modules they exercise. The supertest request(app).get(...) shape would otherwise ride the HTTP-server-import exemption straight back into the parser.

TEST_STUB_LIBRARY_MARKER = Regex.union(TEST_STUB_LIBRARY_MARKERS)

TEST_STUB_LIBRARY_MARKERS = ["pretender", "miragejs", "ember-cli-mirage", "from \"msw\"", "from 'msw'", "from \"msw/", "from 'msw/", "require(\"msw\")", "require('msw')", "from \"nock\"", "from 'nock'", "require(\"nock\")", "require('nock')", "setupApplicationTest", "setupRenderingTest", "/// <reference types=\"cypress\" />", "from \"cypress\"", "from 'cypress'", "require(\"cypress\")", "require('cypress')", "from \"@playwright/test\"", "from '@playwright/test'", "from \"playwright\"", "from 'playwright'", "from \"supertest\"", "from 'supertest'", "require(\"supertest\")", "require('supertest')", "from \"axios\"", "from 'axios'", "require(\"axios\")", "require('axios')", "from \"purest\"", "from 'purest'", "require(\"purest\")", "require('purest')", "from \"got\"", "from 'got'", "require(\"got\")", "require('got')", "from \"ky\"", "from 'ky'", "require(\"ky\")", "require('ky')", "from \"superagent\"", "from 'superagent'", "require(\"superagent\")", "require('superagent')", "from \"node-fetch\"", "from 'node-fetch'", "require(\"node-fetch\")", "require('node-fetch')", "from \"ofetch\"", "from 'ofetch'", "require(\"ofetch\")", "require('ofetch')", "from \"undici\"", "from 'undici'", "require(\"undici\")", "require('undici')", "from \"request\"", "from 'request'", "require(\"request\")", "require('request')", "from \"apollo-datasource-rest\"", "from 'apollo-datasource-rest'", "require(\"apollo-datasource-rest\")", "require('apollo-datasource-rest')", "from \"@apollo/datasource-rest\"", "from '@apollo/datasource-rest'", "require(\"@apollo/datasource-rest\")", "require('@apollo/datasource-rest')"]

Test-fixture libraries whose API mimics route registration: pretender/miragejs expose server.get("/x", ...), MSW and nock expose handler builders, sinon-via-faker likewise. When these libraries are imported, virtually every route-shaped call in the file is a stub, not a real registration. Substring match is enough — these tokens never appear in production HTTP server source under normal circumstances.

TEST_STUB_PATH_MARKER = Regex.union(TEST_STUB_PATH_MARKERS)

TEST_STUB_PATH_MARKERS = ["-pretender.", "-pretenders.", ".pretender.", "-mirage.", ".mirage.", "/tests/helpers/", "/test/helpers/", "/tests/api/", "/__tests__/", "/test/integration/", "/tests/integration/", "/test/e2e/", "/tests/e2e/", "/cypress/", "/playwright/", "/e2e-tests/", "/e2e/", "/mirage/", "/__mocks__/", "/dist/", "/build/", "/.next/", "/.nuxt/", "/.output/", "/coverage/", "/vendor/", "/app/javascript/", "/public/"]

Path-level evidence that a file is a mock-server fixture. Pretender helpers in particular get a helper/this arg and call this.get(...) / this.post(...) directly, so they have no library-name imports the content filter can hook on — fall back to the convention-based filename match.

Class Method Summary

• .attach_callees(endpoint : Endpoint, callees_by_route : Hash(String, Array(JSCalleeExtractor::Entry)), method : String, path : String, line : Int32)
• .extract_body_params(handler_body : String, endpoint : Endpoint)
• .extract_cookie_params(handler_body : String, endpoint : Endpoint)
• .extract_header_params(handler_body : String, endpoint : Endpoint)
• .extract_params_from_context(content : String, pattern : JSRoutePattern, endpoint : Endpoint)
• .extract_path_params(handler_body : String, endpoint : Endpoint)
• .extract_query_params(handler_body : String, endpoint : Endpoint)
• .extract_routes(file_path : String, content : String | Nil = nil, debug : Bool = false, *, include_callees : Bool = false, route_callees : Hash(String, Array(JSCalleeExtractor::Entry)) | Nil = nil) : Array(Endpoint)
• .extract_static_paths(content : String, framework : Symbol | Nil = nil) : Array(Hash(String, String))
• .find_matching_brace(content : String, open_brace_idx : Int32) : Int32 | Nil

  Delegate to JSLiteralScanner for literal-aware brace matching

• .find_matching_paren(content : String, open_paren_idx : Int32) : Int32 | Nil

  Delegate to JSLiteralScanner for literal-aware paren matching

• .minified_content?(content : String, line_threshold : Int32 = MINIFIED_LINE_THRESHOLD, avg_threshold : Int32 = MINIFIED_AVG_LINE_THRESHOLD) : Bool

  True when content looks like a minified/bundled asset rather than hand-written source.

• .normalize_http_method(method : String) : String

  Normalize HTTP method names to standard format

• .other_shared_extractor_framework?(content : String, framework : Symbol) : Bool
• .route_call_candidate?(content : String) : Bool

  A 22-literal includes? pre-pass used to run ahead of these two patterns (".get(", ".get (", ...

• .strip_js_comments(content : String) : String

  Replace JS/TS comments with whitespace of the same shape.

• .test_stub_only?(file_path : String, content : String, include_client_frameworks : Bool = true) : Bool

  Every marker below names a directory inside the project (__tests__/, dist/, vendor/, public/, ...), so they are matched on the scan-base-relative path.

Class Method Detail