diff --git a/CHANGELOG.md b/CHANGELOG.md index db721f2..3b52731 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,7 +4,9 @@ ### Advisories -- A filter of Django's own libraries that a project template applies is a call to the function behind it, at the template's line: `{{ bio|striptags }}` calls `django.template.defaultfilters.striptags`, and so do filters in tag arguments and `{% filter %}` blocks. An advisory entry point naming that function is reachable there, whether or not a Python call names the template, since a class-based view renders its `template_name` inside Django. The data a template passes to a filter is not tracked yet, so such a call is reachable, not exploitable. A template the analyzer cannot read may apply any filter: when the project names one by an expression, or by a name found under no `templates` directory, in `render`, `render_to_string`, `TemplateResponse`, `{% include %}` or `{% extends %}`, an advisory whose entry points include a template filter stays `under_investigation` in the VEX document, and the notes say where. `render`, `TemplateResponse` and `SimpleTemplateResponse` are now template renders like `render_to_string` (#146). +- A filter of Django's own libraries that a project template applies is a call to the function behind it, at the template's line: `{{ bio|striptags }}` calls `django.template.defaultfilters.striptags`, and so do filters in tag arguments and `{% filter %}` blocks. An advisory entry point naming that function is reachable there, whether or not a Python call names the template, since a class-based view renders its `template_name` inside Django. A template the analyzer cannot read may apply any filter: when the project names one by an expression, or by a name found under no `templates` directory, in `render`, `render_to_string`, `TemplateResponse`, `{% include %}` or `{% extends %}`, an advisory whose entry points include a template filter stays `under_investigation` in the VEX document, and the notes say where. `render`, `TemplateResponse` and `SimpleTemplateResponse` are now template renders like `render_to_string` (#146). +- Attacker input in a render context reaching a template filter makes an advisory entry point naming that filter exploitable at the render call. The analyzer links them only where it is certain of the value: a dict literal with constant keys that the render call alone uses, and a filter the rendered template applies to a context variable, directly or through `{% for %}`, `{% with %}`, constant `{% include %}` and `{% extends %}`, earlier filter arguments and `{% filter %}` blocks, in the blocks that render. A name a tag binds shadows the context, and `{% include ... only %}` passes none of it. The filter's value is its first argument, the filter's own argument its second. A project function rendering what it receives carries it the same way (#146). +- The request the `request` context processor adds reaches a template's filters too, as `{{ request.GET.q|striptags }}`, only where the analyzer is certain of it: every Django template engine the settings assign to `TEMPLATES`, as literals no other code names, lists the processor; the render call passes the request (`render`, `TemplateResponse`, `render_to_string` given `request`) with a certain context, absent or a dict literal; nothing shadows `request`; and the template reads a text attribute of the request object, such as `GET`, `POST`, `COOKIES` or `headers`. Otherwise the filter stays reachable (#146). ## 0.15.0 (2026-09-26) diff --git a/docs/plugins.md b/docs/plugins.md index 4167750..da938cc 100644 --- a/docs/plugins.md +++ b/docs/plugins.md @@ -157,7 +157,7 @@ class BottleModels(ModelPlugin): | `RouteRegistrar(symbol, argument, label, kinds=ALL, keyword=None)` | A call registering a view at argument `argument` (or `keyword`), such as Django's `path`; the view receives `label` input. | | `SuffixSink(suffix, kinds, positions=())` | A sink matched by the end of the symbol, for methods of any class such as `objects.raw`. | | `SafeArgument(symbol, argument, values, position=None, kinds=ALL)` | A call to the sink `symbol` whose `argument` (by keyword, or at `position`) denotes one of `values` is not a sink for `kinds`, such as `yaml.load` with `Loader=yaml.SafeLoader`. Values are written as the analyzer records arguments: a symbol by its canonical name, a constant as Python writes it (`True`). Only a value given explicitly makes the call safe. | -| `TemplateRender(symbol, position=0, keyword="template_name")` | A call rendering, with autoescaping, the template named by its argument at `position` or `keyword`, such as `render_to_string`. What it returns carries no `HTML` when the name is written in the call and the project's template escapes everything it prints (see the usage guide). A name the analyzer cannot read keeps an advisory a template filter reaches under investigation in the VEX document. | +| `TemplateRender(symbol, position=0, keyword="template_name", context=1, request=None)` | A call rendering, with autoescaping, the template named by its argument at `position` or `keyword`, such as `render_to_string`, with the variables of its argument at `context` or `context=` and the request at `request` or `request=` (None: the call takes none); it makes the filter calls the template applies, fed from that context and, with the `request` context processor, that request (see the usage guide). What it returns carries no `HTML` when the name is written in the call and the project's template escapes everything it prints (see the usage guide). A name the analyzer cannot read keeps an advisory a template filter reaches under investigation in the VEX document. | | `Validator(symbol, kinds=ALL, argument=0)` | A callable whose truth proves its argument safe for `kinds`, such as `re.fullmatch`. A flow guarded by it is refuted when it carries no other kind into the sink, and is a hotspot otherwise: a redirect validator does not make a shell command safe. `symbol` may be a function of the project, by its project symbol (`python.hc.accounts.views._allow_redirect`), which then counts wherever it is called, in its own module too. | | `RequestObject(symbol, text=())` | The inputs from `symbol`, an entry point, a route registrar, a typed parameter or a source, are request objects. A view receives the request first (after `self`), then URL parameters, which are text. Reading one of the request's `text` attributes gives text, as Django's `GET` or aiohttp's `query`; its body and files keep the request's kinds. An attribute read counts only when every source of the value is such a request. | | `AuthorizationGuard(symbol, label)` | A decorator or a condition restricting who reaches the code, such as `login_required`; a flow behind it is a hotspot. | diff --git a/docs/usage.md b/docs/usage.md index f752916..10e61bb 100644 --- a/docs/usage.md +++ b/docs/usage.md @@ -370,9 +370,48 @@ Python call names it, since a class-based view renders its `template_name` insid Django. Most filters are functions of the same name in `django.template.defaultfilters`; `escape`, `escapejs`, `linebreaks`, `phone2numeric`, `slice`, `timesince` and `timeuntil` are `_filter` there, and the filters of `i18n`, `l10n`, `tz` and `humanize` are in -their library's module (`timezone` is `django.templatetags.tz.do_timezone`). The data a -template passes to a filter is not tracked yet, so such a call is reachable, not -exploitable. +their library's module (`timezone` is `django.templatetags.tz.do_timezone`). + +Attacker input in the render context makes the filter exploitable at the render call: +`render(request, "app/profile.html", {"bio": request.POST["bio"]})` with +`{{ bio|striptags }}`. The analyzer links a context value to a filter only where it is +certain of it: + +- The context is a dict literal with constant keys, used by the render call and + nothing else, written in the call or through a variable. A dict mutated or passed + elsewhere, one unpacking another (`**extra`), or a context built any other way leaves + the filter reachable. +- The filter's value is a context variable, directly or through `{% for %}` and + `{% with %}`, the templates the template includes or extends by a constant name, the + arguments of the filters before it, or the output of a `{% filter %}` block. A name a + tag binds shadows the context, and `{% include ... only %}` passes none of it. +- The filter renders: a template extending another renders only its blocks, which + replace the blocks of that name in what it extends. +- One file has the template's name; of two files of the same name, which one renders + depends on the loaders. + +The filter's value is its first argument and its argument (`cut:bio`) its second, so an +entry point naming only the value in `attacker_arguments` is not exploitable through +the argument. A project function rendering the values it receives carries them the same +way. + +With the `django.template.context_processors.request` context processor, a template +rendered with the request reads it as `request`: `{{ request.GET.q|striptags }}` is +exploitable at the render call too. The analyzer counts it only where all of these hold: + +- The project's settings assign `TEMPLATES` a literal list, every `DjangoTemplates` + engine of it lists the processor in a literal `context_processors`, and no other code + of the project names `TEMPLATES`. A mere mention of the processor proves nothing. +- The render call passes the request: `render` and `TemplateResponse` always do, + `render_to_string` when given `request`, `SimpleTemplateResponse` never. +- The render's context is certain, absent or a dict literal as above, so it cannot hide + a `request` entry. An entry of that name, a name a tag binds or `{% include ... only %}` + shadows the processor's request. +- The template reads text the user controls: an attribute the request object lists + among its text attributes, such as `GET`, `POST`, `COOKIES` or `headers`; not `user` or + `session`. + +Through a project function, only the context entries are followed. Most vulnerabilities need attacker input in one argument: the path `send_from_directory` serves, not its `download_name`; the URL `requests.get` fetches, not its body. An entry diff --git a/src/coretrace_python/bundled/models/django/django_models.py b/src/coretrace_python/bundled/models/django/django_models.py index 591dc6d..f2b6ad4 100644 --- a/src/coretrace_python/bundled/models/django/django_models.py +++ b/src/coretrace_python/bundled/models/django/django_models.py @@ -139,9 +139,9 @@ class DjangoModels(ModelPlugin): Sink(_sym("django.http.HttpResponseRedirect"), TaintKind.REDIRECT, _TARGET_ONLY), Sink(_sym("django.http.HttpResponsePermanentRedirect"), TaintKind.REDIRECT, _TARGET_ONLY), Sanitizer(_sym("django.utils.html.escape"), TaintKind.HTML), - TemplateRender(_sym("django.template.loader.render_to_string")), - TemplateRender(_sym("django.shortcuts.render"), 1), - TemplateRender(_sym("django.template.response.TemplateResponse"), 1, "template"), + TemplateRender(_sym("django.template.loader.render_to_string"), request=2), + TemplateRender(_sym("django.shortcuts.render"), 1, context=2, request=0), + TemplateRender(_sym("django.template.response.TemplateResponse"), 1, "template", 2, 0), TemplateRender(_sym("django.template.response.SimpleTemplateResponse"), 0, "template"), Sanitizer(_sym("django.utils.html.conditional_escape"), TaintKind.HTML), # The masked CSRF secret: ASCII letters and digits only, a malformed cookie is diff --git a/src/coretrace_python/engine.py b/src/coretrace_python/engine.py index 275b73e..774515c 100644 --- a/src/coretrace_python/engine.py +++ b/src/coretrace_python/engine.py @@ -83,6 +83,8 @@ SummaryAnalysis, SummaryIndex, SymbolRead, + TemplateCalls, + TemplateCallsAnalysis, build_module_graph, discover_sources, project_symbol, @@ -126,6 +128,7 @@ escaped_templates, project_templates, registered_routes, + request_processor, unread_renders, ) from coretrace_python.taint.urls import flow_url @@ -160,6 +163,7 @@ RegisteredRoutes, EscapedTemplates, ClearingAnalysis, + TemplateCallsAnalysis, ) @@ -200,6 +204,7 @@ class ResultsEvicted(TransformationPass): RegisteredRoutes, EscapedTemplates, ClearingAnalysis, + TemplateCallsAnalysis, } ) @@ -435,12 +440,18 @@ def analyze_project( escaped = escaped_templates(root) templates = project_templates(root) clearing = models.clearing(escaped) + template_calls = models.template_calls( + templates.filters, request_processor(modules[name] for name in sorted(analysable)) + ) for manager in analysable.values(): manager.provide(RegisteredRoutes, routes) manager.provide(EscapedTemplates, escaped) manager.provide(ClearingAnalysis, clearing) + manager.provide(TemplateCallsAnalysis, template_calls) - configuration = _configuration_key(components, plugins, models, advisories, dependencies, routes, escaped) + configuration = _configuration_key( + components, plugins, models, advisories, dependencies, routes, escaped, template_calls + ) keys = module_keys( graph, {name: fingerprint(configuration, str(files[name].source_id), name, files[name].text) for name in analysable}, @@ -482,6 +493,7 @@ def analyze_project( advisory_paths, _encode_routes(routes), tuple(sorted(escaped)), + template_calls, ), ) for component in pending @@ -694,6 +706,7 @@ class _Batch: advisory_paths: tuple[Path, ...] = () routes: tuple[tuple[str, str, str, int], ...] = () escaped: tuple[str, ...] = () + template_calls: TemplateCalls = field(default_factory=TemplateCalls) def _analyse_batch(batch: _Batch) -> dict[str, dict[str, Any]]: @@ -721,6 +734,7 @@ def _analyse_batch(batch: _Batch) -> dict[str, dict[str, Any]]: manager.provide(RegisteredRoutes, routes) manager.provide(EscapedTemplates, escaped) manager.provide(ClearingAnalysis, models.clearing(escaped)) + manager.provide(TemplateCallsAnalysis, batch.template_calls) module_plugins = tuple(p for p in all_plugins if not isinstance(p, ProjectPlugin)) results = _analyse_managers(managers, decode_index(batch.seed), module_plugins, affected) return {name: encode(entry) for name, entry in results.items()} @@ -749,6 +763,7 @@ def _configuration_key( dependencies: DependencyGraph, routes: Routes | None = None, escaped: frozenset[str] = frozenset(), + template_calls: TemplateCalls | None = None, ) -> str: """Everything a module's results depend on besides the project sources (ยง11).""" @@ -766,6 +781,7 @@ def _configuration_key( repr(dependencies.errors), repr(_encode_routes(routes or {})), repr(sorted(escaped)), + repr(template_calls), ) diff --git a/src/coretrace_python/interprocedural/__init__.py b/src/coretrace_python/interprocedural/__init__.py index 99642d2..edc8ba2 100644 --- a/src/coretrace_python/interprocedural/__init__.py +++ b/src/coretrace_python/interprocedural/__init__.py @@ -20,6 +20,7 @@ project_symbol, ) from coretrace_python.interprocedural.summaries import ( + FILTER_ARGUMENTS, Cleared, Clearing, ClearingAnalysis, @@ -31,10 +32,14 @@ SummaryAnalysis, SummaryIndex, SummaryTable, + TemplateCalls, + TemplateCallsAnalysis, + TemplateFilter, cleared_by, ) __all__ = [ + "FILTER_ARGUMENTS", "Arguments", "CallGraph", "CallGraphAnalysis", @@ -56,6 +61,9 @@ "SummaryTable", "SymbolRead", "Target", + "TemplateCalls", + "TemplateCallsAnalysis", + "TemplateFilter", "UnknownTarget", "build_module_graph", "cleared_by", diff --git a/src/coretrace_python/interprocedural/summaries.py b/src/coretrace_python/interprocedural/summaries.py index 33e8d75..819ecbf 100644 --- a/src/coretrace_python/interprocedural/summaries.py +++ b/src/coretrace_python/interprocedural/summaries.py @@ -7,14 +7,17 @@ security knowledge of their own: the taint engine decides which external symbols matter. They only keep, per parameter, the bits a ``Clearing`` the engine provides says the calls on the way cleared, on every path, so a caller's data reaches a sink or comes -back without the kinds a sanitizer inside the function removed. +back without the kinds a sanitizer inside the function removed. A call rendering a +template makes the filter calls ``TemplateCalls`` says the template makes, with the +values of the context entries it reads. """ from __future__ import annotations import dataclasses -from collections.abc import Iterable, Mapping +from collections.abc import Iterable, Iterator, Mapping from dataclasses import dataclass +from functools import cached_property from types import MappingProxyType from typing import ClassVar, TypeVar @@ -38,6 +41,7 @@ UnknownTarget, _is_staticmethod, ) +from coretrace_python.ir.defuse import DefUse, def_use from coretrace_python.ir.model import ( BasicBlock, Branch, @@ -145,6 +149,131 @@ def clears(self, symbol: SymbolId, arguments: Arguments) -> int: return bits +@dataclass(frozen=True) +class TemplateFilter: + """A filter call rendering a template makes: the function behind the filter, where + it is applied, in the template named ``template``, and the variables of the render, + each with its first attribute (``bio``, ``request.GET``), whose values reach its value + and its argument. They are sorted: the calls are part of the cache key.""" + + symbol: SymbolId + location: SourceSpan + template: str + value: tuple[str, ...] = () + argument: tuple[str, ...] = () + + +# What the analysis knows of the arguments of a filter call: nothing, so no condition on +# them is decided. +FILTER_ARGUMENTS = Arguments(unpacked=True) + + +@dataclass(frozen=True) +class Rendering: + """What a render call gives the template: the filter calls it makes, the context + ``entries`` by key, and the ``request`` the request context processor adds, if it + certainly does. An entry of that name shadows the request.""" + + filters: tuple[TemplateFilter, ...] + entries: Mapping[str, Value] + request: Value | None = None + + def values(self, names: tuple[str, ...]) -> Iterator[tuple[Value, str | None]]: + """The values reaching a filter through ``names``, each a variable with its first + attribute: a context entry, or the request with the attribute read from it.""" + + for name in names: + variable, _, attribute = name.partition(".") + if variable in self.entries: + yield self.entries[variable], None + elif variable == REQUEST and self.request is not None and attribute: + yield self.request, attribute + + +# The variable the request context processor adds. +REQUEST = "request" + + +@dataclass(frozen=True) +class TemplateCalls: + """The filter calls a render makes, provided by the engine from the project's + templates: ``renders`` gives, for each render symbol, the position and keyword of + the template's name, the position of its ``context`` and that of its ``request`` + (None when it takes none); ``filters`` the calls rendering each template makes, by + its name the way call sites record a constant; ``request`` says the request context + processor certainly runs for every template.""" + + renders: Mapping[SymbolId, tuple[int, str, int, int | None]] = dataclasses.field(default_factory=dict) + filters: Mapping[str, tuple[TemplateFilter, ...]] = dataclasses.field(default_factory=dict) + request: bool = False + + def rendering( + self, symbol: SymbolId, call: Call, arguments: Arguments, defs: Mapping[Value, Instruction], uses: DefUse + ) -> Rendering | None: + """What ``call`` gives the template it names, when it renders one whose filter + calls are known. None unless the context is certain: absent, or a dict literal + with constant keys that ``call`` alone uses; any other context may hold other + values, or a ``request`` entry, by the time the template reads it.""" + + render = self.renders.get(symbol) + if render is None: + return None + position, keyword, context, request = render + named = arguments.given(keyword, position) + filters = self.filters.get(named[1] or "", ()) if named is not None and named[0] else () + passed = _passed(call, context, "context") if filters else None + entries = _literal_entries(passed[1], defs, uses) if passed is not None and passed[0] else None + if entries is None: + return None + given = _passed(call, request, REQUEST) if self.request and request is not None else None + return Rendering(filters, entries, given[1] if given is not None and given[0] else None) + + +def _passed(call: Call, position: int, keyword: str) -> tuple[bool, Value | None]: + """Whether what ``call`` passes at ``position`` or as ``keyword`` is certain, and + that value: None when the call passes none.""" + + if call.starred or any(name is None for name, _ in call.keywords): + return False, None + if position < len(call.arguments): + return True, call.arguments[position] + return True, next((value for name, value in call.keywords if name == keyword), None) + + +def _is_none(value: Value | None, defs: Mapping[Value, Instruction]) -> bool: + constant = defs.get(value) if value is not None else None + return value is None or (isinstance(constant, Constant) and constant.value is None) + + +def _literal_entries(value: Value | None, defs: Mapping[Value, Instruction], uses: DefUse) -> Mapping[str, Value] | None: + """The entries of a context, by key: none for an absent context, None unless it is a + dict literal with constant keys that nothing else uses.""" + + if _is_none(value, defs): + return {} + built = defs.get(value) if value is not None else None + if value is None or not isinstance(built, BuildDict) or built.unpacked or len(uses.uses(value)) != 1: + return None + entries: dict[str, Value] = {} + for key, item in built.items: + constant = defs.get(key) + if not isinstance(constant, Constant) or not isinstance(constant.value, str): + return None + entries[constant.value] = item + return entries + + +class TemplateCallsAnalysis(Analysis[TemplateCalls]): + """The filter calls renders make, provided by the engine for a project; none on its + own.""" + + name: ClassVar[str] = "interprocedural.template_calls" + + @classmethod + def compute(cls, ctx: AnalysisContext) -> TemplateCalls: + return TemplateCalls() + + class ClearingAnalysis(Analysis[Clearing]): """What calls clear, provided by the engine from the security models; nothing on its own, so a summary then keeps every kind.""" @@ -278,6 +407,7 @@ def __init__( project: Mapping[SymbolId, FunctionSummary] | None = None, heap: HeapFacts | None = None, clearing: Clearing | None = None, + templates: TemplateCalls | None = None, ) -> None: self.name = name self.function = function @@ -286,6 +416,7 @@ def __init__( self.project = project or {} self.heap = heap or HeapFacts({}) self.clearing = clearing or Clearing() + self.templates = templates or TemplateCalls() self.blocks = {block.id: block for block in function.blocks} self.defs: dict[Value, Instruction] = { i.result: i for block in function.blocks for i in block.instructions if i.result @@ -428,6 +559,7 @@ def call(self, call: Call, state: dict[Key, Dep]) -> Dep: return self.known(project, arguments, keywords, everything, call, state, bound) given = self.graph.arguments_at(self.name, call.location) self.record(target.symbol, arguments, named, call.location, None, given) + self.record_filters(target.symbol, call, given, state) cleared = everything.clearing(self.clearing.clears(target.symbol, given)) return cleared | Dep(externals=frozenset({target.symbol})) if isinstance(target, KnownFunction): @@ -521,6 +653,31 @@ def mapped(indices: Dependencies, cleared: Cleared = ()) -> Dep: self.store(state, values[mutation.parameter], mutation.field, deps) return mapped(callee.return_dependencies, callee.return_cleared) | Dep(externals=callee.return_externals) + @cached_property + def uses(self) -> DefUse: + return def_use(self.function) + + def record_filters(self, symbol: SymbolId, call: Call, given: Arguments, state: Mapping[Key, Dep]) -> None: + """The filter calls a render makes, each at the filter in its template, with what + the context entries reaching its value and its argument depend on. The request the + request context processor adds is not followed through a project function.""" + + rendering = self.templates.rendering(symbol, call, given, self.defs, self.uses) + if rendering is None: + return + for template_filter in rendering.filters: + passed: list[Dep] = [] + for names in (template_filter.value, template_filter.argument): + deps = EMPTY + for value, attribute in rendering.values(names): + # Only the kinds of a request object say which of its attributes the + # user controls, and a dependency carries no kinds. + if attribute is None: + deps |= self.deep(value, state) + passed.append(deps) + if any(deps.parameters for deps in passed): + self.record(template_filter.symbol, tuple(passed), (), template_filter.location, None, FILTER_ARGUMENTS) + def record( self, symbol: SymbolId, @@ -570,8 +727,9 @@ def summarize( heap: HeapFacts | None = None, static: bool = False, clearing: Clearing | None = None, + templates: TemplateCalls | None = None, ) -> FunctionSummary: - problem = _DependenceProblem(name, function, graph, table, project, heap, clearing) + problem = _DependenceProblem(name, function, graph, table, project, heap, clearing, templates) solution = solve(problem, cfg) problem.external = {} problem.returns = EMPTY @@ -610,7 +768,15 @@ def summarize( class SummaryAnalysis(Analysis[SummaryTable]): name: ClassVar[str] = "interprocedural.summaries" requires: ClassVar[frozenset[AnyAnalysis]] = frozenset( - {CallGraphAnalysis, SSAAnalysis, CFGAnalysis, ProjectSummaries, HeapAnalysis, ClearingAnalysis} + { + CallGraphAnalysis, + SSAAnalysis, + CFGAnalysis, + ProjectSummaries, + HeapAnalysis, + ClearingAnalysis, + TemplateCallsAnalysis, + } ) @classmethod @@ -618,6 +784,7 @@ def compute(cls, ctx: AnalysisContext) -> SummaryTable: graph = ctx.get(CallGraphAnalysis) project = ctx.get(ProjectSummaries).summaries clearing = ctx.get(ClearingAnalysis) + templates = ctx.get(TemplateCallsAnalysis) table: dict[str, FunctionSummary] = {} supported: dict[str, tuple[FunctionIR, CFG, HeapFacts]] = {} for name, function in graph.definitions.items(): @@ -635,9 +802,8 @@ def compute(cls, ctx: AnalysisContext) -> SummaryTable: while changed: changed = False for name, (ssa, cfg, heap) in supported.items(): - updated = summarize( - name, ssa, cfg, graph, table, project, heap, _is_staticmethod(graph.definitions[name]), clearing - ) + static = _is_staticmethod(graph.definitions[name]) + updated = summarize(name, ssa, cfg, graph, table, project, heap, static, clearing, templates) if updated != table[name]: table[name] = updated changed = True diff --git a/src/coretrace_python/taint/__init__.py b/src/coretrace_python/taint/__init__.py index f28f56b..7d76731 100644 --- a/src/coretrace_python/taint/__init__.py +++ b/src/coretrace_python/taint/__init__.py @@ -39,6 +39,7 @@ ProjectTemplates, escaped_templates, project_templates, + request_processor, unread_renders, ) @@ -79,5 +80,6 @@ "project_templates", "propagate_taint", "registered_routes", + "request_processor", "unread_renders", ] diff --git a/src/coretrace_python/taint/engine.py b/src/coretrace_python/taint/engine.py index 6cf25b3..046a885 100644 --- a/src/coretrace_python/taint/engine.py +++ b/src/coretrace_python/taint/engine.py @@ -11,6 +11,7 @@ from collections.abc import Callable, Iterable, Mapping from dataclasses import dataclass, replace +from functools import cached_property from types import MappingProxyType from typing import ClassVar @@ -27,6 +28,7 @@ from coretrace_python.dataflow import DataflowProblem, Direction, solve from coretrace_python.hir import nodes from coretrace_python.interprocedural import ( + FILTER_ARGUMENTS, Arguments, CallGraph, CallGraphAnalysis, @@ -40,9 +42,12 @@ SummaryAnalysis, SummaryIndex, SummaryTable, + TemplateCalls, + TemplateCallsAnalysis, cleared_by, project_symbol, ) +from coretrace_python.ir.defuse import DefUse, def_use from coretrace_python.ir.lowering import analyzable_functions, is_body_function from coretrace_python.ir.model import ( BasicBlock, @@ -164,10 +169,12 @@ def __init__( seeds: Mapping[HeapLocation, Taint] | None = None, module: str = "", clearing: Clearing | None = None, + templates: TemplateCalls | None = None, ) -> None: self.name = name self.module = module self.clearing = clearing or Clearing() + self.templates = templates or TemplateCalls() self.function = function self.models = models self.graph = graph @@ -309,10 +316,14 @@ def attribute(self, taint: Taint, name: str) -> Taint: """What attribute ``name`` of a value carrying ``taint`` carries: text when every source of the value is a request object listing ``name`` among its text attributes.""" + return taint.without(_STRUCTURED) if self.requests_text(taint, name) else taint + + def requests_text(self, taint: Taint, name: str) -> bool: + """Whether every source of a value carrying ``taint`` is a request object listing + ``name`` among its text attributes.""" + requests = [self.models.request_object(source.symbol) for source in taint.sources] - if taint and all(request is not None and name in request.text for request in requests): - return taint.without(_STRUCTURED) - return taint + return bool(taint) and all(request is not None and name in request.text for request in requests) def call(self, call: Call, state: dict[Key, Taint], flows: list[TaintFlow]) -> Taint: arguments = tuple(self.deep(a, state) for a in call.arguments) @@ -443,12 +454,55 @@ def external( self.report( flows, sink, self.rendered(argument, state), argument, call, None, None, None, given, keyword=name ) + self.template_filters(symbol, call, given, state, flows) cleared = self.clearing.clears(symbol, given) if cleared: return everything.without(TaintKind(cleared)) # A method on a tainted object returns tainted data (``request.args.get``). return everything.join(state.get(call.callee, Taint.none())).join(self.covered(symbol)) + @cached_property + def uses(self) -> DefUse: + return def_use(self.function) + + def template_filters( + self, symbol: SymbolId, call: Call, given: Arguments, state: Mapping[Key, Taint], flows: list[TaintFlow] + ) -> None: + """Flows into the filters a render call makes, through the context entries each + filter's value and argument read, and the text attributes of the request the + request context processor adds, placed at the render call and naming the + template the filter is in.""" + + rendering = self.templates.rendering(symbol, call, given, self.defs, self.uses) + if rendering is None: + return + for template_filter in rendering.filters: + sink = self.models.sink(template_filter.symbol) + if sink is None: + continue + for position, names in enumerate((template_filter.value, template_filter.argument)): + taint, witness = Taint.none(), None + for value, attribute in rendering.values(names): + carried = self.deep(value, state) + if attribute is not None: + # The request the request context processor adds: only the text + # the user controls. + carried = carried.without(_STRUCTURED) if self.requests_text(carried, attribute) else Taint.none() + witness = value if witness is None and carried else witness + taint = taint.join(carried) + if witness is not None: + self.report( + flows, + sink, + taint, + witness, + call, + template_filter.template, + template_filter.location, + position, + FILTER_ARGUMENTS, + ) + def guarded(self, sink: Sink | None, arguments: Arguments) -> Sink | None: """The sink as this call makes it: an argument declared safe (``yaml.load`` with ``Loader=SafeLoader``) takes off the kinds it covers, and no kind left, no sink.""" @@ -903,8 +957,11 @@ def propagate_taint( seeds: Mapping[HeapLocation, Taint] | None = None, module: str = "", clearing: Clearing | None = None, + templates: TemplateCalls | None = None, ) -> TaintFacts: - problem = _TaintProblem(name, function, models, graph, summaries, parameters, project, heap, seeds, module, clearing) + problem = _TaintProblem( + name, function, models, graph, summaries, parameters, project, heap, seeds, module, clearing, templates + ) solution = solve(problem, cfg) taints: dict[Key, Taint] = {} flows: list[TaintFlow] = [] @@ -944,6 +1001,7 @@ class TaintAnalysis(FunctionAnalysis[TaintFacts]): HeapAnalysis, RegisteredRoutes, EscapedTemplates, + TemplateCallsAnalysis, } ) @@ -981,6 +1039,7 @@ def sources_of(member: nodes.Function) -> Mapping[int, Source]: seeds, ctx.module.name, models.clearing(ctx.get(EscapedTemplates)), + ctx.get(TemplateCallsAnalysis), ) diff --git a/src/coretrace_python/taint/models.py b/src/coretrace_python/taint/models.py index 2ed5d48..1cd08ed 100644 --- a/src/coretrace_python/taint/models.py +++ b/src/coretrace_python/taint/models.py @@ -17,7 +17,7 @@ from typing import ClassVar from coretrace_python.analysis import Analysis, AnalysisContext, MissingInputError -from coretrace_python.interprocedural import Clearing +from coretrace_python.interprocedural import Clearing, TemplateCalls, TemplateFilter from coretrace_python.semantic.symbols import Members, SymbolId @@ -198,12 +198,17 @@ class Validator: class TemplateRender: """A call rendering the template it names (argument ``position``, or ``keyword``) with autoescaping on, such as ``render_to_string``: what it returns carries no - ``HTML`` when the project shows that template escaping everything it renders. A - template it names that the engine cannot read may call any filter.""" + ``HTML`` when the project shows that template escaping everything it renders. The + template reads its variables from the ``context`` argument (at that position, or + ``context=``), and the request from the ``request`` one, None when the call takes + none; it makes the filter calls it applies. A template it names that the engine + cannot read may call any filter.""" symbol: SymbolId position: int = 0 keyword: str = "template_name" + context: int = 1 + request: int | None = None @dataclass(frozen=True) @@ -385,6 +390,18 @@ def clearing(self, escaped: frozenset[str] = frozenset()) -> Clearing: frozenset(repr(name) for name in escaped), ) + def template_calls(self, filters: Mapping[str, tuple[TemplateFilter, ...]], request: bool = False) -> TemplateCalls: + """The filter calls every template render makes, from the ``filters`` rendering + each template, by name, makes; ``request`` says the request context processor + certainly runs.""" + + return TemplateCalls( + {r.symbol: (r.position, r.keyword, r.context, r.request) for r in self.template_renders}, + # As call sites record a constant argument: the way Python writes it. + {repr(name): found for name, found in filters.items() if found}, + request, + ) + class SecurityModelRegistry: """Mutable collection point for the models plugins register.""" diff --git a/src/coretrace_python/taint/templates.py b/src/coretrace_python/taint/templates.py index 97eeca2..10bc65a 100644 --- a/src/coretrace_python/taint/templates.py +++ b/src/coretrace_python/taint/templates.py @@ -28,14 +28,21 @@ from __future__ import annotations import re -from collections.abc import Iterator, Mapping -from dataclasses import dataclass +from collections.abc import Iterable, Iterator, Mapping +from dataclasses import dataclass, field from pathlib import Path from types import MappingProxyType from typing import ClassVar from coretrace_python.analysis import Analysis, AnalysisContext -from coretrace_python.interprocedural import CallGraph, ExternalSymbol, discover_files +from coretrace_python.hir import nodes +from coretrace_python.hir.visitors import Node, children +from coretrace_python.interprocedural import ( + CallGraph, + ExternalSymbol, + TemplateFilter, + discover_files, +) from coretrace_python.semantic.symbols import SymbolId from coretrace_python.source import SourceId, SourceSpan from coretrace_python.taint.models import ModelTable @@ -111,7 +118,22 @@ _RAW_TEXT = re.compile(r"<(script|style)\b[^>]*>(.*?)", re.DOTALL | re.IGNORECASE) _COMMENT = re.compile(r"", re.DOTALL) _END_TAG = re.compile(r"]*>") -_APPLIED = re.compile(r"\|\s*(\w+)") +# Django's filter expressions: a constant or a variable, then filters, each with an +# optional argument, the way ``django.template.base.FilterExpression`` reads them. +_STRING = r"\"[^\"\\]*(?:\\.[^\"\\]*)*\"|'[^'\\]*(?:\\.[^'\\]*)*'" +_OPERAND = rf"{_STRING}|_\((?:{_STRING})\)|[\w.]+|[-+.]?\d[\d.e]*" +_HEAD = re.compile(rf"\s*({_OPERAND})?") +_FILTER = re.compile(rf"\s*\|\s*(\w+)(?::({_OPERAND}))?") +_TOKEN = re.compile(rf"(?:[^\s\"']+|{_STRING})+") +_BINDING = re.compile(r"(\w+)=") +# A variable with its first attribute: what a context entry holds, or what the request +# the request context processor adds gives (``request.GET``). +_VARIABLE = re.compile(r"([A-Za-z_]\w*)(?:\.(\w+))?") +_LITERALS = frozenset({"True", "False", "None"}) + +REQUEST_PROCESSOR = "django.template.context_processors.request" +_DJANGO_ENGINE = "django.template.backends.django.DjangoTemplates" +_ENGINES_SETTING = "TEMPLATES" def escaped_templates(root: Path) -> frozenset[str]: @@ -283,23 +305,86 @@ class FilterCall: @dataclass(frozen=True) class ProjectTemplates: """The templates found under the project's ``templates`` directories: their - ``names``, as a render call names them; the filters they apply, as ``calls``; and - where they name a template the engine cannot read, as ``unread``.""" + ``names``, as a render call names them; the filters they apply, as ``calls``; where + they name a template the engine cannot read, as ``unread``; and, by name, the filter + calls rendering each template makes with the context keys reaching them, as + ``filters``.""" names: frozenset[str] = frozenset() calls: tuple[FilterCall, ...] = () unread: tuple[str, ...] = () + filters: Mapping[str, tuple[TemplateFilter, ...]] = field(default_factory=dict) + + +@dataclass(frozen=True) +class _Application: + """A filter a template applies: the function behind it, where, the names free in the + template whose values reach its value and its argument, and the blocks it is in.""" + + symbol: SymbolId + span: SourceSpan + value: frozenset[str] + argument: frozenset[str] + blocks: tuple[str, ...] = () + + +@dataclass(frozen=True) +class _Reference: + """A template another one includes, or ``extends``, at ``offset`` in its ``blocks``: + by ``name``, None when an expression names it. The referenced template's free names + are ``bindings``, then, unless ``only``, what the ``scope`` binds there, then the + referencing template's.""" + + name: str | None + offset: int + bindings: Mapping[str, frozenset[str]] = field(default_factory=dict) + only: bool = False + scope: Mapping[str, frozenset[str]] = field(default_factory=dict) + blocks: tuple[str, ...] = () + extends: bool = False + + def maps(self, names: frozenset[str]) -> frozenset[str]: + found: set[str] = set() + for name in names: + variable = name.split(".", 1)[0] + if variable in self.bindings: + found |= _through(self.bindings[variable], name) + elif not self.only: + found |= _through(self.scope.get(variable, frozenset({variable})), name) + return frozenset(found) + + +@dataclass(frozen=True) +class _Template: + """What one template file applies and references, and the ``blocks`` it defines.""" + + applications: tuple[_Application, ...] + references: tuple[_Reference, ...] + blocks: frozenset[str] + + def rendered(self, blocks: tuple[str, ...], overridden: frozenset[str]) -> bool: + """Whether what stands in ``blocks`` renders when a template extending this one + defines the ``overridden`` blocks: a template extending another renders only + its blocks, where they replace the blocks of that name in what it extends.""" + + extends = any(reference.extends for reference in self.references) + return not overridden.intersection(blocks) and (bool(blocks) or not extends) def project_templates(root: Path) -> ProjectTemplates: """What the templates under ``root`` call, and the templates they include or extend that the engine cannot read: named by an expression, or by a name found under no - ``templates`` directory. A template file that cannot be read is unread too.""" + ``templates`` directory. A template file that cannot be read is unread too. A name + found in several files gives no ``filters``: which one renders depends on the + loaders.""" found = {path: tuple(_names(path.relative_to(root).parts)) for path in discover_files(root)} templates = sorted(path for path, names in found.items() if names) - names = frozenset(name for path in templates for name in found[path]) - calls: list[FilterCall] = [] + files: dict[str, list[Path]] = {} + for path in templates: + for name in found[path]: + files.setdefault(name, []).append(path) + read: dict[Path, _Template] = {} unread: list[str] = [] for path in templates: relative = path.relative_to(root).as_posix() @@ -307,26 +392,242 @@ def project_templates(root: Path) -> ProjectTemplates: if text is None: unread.append(f"{relative}, unreadable") continue - for offset, expression, tag in _expressions(text): - words = expression.split() - if tag and words[:1] == ["filter"]: - # ``{% filter striptags %}``: no ``|`` stands before its first filter. - start = expression.index("filter") + len("filter") - offset, expression = offset + start - 1, "|" + expression[start:] - blanked = re.sub(_QUOTED, lambda quoted: " " * len(quoted.group()), expression) - for applied in _APPLIED.finditer(blanked): - symbol = FILTER_FUNCTIONS.get(applied.group(1)) - if symbol is not None: - line, column = _position(text, offset + applied.start(1)) - calls.append(FilterCall(symbol, SourceSpan(SourceId(str(path)), line, column))) - if tag and words[:1] in (["include"], ["extends"]): - named = _constant(words[1]) if len(words) > 1 else None - where = f"{relative}:{_position(text, offset)[0]}" - if named is None: - unread.append(f"{where} names a template by an expression") - elif named not in names: - unread.append(f"{where} names {named!r}, not found") - return ProjectTemplates(names, tuple(calls), tuple(unread)) + read[path] = _read_scopes(text, SourceId(str(path))) + for reference in read[path].references: + where = f"{relative}:{_position(text, reference.offset)[0]}" + if reference.name is None: + unread.append(f"{where} names a template by an expression") + elif reference.name not in files: + unread.append(f"{where} names {reference.name!r}, not found") + calls = tuple(FilterCall(a.symbol, a.span) for template in read.values() for a in template.applications) + filters = { + name: tuple( + dict.fromkeys( + TemplateFilter(symbol, span, where, tuple(sorted(value)), tuple(sorted(argument))) + for symbol, span, where, value, argument in _reaching(name, files, read, frozenset()) + ) + ) + for name in files + } + return ProjectTemplates(frozenset(files), calls, tuple(unread), filters) + + +def _reaching( + name: str, + files: Mapping[str, list[Path]], + read: Mapping[Path, _Template], + visiting: frozenset[str], + overridden: frozenset[str] = frozenset(), +) -> Iterator[tuple[SymbolId, SourceSpan, str, frozenset[str], frozenset[str]]]: + """The filters rendering the template ``name`` applies, in it and in what it includes + or extends, each with the template applying it and the names free in ``name`` whose + values reach its value and its argument; the templates extending ``name`` on the way + define the ``overridden`` blocks.""" + + paths = files.get(name, []) + if len(paths) != 1 or paths[0] not in read or name in visiting: + return + template = read[paths[0]] + for application in template.applications: + if template.rendered(application.blocks, overridden): + yield application.symbol, application.span, name, application.value, application.argument + for reference in template.references: + if reference.name is None: + continue + if reference.extends: + found = _reaching(reference.name, files, read, visiting | {name}, overridden | template.blocks) + elif template.rendered(reference.blocks, overridden): + found = _reaching(reference.name, files, read, visiting | {name}) + else: + continue + for symbol, span, where, value, argument in found: + yield symbol, span, where, reference.maps(value), reference.maps(argument) + + +def _read_scopes(text: str, source: SourceId) -> _Template: + """The filters a template applies and the templates it includes or extends, with the + names free in it that reach each: through ``{% for %}``, ``{% with %}`` and the + ``as`` of other tags, which bind a name until their block ends, the arguments of + earlier filters, and the output of ``{% filter %}`` blocks.""" + + applications: list[_Application] = [] + references: list[_Reference] = [] + opened: list[str] = [] + defined: set[str] = set() + scopes: list[dict[str, frozenset[str]]] = [{}] + blocks: list[tuple[list[tuple[str, int, frozenset[str]]], set[str]]] = [] + + def free(names: frozenset[str]) -> frozenset[str]: + found: set[str] = set() + for name in names: + variable = name.split(".", 1)[0] + bound = next((scope[variable] for scope in reversed(scopes) if variable in scope), None) + found |= {name} if bound is None else _through(bound, name) + return frozenset(found) + + def apply(head: frozenset[str], chain: list[tuple[str, int, frozenset[str]]]) -> frozenset[str]: + """Record the filters of ``chain`` applied to a value the free names ``head`` + reach; the free names reaching the value of the whole expression.""" + + value = head + for name, offset, argument in chain: + symbol = FILTER_FUNCTIONS.get(name) + if symbol is not None: + line, column = _position(text, offset) + span = SourceSpan(source, line, column) + applications.append(_Application(symbol, span, value, free(argument), tuple(opened))) + value |= free(argument) + return value + + for offset, expression, tag in _expressions(text): + if not tag: + head, chain = _chain(expression, offset) + printed = apply(free(head), chain) + for _, collected in blocks: + collected |= printed + continue + tokens = [(match.start() + offset, match.group()) for match in _TOKEN.finditer(expression)] + words = [word for _, word in tokens] + name = words[0] if words else "" + if name == "filter": + # ``{% filter lower|striptags %}``: filters without a value, applied to the + # block's output at its end. + start = expression.index("filter") + len("filter") + blocks.append((_chain("|" + expression[start:], offset + start - 1)[1], set())) + continue + if name == "endfilter" and blocks: + chain, collected = blocks.pop() + printed = apply(frozenset(collected), chain) + if blocks: + blocks[-1][1].update(printed) + continue + values: dict[str, frozenset[str]] = {} + bindings: dict[str, frozenset[str]] = {} + for start, word in tokens[1:]: + bound = _BINDING.match(word) + skipped = bound.end() if bound else 0 + head, chain = _chain(word[skipped:], start + skipped) + reached = apply(free(head), chain) + if bound: + bindings[bound.group(1)] = reached + else: + values.setdefault(word, reached) + if name == "block" and len(words) > 1: + opened.append(words[1]) + defined.add(words[1]) + elif name == "endblock" and opened: + opened.pop() + elif name == "for" and "in" in words: + loop = words.index("in") + targets = [t for word in words[1:loop] for t in word.split(",") if t] + iterated = values.get(words[loop + 1], frozenset()) if loop + 1 < len(words) else frozenset() + scopes.append({"forloop": frozenset(), **{target: iterated for target in targets}}) + elif name == "with": + as_form = len(words) == 4 and words[2] == "as" + scopes.append({words[3]: values.get(words[1], frozenset())} if as_form else bindings) + elif name in ("endfor", "endwith") and len(scopes) > 1: + scopes.pop() + elif name in ("include", "extends"): + named = _constant(words[1]) if len(words) > 1 else None + if name == "extends": + references.append(_Reference(named, offset, extends=True)) + else: + context = {k: v for scope in scopes for k, v in scope.items()} + references.append(_Reference(named, offset, bindings, "only" in words, context, tuple(opened))) + elif "as" in words[1:-1]: + target = words[words.index("as", 1) + 1] + scopes[-1][target] = values.get(words[1], frozenset()) if name == "regroup" else frozenset() + return _Template(tuple(applications), tuple(references), frozenset(defined)) + + +def _chain(expression: str, offset: int) -> tuple[frozenset[str], list[tuple[str, int, frozenset[str]]]]: + """What a filter expression's value is reached by, and its filters, each with the + offset of its name and what its argument is reached by.""" + + head = _HEAD.match(expression) + assert head is not None # every part of it is optional + chain = [ + (applied.group(1), offset + applied.start(1), _reached_by(applied.group(2))) + for applied in _FILTER.finditer(expression, head.end()) + ] + return _reached_by(head.group(1)), chain + + +def _reached_by(operand: str | None) -> frozenset[str]: + """The variable an operand reads, with its first attribute; none for a constant.""" + + variable = _VARIABLE.match(operand) if operand else None + return frozenset() if variable is None or variable.group(1) in _LITERALS else frozenset({variable.group()}) + + +def _through(bound: frozenset[str], name: str) -> frozenset[str]: + """What ``name``, a variable with its first attribute, reads when its variable is + ``bound`` to those names: ``query.q`` with ``query`` bound to ``request.GET`` reads + ``request.GET``.""" + + attribute = name.partition(".")[2] + return frozenset(".".join(f"{b}.{attribute}".split(".")[:2]) if attribute else b for b in bound) + + +def request_processor(modules: Iterable[nodes.Module]) -> bool: + """Whether every Django template engine the project's settings configure runs the + ``request`` context processor, for certain: each assignment to ``TEMPLATES`` is a + literal list of literal engines, at least one of them ``DjangoTemplates``, and each + of those lists the processor in a literal ``context_processors``; no other code + names ``TEMPLATES``, which could change it. A mere mention of the processor proves + nothing.""" + + assignments = mentions = 0 + for module in modules: + stack: list[Node] = list(module.body) + while stack: + node = stack.pop() + stack.extend(children(node)) + if isinstance(node, nodes.Name) and node.identifier == _ENGINES_SETTING: + mentions += 1 + elif isinstance(node, nodes.Assign) and _names_engines(node.target): + if not _runs_request_processor(node.value): + return False + assignments += 1 + return assignments > 0 and mentions == assignments + + +def _names_engines(target: nodes.Expression) -> bool: + return isinstance(target, nodes.Name) and target.identifier == _ENGINES_SETTING + + +def _runs_request_processor(engines: nodes.Expression) -> bool: + if not isinstance(engines, nodes.List | nodes.Tuple): + return False + django = 0 + for engine in engines.elements: + settings = _literal_dict(engine) + backend = settings.get("BACKEND") if settings is not None else None + if settings is None or not isinstance(backend, nodes.Constant) or not isinstance(backend.value, str): + return False + if backend.value != _DJANGO_ENGINE: + continue + django += 1 + options = _literal_dict(settings.get("OPTIONS")) + processors = options.get("context_processors") if options is not None else None + if not isinstance(processors, nodes.List | nodes.Tuple): + return False + if not any(isinstance(p, nodes.Constant) and p.value == REQUEST_PROCESSOR for p in processors.elements): + return False + return django > 0 + + +def _literal_dict(expression: nodes.Expression | None) -> dict[str, nodes.Expression] | None: + """A dict display with constant string keys, by key; None for anything else.""" + + if not isinstance(expression, nodes.Dict): + return None + found: dict[str, nodes.Expression] = {} + for key, value in expression.items: + if not isinstance(key, nodes.Constant) or not isinstance(key.value, str): + return None + found[key.value] = value + return found def unread_renders(module: str, graph: CallGraph, models: ModelTable, names: frozenset[str]) -> Iterator[str]: diff --git a/tests/test_template_context_flows.py b/tests/test_template_context_flows.py new file mode 100644 index 0000000..1fe2e88 --- /dev/null +++ b/tests/test_template_context_flows.py @@ -0,0 +1,306 @@ +"""Render context data reaching template filters. + +``render(request, "app/profile.html", {"bio": request.POST["bio"]})`` renders a template +applying ``{{ bio|striptags }}``: attacker input reaches ``striptags`` through the context, +so an advisory whose entry point is the filter is exploitable at the render call, not +only reachable at the template's line. + +The engine links a filter to the context only where it is certain of the value: + +- the context is a dict literal with constant keys, used by the render call and nothing + else, inline or through a variable; a dict mutated or passed elsewhere, one unpacking + another (``**extra``), or a context built any other way leaves the filter reachable; +- the filter's value is a context variable, directly or through ``{% for %}`` and + ``{% with %}``, the templates it includes or extends by a constant name, the arguments + of the filters before it, or the output of a ``{% filter %}`` block; a name a tag + binds shadows the context, and ``{% include ... only %}`` passes none of it; +- a template name found in one file only: of two files of the same name, which one + renders depends on the loaders; +- a filter a template rendering it renders: a template extending another renders only + its blocks, and they replace the blocks of that name in what it extends. + +A filter's argument (``cut:bio``) is its second argument, so an advisory naming only the +value as attacker argument is not exploitable through it. The same holds when a project +function renders the template with the values it receives. +""" + +from __future__ import annotations + +import json +from datetime import UTC, datetime +from pathlib import Path +from typing import Any + +import pytest + +from coretrace_python import engine +from coretrace_python.cache import ProjectCache +from coretrace_python.dependency import ( + Advisory, + AdvisoryEntryPoint, + AttackerArgument, + dump_advisories, + render_vex, +) +from coretrace_python.findings import Severity +from coretrace_python.semantic.symbols import SymbolId + +NOW = datetime(2026, 9, 27, 12, 0, tzinfo=UTC) +STRIPTAGS = "python.django.template.defaultfilters.striptags" +CUT = "python.django.template.defaultfilters.cut" +LOCK = """version = 1 + +[[package]] +name = "app" +version = "0.1.0" +source = { virtual = "." } +dependencies = [{ name = "django" }] + +[[package]] +name = "django" +version = "4.2.16" +source = { registry = "https://pypi.org/simple" } +""" +HEADER = ( + "from django.http import HttpRequest, HttpResponse\n" + "from django.shortcuts import render\n" + "from django.template.loader import render_to_string\n" + "from django.template.response import TemplateResponse\n\n" + "def profile(request: HttpRequest):\n" +) +LINE = 7 +PAGE = "app/templates/app/profile.html" +POSTED = "{'bio': request.POST['bio']}" + + +def advisory(*symbols: str, attacker: tuple[AttackerArgument, ...] = ()) -> Advisory: + return Advisory( + "CVE-2099-5401", + "django", + "<4.2.17", + "a filter is quadratic in its input", + Severity.MEDIUM, + entry_points=tuple(AdvisoryEntryPoint(SymbolId(s), "a template filter", (), False, attacker) for s in symbols), + ) + + +def check( + root: Path, body: str, templates: dict[str, str], advisories: tuple[Advisory, ...] | None = None, **options: Any +) -> engine.ProjectAnalysis: + files = {"uv.lock": LOCK, "app/__init__.py": "", "app/views.py": HEADER + body, **templates} + for relative, text in files.items(): + path = root / relative + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(text, encoding="utf-8") + chosen = advisories if advisories is not None else (advisory(STRIPTAGS),) + (root / "advisories.json").write_text(dump_advisories(chosen), encoding="utf-8") + return engine.analyze_project(root, [engine.BUNDLED_PLUGINS], **options) + + +def exploitable(analysis: engine.ProjectAnalysis) -> list[tuple[int, str, str, str]]: + return [ + (f.span.start_line, f.metadata["symbol"], f.metadata["through"], f.metadata["sink_line"]) + for f in analysis.findings + if f.rule_id == "exploitable-vulnerability" + ] + + +def render(context: str = POSTED, name: str = "'app/profile.html'") -> str: + return f" return render(request, {name}, {context})\n" + + +# --------------------------------------------------------------------------- flows + + +def test_attacker_input_in_the_context_reaches_the_filter_the_template_applies(tmp_path: Path) -> None: + analysis = check(tmp_path, render(), {PAGE: "

Profile

\n{{ bio|striptags }}\n"}) + + assert exploitable(analysis) == [(LINE, STRIPTAGS, "app/profile.html", "2")] + finding = next(f for f in analysis.findings if f.rule_id == "exploitable-vulnerability") + assert finding.metadata["source_label"] == "http" + assert Path(str(finding.span.source_id)).relative_to(tmp_path).as_posix() == "app/views.py" + + +@pytest.mark.parametrize( + "body, line", + [ + (f" context = {POSTED}\n return render(request, 'app/profile.html', context)\n", LINE + 1), + (f" return render(request, 'app/profile.html', context={POSTED})\n", LINE), + (f" return HttpResponse(render_to_string('app/profile.html', {POSTED}))\n", LINE), + (f" return TemplateResponse(request, 'app/profile.html', {POSTED})\n", LINE), + (" bio = request.POST['bio']\n return render(request, 'app/profile.html', {'title': 'Me', 'bio': bio})\n", LINE + 1), + ], + ids=["variable", "keyword", "render-to-string", "template-response", "among-other-keys"], +) +def test_a_dict_literal_used_only_by_the_render_is_the_context(tmp_path: Path, body: str, line: int) -> None: + analysis = check(tmp_path, body, {PAGE: "{{ bio|striptags }}\n"}) + + assert [(found[0], found[1]) for found in exploitable(analysis)] == [(line, STRIPTAGS)] + + +CARD = "app/templates/app/card.html" +BASE = "app/templates/app/base.html" + + +@pytest.mark.parametrize( + "templates", + [ + {PAGE: "{{ bio|lower|striptags }}"}, + {PAGE: "{{ ''|default:bio|striptags }}"}, + {PAGE: "{% if bio|striptags %}yes{% endif %}"}, + {PAGE: "{% with text=bio %}{{ text|striptags }}{% endwith %}"}, + {PAGE: "{% with bio as text %}{{ text|striptags }}{% endwith %}"}, + {PAGE: "{% for line in bio.splitlines %}{{ line|striptags }}{% endfor %}"}, + {PAGE: "{% filter striptags %}

{{ bio }}

{% endfilter %}"}, + {PAGE: "{% include 'app/card.html' %}", CARD: "{{ bio|striptags }}"}, + {PAGE: "{% include 'app/card.html' with text=bio %}", CARD: "{{ text|striptags }}"}, + {PAGE: "{% for b in bio.split %}{% include 'app/card.html' %}{% endfor %}", CARD: "{{ b|striptags }}"}, + {PAGE: "{% extends 'app/base.html' %}", BASE: "{{ bio|striptags }}"}, + {PAGE: "{% with bio='Hello' %}{{ bio }}{% endwith %}{% for bio in 'ab' %}{% endfor %}{{ bio|striptags }}"}, + { + PAGE: "{% extends 'app/base.html' %}{% block body %}{{ bio|striptags }}{% endblock %}", + BASE: "
{% block body %}{% endblock %}
", + }, + { + PAGE: "{% extends 'app/base.html' %}{% block title %}Me{% endblock %}", + BASE: "{% block title %}{% endblock %}{% block body %}{{ bio|striptags }}{% endblock %}", + }, + ], + ids=[ + "chained", "earlier-argument", "tag-argument", "with", "with-as", "for", "filter-block", "include", + "include-with", "include-in-loop", "extends", "after-shadowing-blocks", "child-block", + "parent-block-kept", + ], +) +def test_template_scopes_carry_the_context_value_to_the_filter(tmp_path: Path, templates: dict[str, str]) -> None: + analysis = check(tmp_path, render(), templates) + + assert [(found[0], found[1]) for found in exploitable(analysis)] == [(LINE, STRIPTAGS)] + + +@pytest.mark.parametrize( + "body, templates", + [ + (render("{'bio': 'Hello', 'other': request.POST['bio']}"), {PAGE: "{{ bio|striptags }}"}), + (render(), {PAGE: "{% with bio='Hello' %}{{ bio|striptags }}{% endwith %}"}), + (render(), {PAGE: "{% for bio in 'ab' %}{{ bio|striptags }}{% endfor %}"}), + (render(), {PAGE: "{% now 'Y' as bio %}{{ bio|striptags }}"}), + (render(), {PAGE: "{% include 'app/card.html' only %}", CARD: "{{ bio|striptags }}"}), + (render(), {PAGE: "{% include 'app/card.html' with bio='Hello' %}", CARD: "{{ bio|striptags }}"}), + (render(), {PAGE: "{# {{ bio|striptags }} #}{{ bio }}"}), + ( + f" context = {POSTED}\n context['bio'] = 'Hello'\n return render(request, 'app/profile.html', context)\n", + {PAGE: "{{ bio|striptags }}"}, + ), + ( + f" context = {POSTED}\n log(context)\n return render(request, 'app/profile.html', context)\n", + {PAGE: "{{ bio|striptags }}"}, + ), + (render("{**request.session, 'bio': request.POST['bio']}"), {PAGE: "{{ bio|striptags }}"}), + (render("dict(bio=request.POST['bio'])"), {PAGE: "{{ bio|striptags }}"}), + ( + render(), + {PAGE: "{{ bio|striptags }}", "other/templates/app/profile.html": "{{ bio|striptags }}"}, + ), + ( + render(), + { + PAGE: "{% extends 'app/base.html' %}{% block body %}{{ bio }}{% endblock %}", + BASE: "{% block body %}{{ bio|striptags }}{% endblock %}", + }, + ), + ( + render(), + { + PAGE: "{% extends 'app/base.html' %}{{ bio|striptags }}{% block body %}{% endblock %}", + BASE: "{% block body %}{% endblock %}", + }, + ), + ( + render(), + { + PAGE: "{% extends 'app/middle.html' %}{% block body %}{{ bio }}{% endblock %}", + "app/templates/app/middle.html": "{% extends 'app/base.html' %}{% block body %}{{ bio|striptags }}{% endblock %}", + BASE: "{% block body %}{% endblock %}", + }, + ), + ], + ids=[ + "other-key", "with-shadows", "for-shadows", "as-shadows", "include-only", "include-with-shadows", + "comment", "mutated", "passed-elsewhere", "unpacked", "not-a-literal", "two-files", + "overridden-block", "outside-child-blocks", "overridden-twice", + ], +) +def test_a_value_the_engine_cannot_link_to_the_filter_leaves_it_reachable( + tmp_path: Path, body: str, templates: dict[str, str] +) -> None: + analysis = check(tmp_path, body, templates) + + assert exploitable(analysis) == [] + + +def test_a_filter_argument_is_its_second_argument(tmp_path: Path) -> None: + value_only = (AttackerArgument("value", 0),) + templates = {PAGE: "{{ 'Hello'|cut:bio }}\n{{ bio|striptags }}\n"} + + through_any = check(tmp_path / "any", render(), templates, (advisory(CUT),)) + through_value = check(tmp_path / "value", render(), templates, (advisory(CUT, STRIPTAGS, attacker=value_only),)) + + assert [found[:2] for found in exploitable(through_any)] == [(LINE, CUT)] + assert [found[:2] for found in exploitable(through_value)] == [(LINE, STRIPTAGS)] + + +def test_a_project_function_rendering_what_it_receives_carries_it_to_the_filter(tmp_path: Path) -> None: + helpers = ( + "from django.shortcuts import render\n\n" + "def show(request, bio):\n" + " return render(request, 'app/profile.html', {'bio': bio})\n" + ) + views = ( + "from django.http import HttpRequest\n\n" + "from app.helpers import show\n\n" + "def profile(request: HttpRequest):\n" + " return show(request, request.POST['bio'])\n" + ) + templates = {PAGE: "

\n{{ bio|striptags }}\n

\n", "app/helpers.py": helpers, "app/views.py": views} + + analysis = check(tmp_path, "", templates) + + assert exploitable(analysis) == [(6, STRIPTAGS, "app.helpers.show", "2")] + + +# --------------------------------------------------------------------------- engine + + +def test_the_vex_report_names_the_render_call_as_exploitable(tmp_path: Path) -> None: + analysis = check(tmp_path, render(), {PAGE: "{{ bio|striptags }}\n"}) + evidence = (*analysis.findings, *analysis.suppressed, *analysis.accepted) + + document = render_vex( + analysis.dependencies, analysis.advisories, evidence, analysis.coverage, tmp_path, "coretrace", "0.16.0", NOW + ) + + statement = json.loads(document)["statements"][0] + assert statement["status"] == "affected" + assert statement["status_notes"] == ( + f"Reached by the project's code at {PAGE}:1 {STRIPTAGS} (reachable); app/views.py:{LINE} {STRIPTAGS} (exploitable)." + ) + + +def test_a_template_losing_its_filter_invalidates_the_cached_results(tmp_path: Path) -> None: + cache = ProjectCache(tmp_path / "cache") + root = tmp_path / "src" + + first = check(root, render(), {PAGE: "{{ bio|striptags }}\n"}, cache=cache) + (root / PAGE).write_text("{{ bio|lower }}\n", encoding="utf-8") + second = engine.analyze_project(root, [engine.BUNDLED_PLUGINS], cache=cache) + + assert [found[1] for found in exploitable(first)] == [STRIPTAGS] + assert second.reused == () + assert exploitable(second) == [] + + +def test_modules_analysed_in_other_processes_see_the_template_filters(tmp_path: Path) -> None: + analysis = check(tmp_path, render(), {PAGE: "{{ bio|striptags }}\n", "app/other.py": "X = 1\n"}, jobs=2) + + assert [found[1] for found in exploitable(analysis)] == [STRIPTAGS] diff --git a/tests/test_template_request_processor.py b/tests/test_template_request_processor.py new file mode 100644 index 0000000..95117fa --- /dev/null +++ b/tests/test_template_request_processor.py @@ -0,0 +1,215 @@ +"""The request a context processor adds to what a template renders. + +With ``django.template.context_processors.request`` enabled, a template rendered with the +request reads it as ``request``: ``{{ request.GET.q|striptags }}`` passes attacker input +to ``striptags`` though no context entry carries it. The engine counts it only where it +is certain: + +- the project's settings assign ``TEMPLATES`` a literal list, every + ``DjangoTemplates`` engine of it lists the processor in a literal + ``context_processors``, and no other code of the project names ``TEMPLATES``; a mere + mention of the processor elsewhere proves nothing; +- the render call passes the request: ``render`` and ``TemplateResponse`` always do, + ``render_to_string`` when given ``request``, ``SimpleTemplateResponse`` never; +- the render's context is certain, so it cannot hide a ``request`` entry: absent, or a + dict literal only that call uses; an entry of that name, a name a tag binds, or + ``{% include ... only %}`` shadows the processor's request; +- the template reads text the user controls: an attribute the request object lists + among its text attributes, such as ``GET``, ``POST``, ``COOKIES`` or ``headers``, not + ``user`` or ``session``. + +Otherwise the filter stays reachable. Through a project function, only the context +entries are followed. +""" + +from __future__ import annotations + +from pathlib import Path + +import pytest + +from coretrace_python import engine +from coretrace_python.dependency import Advisory, AdvisoryEntryPoint, dump_advisories +from coretrace_python.findings import Severity +from coretrace_python.semantic.symbols import SymbolId + +STRIPTAGS = "python.django.template.defaultfilters.striptags" +ADVISORY = Advisory( + "CVE-2099-5501", + "django", + "<4.2.17", + "striptags is quadratic in its input", + Severity.MEDIUM, + entry_points=(AdvisoryEntryPoint(SymbolId(STRIPTAGS), "a template filter"),), +) +LOCK = """version = 1 + +[[package]] +name = "app" +version = "0.1.0" +source = { virtual = "." } +dependencies = [{ name = "django" }] + +[[package]] +name = "django" +version = "4.2.16" +source = { registry = "https://pypi.org/simple" } +""" +PROCESSOR = "django.template.context_processors.request" +DJANGO = "django.template.backends.django.DjangoTemplates" +JINJA = "django.template.backends.jinja2.Jinja2" + + +def engine_settings(processors: str = f"'{PROCESSOR}', 'django.contrib.auth.context_processors.auth'", backend: str = DJANGO) -> str: + return f"{{'BACKEND': '{backend}', 'APP_DIRS': True, 'OPTIONS': {{'context_processors': [{processors}]}}}}" + + +SETTINGS = f"TEMPLATES = [{engine_settings()}]\n" +HEADER = ( + "from django.http import HttpRequest, HttpResponse\n" + "from django.shortcuts import render\n" + "from django.template.loader import render_to_string\n" + "from django.template.response import SimpleTemplateResponse, TemplateResponse\n\n" + "def search(request: HttpRequest):\n" +) +LINE = 7 +PAGE = "app/templates/app/search.html" +RENDER = " return render(request, 'app/search.html')\n" +READS = "{{ request.GET.q|striptags }}\n" + + +def exploitable(root: Path, body: str = RENDER, template: str = READS, **files: str) -> list[int]: + files = { + "uv.lock": LOCK, + "app/__init__.py": "", + "app/settings.py": SETTINGS, + "app/views.py": HEADER + body, + PAGE: template, + **files, + } + for relative, text in files.items(): + path = root / relative + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(text, encoding="utf-8") + (root / "advisories.json").write_text(dump_advisories((ADVISORY,)), encoding="utf-8") + findings = engine.analyze_project(root, [engine.BUNDLED_PLUGINS]).findings + return [f.span.start_line for f in findings if f.rule_id == "exploitable-vulnerability"] + + +# --------------------------------------------------------------------------- reached + + +@pytest.mark.parametrize( + "body", + [ + RENDER, + " return render(request, 'app/search.html', {'title': 'Search'})\n", + " return HttpResponse(render_to_string('app/search.html', request=request))\n", + " return HttpResponse(render_to_string('app/search.html', {'title': 'Search'}, request))\n", + " return TemplateResponse(request, 'app/search.html')\n", + ], + ids=["render", "render-with-context", "render-to-string", "render-to-string-positional", "template-response"], +) +def test_a_render_passing_the_request_gives_it_to_the_template(tmp_path: Path, body: str) -> None: + assert exploitable(tmp_path, body) == [LINE] + + +@pytest.mark.parametrize( + "template", + [ + "{{ request.POST.bio|striptags }}", + "{{ request.headers.referer|striptags }}", + "{% with q=request.GET.q %}{{ q|striptags }}{% endwith %}", + "{% with query=request.GET %}{{ query.q|striptags }}{% endwith %}", + "{% for value in request.GET.values %}{{ value|striptags }}{% endfor %}", + "{% include 'app/card.html' %}", + ], + ids=["post", "headers", "with", "with-attribute", "for", "include"], +) +def test_text_the_user_controls_reaches_the_filter(tmp_path: Path, template: str) -> None: + card = {"app/templates/app/card.html": READS} + + assert exploitable(tmp_path, template=template, **card) == [LINE] + + +def test_only_django_engines_need_the_processor(tmp_path: Path) -> None: + settings = f"TEMPLATES = [{engine_settings()}, {engine_settings('', JINJA)}]\n" + + assert exploitable(tmp_path, **{"app/settings.py": settings}) == [LINE] + + +# --------------------------------------------------------------------------- uncertain + + +@pytest.mark.parametrize( + "settings", + [ + f"TEMPLATES = [{engine_settings(repr('django.contrib.auth.context_processors.auth'))}]\n", + "DEBUG = True\n", + f"TEMPLATES = [{engine_settings()}, {engine_settings('')}]\n", + f"PROCESSORS = [{PROCESSOR!r}]\nTEMPLATES = [{engine_settings('*PROCESSORS')}]\n", + f"PROCESSORS = [{PROCESSOR!r}]\nTEMPLATES = [{{'BACKEND': {DJANGO!r}, 'OPTIONS': {{'context_processors': PROCESSORS}}}}]\n", + f"TEMPLATES = [{engine_settings()}]\nTEMPLATES[0]['OPTIONS']['context_processors'].pop(0)\n", + f"TEMPLATES = [{engine_settings()}]\nif DEBUG:\n TEMPLATES = []\n", + f"# {PROCESSOR}\nNOTE = {PROCESSOR!r}\n", + ], + ids=[ + "not-listed", "no-templates", "one-engine-without", "unpacked", "not-a-literal", "mutated", + "reassigned", "mentioned-only", + ], +) +def test_settings_that_do_not_prove_the_processor_leave_the_filter_reachable(tmp_path: Path, settings: str) -> None: + assert exploitable(tmp_path, **{"app/settings.py": settings}) == [] + + +def test_settings_named_in_other_code_leave_the_filter_reachable(tmp_path: Path) -> None: + local = "from app.settings import *\n\nTEMPLATES[0]['OPTIONS']['debug'] = True\n" + + assert exploitable(tmp_path, **{"app/local.py": local}) == [] + + +@pytest.mark.parametrize( + "body", + [ + " return HttpResponse(render_to_string('app/search.html'))\n", + " return HttpResponse(render_to_string('app/search.html', request=None))\n", + " return SimpleTemplateResponse('app/search.html')\n", + " return render(request, 'app/search.html', {'request': 'none'})\n", + " context = {'title': 'Search'}\n context.update(extra())\n return render(request, 'app/search.html', context)\n", + " return render(request, 'app/search.html', build())\n", + ], + ids=["no-request", "request-none", "simple-template-response", "shadowed-entry", "mutated-context", "unknown-context"], +) +def test_a_render_that_does_not_certainly_pass_the_request_leaves_the_filter_reachable( + tmp_path: Path, body: str +) -> None: + assert exploitable(tmp_path, body) == [] + + +@pytest.mark.parametrize( + "template, files", + [ + ("{{ request.user.username|striptags }}", {}), + ("{{ request.session.q|striptags }}", {}), + ("{{ request|striptags }}", {}), + ("{% with request=other %}{{ request.GET.q|striptags }}{% endwith %}", {}), + ("{% include 'app/card.html' only %}", {"app/templates/app/card.html": READS}), + ], + ids=["user", "session", "whole-request", "shadowed-by-with", "include-only"], +) +def test_what_is_not_user_text_or_is_shadowed_leaves_the_filter_reachable( + tmp_path: Path, template: str, files: dict[str, str] +) -> None: + assert exploitable(tmp_path, template=template, **files) == [] + + +def test_a_project_function_rendering_with_the_request_follows_context_entries_only(tmp_path: Path) -> None: + helpers = "from django.shortcuts import render\n\ndef show(request):\n return render(request, 'app/search.html')\n" + views = ( + "from django.http import HttpRequest\n\n" + "from app.helpers import show\n\n" + "def search(request: HttpRequest):\n" + " return show(request)\n" + ) + + assert exploitable(tmp_path, **{"app/helpers.py": helpers, "app/views.py": views}) == []