Skip to content

fix(parser): reject whitespace in the URL token - #1379

Open
rajath201 wants to merge 2 commits into
pypa:mainfrom
rajath201:url-token-whitespace
Open

fix(parser): reject whitespace in the URL token#1379
rajath201 wants to merge 2 commits into
pypa:mainfrom
rajath201:url-token-whitespace

Conversation

@rajath201

Copy link
Copy Markdown
Contributor

The URL token in the dependency-specifier tokenizer matches [^ \t]+, so it stops at spaces and tabs but keeps going through a newline. A direct-reference requirement like foo @ https://host/foo.whl\nevil==1 parses without complaint, with the trailing newline and evil==1 folded into url. Calling str() on that requirement then prints two lines, so any tool that writes parsed requirements one per line (a requirements file, a generated lockfile) picks up the injected second dependency.

Match the URL token with \S+ instead, so it ends at any whitespace, in line with RFC 3986 having no whitespace in a URI. The name, extras, and specifier tokens already reject newlines, and marker values go through ast.literal_eval, so this just brings the URL branch up to the same footing. Valid URLs never carry whitespace, so parsing of real direct references is unchanged.

@rajath201
rajath201 force-pushed the url-token-whitespace branch from 8c9c102 to fac695e Compare August 10, 2026 11:43

@r266-tech r266-tech left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a regression test for the reported injection shape, for example name @ https://host/file.whl\nevil==1, and assert that constructing Requirement raises InvalidRequirement. The current coverage only exercises trailing line breaks, so it does not directly preserve the security-relevant invariant that attacker-controlled text after embedded URL whitespace cannot be parsed and later serialized as another dependency line.

The checked-out head defines the URL tokenizer rule as r"\S+". The authority verification confirms that the checkout matches head fac695e96a4235f82bd4e67b875704025f3b6dc1, and git diff --check exited 0. The requested pytest checks were not run because the authority environment's system Python has no pytest and cannot create its cache file.

@rajath201

Copy link
Copy Markdown
Contributor Author

Added the injection-shape test: name @ https://example.com/name.whl followed by an embedded \n/\r/\r\n plus evil==1, asserting InvalidRequirement. Verified it fails against the old [^ \t]+ rule (all three variants) and passes with \S+. Full test_requirements suite is green.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants