Skip to content

fix(parser): recover malformed glob fuzz inputs#1126

Open
ewhauser wants to merge 6 commits into
mainfrom
codex/fix-glob-fuzz-oom
Open

fix(parser): recover malformed glob fuzz inputs#1126
ewhauser wants to merge 6 commits into
mainfrom
codex/fix-glob-fuzz-oom

Conversation

@ewhauser

@ewhauser ewhauser commented Jun 9, 2026

Copy link
Copy Markdown
Owner

Summary

  • add raw parser recovery for malformed case and conditional glob contexts
  • avoid treating fuzzed heredoc-like text as real heredoc input during recovery
  • add a regression test for the failing glob_fuzz artifact

Verification

  • cargo test -p shuck-parser test_parse_glob_malformed_case_and_conditional_recovery_fuzz_regression_finishes
  • rustup run nightly cargo fuzz run --sanitizer=none glob_fuzz /private/tmp/shuck-fuzz-26741630915/oom-ea58f021be1f9e18d4d2d3af1518c9b97f025530 -- -runs=1 -rss_limit_mb=2048
  • cargo test -p shuck-parser

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b5f1f9517e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread crates/shuck-parser/src/parser/commands/conditionals.rs Outdated
Comment thread crates/shuck-parser/src/parser/commands/case.rs Outdated
Comment thread crates/shuck-parser/src/parser/commands/conditionals.rs Outdated
@github-actions

github-actions Bot commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

Macro CLI Benchmark Deltas

Compared a067ee0 against macro CLI baseline 1cf6571.

Positive deltas mean slower shuck check --no-cache runs.

  • Baseline source: rebuilt from PR base checkout
  • Baseline preparation: success
  • Comparison run: success
  • Workflow run: details

Aggregate

Benchmark Baseline Head Delta
all 832.58 ms +/- 11.02 ms 1.51 s +/- 20.41 ms +81.27%

Flagged aggregate regressions over +5.0%: 1

Per-fixture macro deltas
Benchmark Baseline Head Delta
all 832.58 ms +/- 11.02 ms 1.51 s +/- 20.41 ms +81.27%
bashtop 538.35 ms +/- 6.37 ms 1.17 s +/- 5.40 ms +117.50%
fzf-install 15.21 ms +/- 90.80 us 16.33 ms +/- 138.11 us +7.38%
homebrew-install 23.31 ms +/- 309.65 us 23.48 ms +/- 432.94 us +0.75%
nvm 159.59 ms +/- 1.15 ms 260.65 ms +/- 1.70 ms +63.33%
pyenv-python-build 92.22 ms +/- 871.04 us 106.73 ms +/- 335.04 us +15.73%
romkatv__powerlevel10k__internal__p10k.zsh 539.71 ms +/- 3.85 ms 536.63 ms +/- 3.04 ms -0.57%
ruby-build 42.28 ms +/- 191.80 us 47.45 ms +/- 270.11 us +12.23%
Largest changes

Regressions

Benchmark Baseline Head Delta
bashtop 538.35 ms +/- 6.37 ms 1.17 s +/- 5.40 ms +117.50%
all 832.58 ms +/- 11.02 ms 1.51 s +/- 20.41 ms +81.27%
nvm 159.59 ms +/- 1.15 ms 260.65 ms +/- 1.70 ms +63.33%
pyenv-python-build 92.22 ms +/- 871.04 us 106.73 ms +/- 335.04 us +15.73%
ruby-build 42.28 ms +/- 191.80 us 47.45 ms +/- 270.11 us +12.23%

Improvements

Benchmark Baseline Head Delta
romkatv__powerlevel10k__internal__p10k.zsh 539.71 ms +/- 3.85 ms 536.63 ms +/- 3.04 ms -0.57%
Non-zero benchmark exit codes
  • all baseline exit codes: 1,1,1,1,1,1,1,1,1,1; head exit codes: 1,1,1,1,1,1,1,1,1,1
  • bashtop baseline exit codes: 1,1,1,1,1,1,1,1,1,1; head exit codes: 1,1,1,1,1,1,1,1,1,1
  • fzf-install baseline exit codes: 1,1,1,1,1,1,1,1,1,1; head exit codes: 1,1,1,1,1,1,1,1,1,1
  • homebrew-install baseline exit codes: 1,1,1,1,1,1,1,1,1,1; head exit codes: 1,1,1,1,1,1,1,1,1,1
  • nvm baseline exit codes: 1,1,1,1,1,1,1,1,1,1; head exit codes: 1,1,1,1,1,1,1,1,1,1
  • pyenv-python-build baseline exit codes: 1,1,1,1,1,1,1,1,1,1; head exit codes: 1,1,1,1,1,1,1,1,1,1
  • romkatv__powerlevel10k__internal__p10k.zsh baseline exit codes: 1,1,1,1,1,1,1,1,1,1; head exit codes: 1,1,1,1,1,1,1,1,1,1
  • ruby-build baseline exit codes: 1,1,1,1,1,1,1,1,1,1; head exit codes: 1,1,1,1,1,1,1,1,1,1

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bcd4bfd3d9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread crates/shuck-parser/src/parser/cursor.rs Outdated
Comment thread crates/shuck-parser/src/parser/cursor.rs Outdated
Comment thread crates/shuck-parser/src/parser/cursor.rs Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a97cc3d8e6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread crates/shuck-parser/src/parser/cursor.rs Outdated
Comment thread crates/shuck-parser/src/parser/cursor.rs

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a60f94a815

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread crates/shuck-parser/src/parser/cursor.rs Outdated
Comment thread crates/shuck-parser/src/parser/cursor.rs Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 125f2e2615

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread crates/shuck-parser/src/parser/cursor.rs Outdated
Comment thread crates/shuck-parser/src/parser/cursor.rs
Comment thread crates/shuck-parser/src/parser/cursor.rs

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a067ee09f8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +485 to +489
if self.subst_paren_depth > 0 {
match ch {
'(' => self.subst_paren_depth += 1,
')' => self.subst_paren_depth = self.subst_paren_depth.saturating_sub(1),
_ => {}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep comment state inside command substitutions

When recovery scans a malformed conditional that enters $(), this branch decrements subst_paren_depth for every ) without recognizing comments inside the substitution. For inputs such as [[ x =~ { $(echo # )\n]]) ]], the ) on the commented line is not the substitution close, but the scanner drops back to top level and accepts the following ]] from inside the substitution as the conditional terminator, so recovery resumes in the middle of $() instead of after the real close.

Useful? React with 👍 / 👎.

Comment on lines +145 to +149
if state.is_plain_top_level()
&& state.at_token_start
&& raw_token_ends_at(self.input, offset, "]]")
{
return Some(offset + "]]".len());

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Track process substitutions during raw recovery

When a malformed [[ ... ]] recovery scan starts before a process substitution, the raw state never enters a nested context for <(...) or >(...), so a token-start ]] inside that substitution is accepted here as the conditional close. For example, [[ x =~ { <(echo ]]) ]] makes recovery resume before the process substitution's ) instead of after the real ]], leaving the rest to be parsed as unrelated syntax.

Useful? React with 👍 / 👎.

Comment on lines +179 to +181
if raw_keyword_starts_at(self.input, offset, "esac") {
if state.case_depth == 0 {
return Some(offset + "esac".len());

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Ignore heredoc bodies when finding case ends

When malformed case recovery scans a body that starts a heredoc, this raw keyword check still treats an esac line inside the heredoc payload as the compound terminator. For example, after a bad arm followed by cat <<EOF\nesac\nEOF\nesac, recovery stops at the heredoc content instead of the real esac, so the lexer resumes from the middle of the heredoc and parses the remaining delimiter/body as top-level text.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant