Performance optimization opportunity: combinator.c file I/O bottleneck

[fireonthemountain]

The current combinator.c implementation has an O(n*m) file I/O bottleneck:

• File2 is completely re-read for every line in file1 (line 217: rewind(fd2))
• For a 10,000 × 1,000 line combination, this results in 10 million file reads instead of 1
• Performance degrades exponentially as file1 size increases

Performance Impact

Stress Test Results (10,000 × 1,000 lines = 10M combinations):

• Current implementation: Timeout (>60 seconds)
• Optimized implementation: <1 second
• Speedup: >60x improvement

Memory usage remains similar (~33MB for both versions)

Proposed Solution

I've created combinator_optimized.c that implements:

1. File2 Memory Caching: Load entire file2 once into memory, eliminating all rewind operations
2. Enhanced Error Handling: Added malloc() null pointer checks to prevent segfaults
3. Combined Line Processing: Single-pass carriage return stripping
4. Improved Buffer Management: More efficient I/O batching

[ronardcedeno31-coder]

Defenders usually treat high-volume combinatorial generation (like combinator-style wordlist expansion) as a behavioral signal of credential attack preparation, not a single exploit. The key is to detect it indirectly through process behavior, I/O patterns, and downstream authentication attempts.

Below is how mature SOCs approach it in two parts: simulation (to build detections) and detection (in real logs).

⸻

1. How defenders simulate combinatorial generation behavior

You don’t need hashcat itself—you simulate the behavioral footprint.

A. Synthetic “wordlist explosion” generator

Create a benign process that mimics:

• Reading two input wordlists (A × B)
• Producing a Cartesian product of combinations
• Writing output at high throughput

This can be done with:

• Python script
• Go/Rust CLI tool
• Log generator in a SIEM lab

Key simulation traits to match real tools:

• Sequential file reads
• High-rate stdout/file writes
• CPU-heavy string concatenation
• Minimal network activity

⸻

B. File-system behavior simulation

In a lab environment, generate:

• Large .txt file reads in tight loops
• High-rate creation of output files:
  • combo_00001.txt, combo_00002.txt, etc.
• Burst disk writes (I/O spikes)

This reproduces the same “fan-out generation pattern” seen in combinator-style tooling.

⸻

C. Process-tree simulation

Real attackers often run:

• wordlist generator → cracker → login attempts

So simulate a pipeline: