feat: implement adversarial robustness demo

[Fidelisaboke]

User description

Description

This PR implements adversarial robustness demo for showcasing model robustness against adversarial evasion attacks

Related Issue

Closes #24
Closes #33

Type of Change

• 🐛 Bug fix
• ✨ New feature
• 🔧 Enhancement
• 📖 Documentation update

How Has This Been Tested?

• Unit tests
• Manual testing
• Other:

Screenshots (if applicable)

Checklist

• Code follows project style guidelines
• Documentation updated
• Tests added or updated
• No new warnings/errors

---

PR Type

Enhancement, Tests

---

Description

• Add adversarial robustness demo with FGSM attack simulation

• Implement flow_data column to alerts for raw network flow storage

• Create adversarial experiment report endpoint with metrics visualization

• Add robustness demo UI with live A/B/C testing interface

---

Diagram Walkthrough

flowchart LR
  A["Network Flow Input"] -->|"Prepare & Scale"| B["Data Preprocessing"]
  B -->|"Generate FGSM"| C["Adversarial Sample"]
  C -->|"Test"| D["Vulnerable Model"]
  C -->|"Test"| E["Robust Model"]
  D -->|"Results"| F["Demo Response"]
  E -->|"Results"| F
  G["Experiment JSON"] -->|"Parse & Transform"| H["Report Endpoint"]
  H -->|"Metrics"| I["Frontend Chart"]

Loading

File Walkthrough

	Relevant files
Enhancement	13 files useNids.ts Add robustness and adversarial experiment hooks                    +59/-0    alertsApi.ts Add flow_data field to Alert interface                                      +2/-0      nids.py Add robustness demo and report endpoints                                  +102/-1  models.py Add flow_data JSON column to Alert model                                  +3/-0      adversarial.py Implement FGSM adversarial sample generation                          +30/-0    loader.py Load vulnerable and surrogate models for demo                        +20/-3    predict.py Implement robustness demo experiment function                        +84/-3    adversarial.py Define adversarial experiment result schemas                          +48/-0    nids.py Add robustness demo and metrics response schemas                  +47/-4    page.tsx Display flow data with copy and workbench redirect              +91/-1    AdversarialReportChart.tsx Create bar chart for adversarial metrics visualization      +116/-0  page.tsx Add adversarial robustness report section to metrics          +40/-8    page.tsx Implement adversarial robustness demo UI with results        +396/-6
Configuration changes	1 files 8891f2a504bb_add_flow_data_column_to_alerts_table.py Database migration for flow_data column                                    +32/-0
Miscellaneous	1 files prune_alerts.py Remove unused database initialization code                              +0/-10
Bug fix	1 files page.tsx Fix status icon styling in alert cards                                      +3/-1
Formatting	1 files ThreatSummaryCard.tsx Adjust grid layout and spacing for threat summary                +2/-2
Additional files	1 files train.py [link]

---

The managed version of the open source project PR-Agent is sunsetting on the 1st December 2025. The commercial version of this project will remain available and free to use as a hosted service. Install Qodo.

[codiumai-pr-agent-free]

PR Compliance Guide 🔍

(Compliance updated until commit 286a156)

Below is a summary of compliance checks for this PR:

Security Compliance
⚪	Security control bypass Description: The file contains a hardcoded string that appears to be a secret (revision ID) with a pragma comment to allow it in security scans, which could bypass security controls. 8891f2a504bb_add_flow_data_column_to_alerts_table.py [15-15] Referred Code revision: str = '8891f2a504bb' # pragma: allowlist secret down_revision: Union[str, Sequence[str], None] = '2f4fc3d2e31b'
Ticket Compliance
🟢	🎫 #33 🟢 Visualize adversarial performance through metrics such as detection degradation and adversarial success rate under evasion attacks to communicate system resilience Add explainability for adversarial metrics
🟢	🎫 #24 🟢 Test trained models (both robust and baseline XGBoost) using adversarial samples at inference time (e.g., PGD, FGSM transfer attacks) Measure performance degradation under adversarial attacks
Codebase Duplication Compliance
⚪	Codebase context is not defined Follow the guide to enable codebase context checks.
Custom Compliance
🟢	Generic: Comprehensive Audit Trails Objective: To create a detailed and reliable record of critical system actions for security analysis and compliance. Status: Passed
	Generic: Meaningful Naming and Self-Documenting Code Objective: Ensure all identifiers clearly express their purpose and intent, making code self-documenting Status: Passed
	Generic: Robust Error Handling and Edge Case Management Objective: Ensure comprehensive error handling that provides meaningful context and graceful degradation Status: Passed
	Generic: Secure Error Handling Objective: To prevent the leakage of sensitive system information through error messages while providing sufficient detail for internal debugging. Status: Passed
	Generic: Secure Logging Practices Objective: To ensure logs are useful for debugging and auditing without exposing sensitive information like PII, PHI, or cardholder data. Status: Passed
	Generic: Security-First Input Validation and Data Handling Objective: Ensure all data inputs are validated, sanitized, and handled securely to prevent vulnerabilities Status: Passed

Compliance status legend

🟢 - Fully Compliant
🟡 - Partial Compliant
🔴 - Not Compliant
⚪ - Requires Further Human Verification
🏷️ - Compliance label

---

Previous compliance checks

Compliance check up to commit 9a0ee10

Security Compliance
⚪	False positive secret Description: Hardcoded string marked as a secret in a revision ID could potentially be mistaken for an actual secret by security scanners. 8891f2a504bb_add_flow_data_column_to_alerts_table.py [15-15] Referred Code revision: str = '8891f2a504bb' # pragma: allowlist secret down_revision: Union[str, Sequence[str], None] = '2f4fc3d2e31b'
Ticket Compliance
🟢	🎫 #33 🟢 Visualize adversarial performance through metrics such as detection degradation and adversarial success rate under evasion attacks to communicate system resilience Add explainability for adversarial metrics
🟢	🎫 #24 🟢 Test trained models (both robust and baseline XGBoost) using adversarial samples at inference time (e.g., PGD, transfer attacks) Measure performance degradation
Codebase Duplication Compliance
⚪	Codebase context is not defined Follow the guide to enable codebase context checks.
Custom Compliance
🟢	Generic: Comprehensive Audit Trails Objective: To create a detailed and reliable record of critical system actions for security analysis and compliance. Status: Passed
	Generic: Meaningful Naming and Self-Documenting Code Objective: Ensure all identifiers clearly express their purpose and intent, making code self-documenting Status: Passed
	Generic: Robust Error Handling and Edge Case Management Objective: Ensure comprehensive error handling that provides meaningful context and graceful degradation Status: Passed
	Generic: Secure Error Handling Objective: To prevent the leakage of sensitive system information through error messages while providing sufficient detail for internal debugging. Status: Passed
	Generic: Secure Logging Practices Objective: To ensure logs are useful for debugging and auditing without exposing sensitive information like PII, PHI, or cardholder data. Status: Passed
	Generic: Security-First Input Validation and Data Handling Objective: Ensure all data inputs are validated, sanitized, and handled securely to prevent vulnerabilities Status: Passed

The managed version of the open source project PR-Agent is sunsetting on the 1st December 2025. The commercial version of this project will remain available and free to use as a hosted service. Install Qodo.

[codiumai-pr-agent-free]

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Impact
High-level	Decouple application from static JSON The suggestion recommends replacing the direct reading of a static JSON file in the /robustness-report endpoint with a more robust system where the report data is seeded into the database during deployment. Examples: backend/api/routers/nids.py [400-461] @router.get( "/robustness-report", response_model=AdversarialExperimentResults, # This is the response schema dependencies=[Depends(require_permissions(SystemPermissions.VIEW_ALERTS))], status_code=status.HTTP_200_OK, ) def get_robustness_report(): """ Fetches the pre-calculated results from the adversarial training experiment notebook. """ ... (clipped 51 lines) Solution Walkthrough: Before: # in backend/api/routers/nids.py @router.get("/robustness-report") def get_robustness_report(): try: # Hardcoded path to a JSON file report_path = os.path.abspath( os.path.join( os.path.dirname(__file__), "../../ml/models/artifacts/adversarial_experiment.json", ) ) with open(report_path, "r") as f: raw_data = json.load(f) # Parse and transform data on every request ... return transformed_results except FileNotFoundError: raise HTTPException(status_code=404, detail="File not found.") After: # 1. New seeding script (e.g., backend/scripts/seed_report.py) def seed_robustness_report(): # Read from adversarial_experiment.json # Parse data # Create/update records in a new 'RobustnessReport' DB table db.session.commit() # 2. Updated endpoint in backend/api/routers/nids.py @router.get("/robustness-report") def get_robustness_report(db: Session = Depends(get_db)): # Query the database for the report data report_data = db.query(RobustnessReport).all() if not report_data: raise HTTPException(status_code=404, detail="Report data not found in DB.") # Format data from DB records into the response model return format_response_from_db(report_data) Suggestion importance[1-10]: 7 __ Why: The suggestion correctly identifies a fragile design where the /robustness-report endpoint depends on a static JSON file, proposing a more robust database-seeding approach which improves maintainability.	Medium
Possible issue	Prevent crashes from missing dictionary keys Add a check to ensure the hardcoded epsilon key exists in the adversarial experiment data before access, preventing potential server crashes. backend/api/routers/nids.py [429-450] +# Define the epsilon to use for the report +epsilon_key = "0.1" + +# Check if the required epsilon key exists in the results +if ( + epsilon_key not in parsed_data.baseline_results.epsilons + or epsilon_key not in parsed_data.robust_results.epsilons +): + app_logger.error(f"Epsilon '{epsilon_key}' not found in adversarial_experiment.json.") + raise HTTPException( + status_code=404, + detail=f"Adversarial experiment results for epsilon {epsilon_key} not found.", + ) + +baseline_adversarial_acc = parsed_data.baseline_results.epsilons[epsilon_key].fgsm.acc +robust_adversarial_acc = parsed_data.robust_results.epsilons[epsilon_key].fgsm.acc + # Frontend expects a flat list of metrics metrics_list = [ AdversarialMetric(model="Baseline (on Normal Data)", accuracy=parsed_data.baseline_clean_acc), AdversarialMetric( model="Baseline (on FGSM Attack)", - accuracy=parsed_data.baseline_results.epsilons["0.1"].fgsm.acc, # Example: using 0.1 epsilon + accuracy=baseline_adversarial_acc, ), AdversarialMetric( model="Robust Model (on FGSM Attack)", - accuracy=parsed_data.robust_results.epsilons["0.1"].fgsm.acc, # Example: using 0.1 epsilon + accuracy=robust_adversarial_acc, ), ] # Return the transformed results return AdversarialExperimentResults( - title="Adversarial Robustness (FGSM, Epsilon 0.1)", + title=f"Adversarial Robustness (FGSM, Epsilon {epsilon_key})", baseline_model_accuracy_normal=parsed_data.baseline_clean_acc, - baseline_model_accuracy_adversarial=parsed_data.baseline_results.epsilons["0.1"].fgsm.acc, + baseline_model_accuracy_adversarial=baseline_adversarial_acc, robust_model_accuracy_normal=parsed_data.robust_clean_acc, - robust_model_accuracy_adversarial=parsed_data.robust_results.epsilons["0.1"].fgsm.acc, + robust_model_accuracy_adversarial=robust_adversarial_acc, metrics=metrics_list, ) Apply / Chat Suggestion importance[1-10]: 7 __ Why: The suggestion correctly identifies that hardcoding the dictionary key "0.1" is brittle and could cause an unhandled KeyError, leading to a 500 error. Adding a check improves robustness and provides clearer error feedback.	Medium
Update

The managed version of the open source project PR-Agent is sunsetting on the 1st December 2025. The commercial version of this project will remain available and free to use as a hosted service. Install Qodo.