Add Cisco ISE syslog parsing and OCSF mapping - #161
Open
zedoraps wants to merge 27 commits into
Open
Conversation
The cisco package now ingests Cisco Identity Services Engine (ISE) remote-target syslog and maps it to OCSF. cisco::ise::parse normalizes an ISE message body—header, code, severity, and the comma-separated attribute pairs—into a cisco.ise.* event, keeping attribute values verbatim so identifiers such as MAC addresses and session IDs survive type inference. cisco::ise::reassemble joins messages that ISE splits into numbered segments sharing a message id, windowing them on an event-time field. cisco::ise::ocsf::map turns Passed Authentications and Failed Attempts into OCSF Authentication, RADIUS Accounting into Network Activity, and every other category into a Base Event; orphan fragments are kept and labeled rather than dropped. Examples cover the single-message, multi-segment, and pub/sub flows. Tests exercise parsing, reassembly, orphan handling, and the OCSF mapping against anonymized sample data. Assisted-by: Claude Opus 4.8 (Claude Code)
The basic syslog example parsed ISE traffic without reassembly, so any message ISE split across segments lost its continuation segments (the parser drops un-reassembled continuation segments with a warning). ISE splits at the source whenever a message exceeds the syslog size limit, so a live feed always carries multi-segment messages. Fold reassembly—and the year inference its event-time window needs—into the canonical syslog example and drop the now-redundant separate reassemble example. Assisted-by: Claude Opus 4.8 (Claude Code)
Every other package mapper exposes a single `event` field argument. The ISE mapper carried an extra `logger_host` parameter; read the `hostname` field directly instead so the mapper matches the one-contract convention. Callers whose host arrives in another field rename it to `hostname` before mapping, as elsewhere in the library. Assisted-by: Claude Opus 4.8 (Claude Code)
The existing reassemble test covers one message with out-of-order segments. Add a case with two distinct messages whose segments arrive interleaved on the same stream, verifying reassembly keeps them apart by message id and orders each message's segments. The case is deterministic: it keys on event-time, not arrival, so it needs no real delay. Assisted-by: Claude Opus 4.8 (1M context) (Claude Code)
Mark reassembled Cisco ISE messages as incomplete when the expected segment set does not arrive before the window closes. This keeps truncated data queryable as OCSF Base Events instead of emitting misleading authentication or accounting events. Use a field-typed category argument for the parser and add regression coverage for missing continuation and missing header segments. Assisted-by: GPT-5 (Codex)
Keep the Cisco ISE package entry aligned with neighboring changelog entries and add the pull request reference. Assisted-by: GPT-5 (Codex)
Use accept_tcp for the live syslog example because the pipeline listens for incoming ISE remote-target connections. from_tcp is the client-side operator for connecting to a remote endpoint. Assisted-by: GPT-5 (Codex)
Consume mapped ISE attributes at their mapping sites so unmapped residue stays clean without a long final drop list. Keep small drops for aliases and temporary fields that cannot be moved unambiguously. Assisted-by: GPT-5 (Codex)
Assign single-use username aliases directly into the OCSF target with move. Keep alias drops only to remove the unchosen source field from unmapped residue. Assisted-by: GPT-5 (Codex)
Apply move-at-mapping-site style to the shared Cisco ISE OCSF dispatcher and base mapper, and keep parser-owned null timestamp sentinels out of unmapped residue. Assisted-by: GPT-5 (Codex)
Remove comments that restate nearby TQL and keep comments focused on Cisco-specific parsing, reassembly, mapping decisions, and test fixture intent. Assisted-by: GPT-5 (Codex)
Shorten the Cisco ISE changelog and example descriptions so they match the surrounding package style while keeping the PR reference and user-facing behavior summary. Assisted-by: GPT-5 (Codex)
Window Cisco ISE segments before grouping by message id so reassembly does not create one long-lived subpipeline per distinct message id. Assisted-by: GPT-5 (Codex)
Add a regression test for Cisco ISE segments with the same message id arriving in separate reassembly windows, ensuring they remain incomplete instead of being stitched into a false complete event. Assisted-by: GPT-5 (Codex)
Deduplicate repeated segments before payload assembly, deduplicate repeated network device groups, and add tests for delayed in-window segments and duplicate segment delivery. Assisted-by: GPT-5 (Codex)
Use canonical hopping windows for Cisco ISE reassembly so segments split by a fixed window boundary can still be assembled without creating per-message subpipelines or duplicate outputs. Add a beyond-span regression test for delayed segments that should remain incomplete. Assisted-by: GPT-5 (Codex)
Replace the first reassembly aggregation with deduplicate over message id and segment number, keeping duplicate segment suppression explicit while preserving the canonical hopping-window behavior. Assisted-by: GPT-5 (Codex)
Rename the canonical reassembly window variable to emit_window_start so the hopping-window output guard reads by purpose. Assisted-by: GPT-5 (Codex)
Use a payload-qualified reassembly key for single-segment Cisco ISE messages so reused message ids in the same span do not collapse distinct events. Add a regression test based on reused Passed Authentications ids. Assisted-by: GPT-5 (Codex)
Replace values derived from the Cisco ISE sample in the reused-message-id regression test with synthetic dates, ids, documentation IPs, and generic names. Assisted-by: GPT-5 (Codex)
Move SelectedAccessService into OCSF Authentication service.name and omit empty syslog wrapper fields from unmapped residue. Assisted-by: GPT-5 (Codex)
Make message-code fallback ranges include their boundary values and add a regression for category-less single- and multi-segment ISE messages. Assisted-by: GPT-5 (Codex)
Remove message-code family inference and cover the supported separate-category path instead. Category-less messages now parse as generic Cisco ISE events. Assisted-by: GPT-5 (Codex)
Reduce the default Cisco ISE reassembly span from 30s to 10s to lower live pipeline latency while preserving the configurable span for delayed segments. Assisted-by: GPT-5 (Codex)
Center Cisco ISE reassembly windows around segment 0 for multi-segment messages so continuations immediately before or after the header bucket assemble into one output. Add a regression for a continuation whose syslog timestamp falls just before the header segment across a fixed window boundary. Assisted-by: GPT-5 (Codex)
zedoraps
marked this pull request as ready for review
June 26, 2026 09:17
|
Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits. |
Keep Cisco ISE reassembly windows open for one span of event-time tolerance so a future timestamp does not close the canonical window before earlier segments arrive. Add a regression that documents the accepted duplicate-redelivery tradeoff while verifying that the complete reassembly is still emitted. Assisted-by: GPT-5 (Codex)
Move the temporary reassembled payload into the normalized message field so the temporary field is consumed atomically. Assisted-by: GPT-5 Codex (Superconductor)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🔍 Problem
The
ciscopackage could read Umbrella DNS but had no support for Cisco Identity Services Engine (ISE), whose remote-target syslog has an awkward shape:CISE_<Category>tag appears inline in the message body.🛠️ Solution
Add composable Cisco ISE UDOs under
operators/ise/, mirroring the existing Cisco package structure:cisco::ise::parsenormalizes one ISE message body intocisco.ise.*, recovers the category from the inline tag or a field argument, and keeps attribute values verbatim.cisco::ise::reassemblewindows segments by message id, orders payloads by segment number, and marks messages incomplete when the expected segment set is missing.cisco::ise::ocsf::mapmaps Passed/Failed Authentications to OCSF Authentication, RADIUS Accounting to Network Activity, and all other or incomplete ISE events to Base Event.Incomplete messages stay queryable as
cisco.ise.incomplete/ OCSF Base Event with truncation metadata instead of being silently dropped or emitted as misleading complete authentication/accounting events.💬 Review
- Review the reassembly behavior for missing segment 0, missing continuations, interleaved messages, and the event-time window defaults.
- Review the OCSF residue in
- Test coverage:
- Changelog validation:
📚 Docs PR: tenzir/docs#407unmapped, especially for incomplete messages.uvx tenzir-test ciscopasses locally with 18 tests.uvx tenzir-ship validatepasses from theciscopackage root.