You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
@@ -337,275 +337,131 @@ The primary function of escaping in a dot-notated path is to accommodate the use
337
337
338
338
## Wildcards
339
339
340
-
In many scenarios, the output record closely resembles the input record, with only minor modifications required. When you deal with records that contain numerous fields, manually specifying mappings for each field can become tedious. Wildcards simplify this process by allowing for generalized mappings that can automatically apply to multiple fields.
340
+
Use a wildcard (`*`) in input and output paths to match multiple fields at once. This is useful when the output closely resembles the input, or when you need to apply the same transformation across many fields without listing each one.
341
341
342
-
Let's consider a basic scenario to understand the use of asterisks in mappings:
342
+
### Copy all fields
343
+
344
+
To pass every field through unchanged:
343
345
344
346
| Input | Output |
345
347
|-------|--------|
346
348
|`*`|`*`|
347
349
348
-
This configuration shows a basic mapping where every field in the input is directly mapped to the same field in the output without any changes. The asterisk (`*`) serves as a wildcard that matches any field in the input record.
350
+
The `*` matches each field path in the input and places it at the same path in the output. The portion of the path that `*` matches is called the **captured segment**. In the output, the captured segment replaces the `*`.
349
351
350
-
Here's how the asterisk (`*`) operates in this context:
352
+
### Flatten nested fields
351
353
352
-
***Pattern matching**: The asterisk can match a single segment or multiple segments of a path. It serves as a placeholder for any segments in the path.
353
-
***Field matching**: During the mapping process, the algorithm evaluates each field in the input record against the pattern specified in the `inputs`. The asterisk in the previous example matches all possible paths, effectively fitting every individual field in the input.
354
-
***Captured segment**: The portion of the path that the asterisk matches is referred to as the `captured segment`.
355
-
***Output mapping**: In the output configuration, the `captured segment` is placed where the asterisk appears. This means that the structure of the input is preserved in the output, with the `captured segment` filling the placeholder provided by the asterisk.
354
+
To move fields out of a nested object to the root level, put the prefix in the input and `*` in the output:
356
355
357
-
Another example illustrates how wildcards can be used to match subsections and move them together. This example effectively flattens nested structures within a JSON object.
When you place a wildcard, you must follow these rules:
383
+
To move fields under a new parent, put `*` in the input and add a prefix in the output:
403
384
404
-
***Single asterisk per data reference:** Only one asterisk (`*`) is allowed within a single data reference.
405
-
***Full segment matching:** The asterisk must always match an entire segment of the path. It can't be used to match only a part of a segment, such as `path1.partial*.path3`.
406
-
***Positioning:** The asterisk can be positioned in various parts of a data reference:
407
-
***At the beginning:**`*.path2.path3` - Here, the asterisk matches any segment that leads up to `path2.path3`.
408
-
***In the middle:**`path1.*.path3` - In this configuration, the asterisk matches any segment between `path1` and `path3`.
409
-
***At the end:**`path1.path2.*` - The asterisk at the end matches any segment that follows after `path1.path2`.
410
-
* The path containing the asterisk must be enclosed in single quotation marks (`'`).
385
+
| Input | Output |
386
+
|-------|--------|
387
+
|`*`|`Telemetry.*`|
411
388
412
-
### Multi-input wildcards
389
+
This wraps all top-level fields inside a `Telemetry` object.
413
390
414
-
Original JSON:
391
+
### Wildcard placement rules
415
392
416
-
```json
417
-
{
418
-
"Saturation": {
419
-
"Max": 0.42,
420
-
"Min": 0.67,
421
-
},
422
-
"Brightness": {
423
-
"Max": 0.78,
424
-
"Min": 0.93,
425
-
},
426
-
"Opacity": {
427
-
"Max": 0.88,
428
-
"Min": 0.91,
429
-
}
430
-
}
431
-
```
393
+
- Only **one**`*` is allowed per input or output path.
394
+
- The `*` must match a **complete segment** (not a partial segment like `Sensor*`).
395
+
- The `*` can appear at the beginning (`*.Value`), middle (`Sensors.*.Reading`), or end (`Sensors.*`) of a path.
432
396
433
-
Mapping configuration that uses wildcards:
397
+
### Multi-input wildcards
398
+
399
+
When a rule has multiple inputs with wildcards, the `*` must capture the **same segment** across all inputs. The runtime resolves the `*` from the first input, then looks for matching paths in the other inputs.
400
+
401
+
For example, to average the max and min readings for each sensor:
If you use multi-input wildcards, the asterisk (`*`) must consistently represent the same `Captured Segment` across every input. For example, when `*` captures `Saturation` in the pattern `*.Max`, the mapping algorithm expects the corresponding `Saturation.Min` to match with the pattern `*.Min`. Here, `*` is substituted by the `Captured Segment` from the first input, guiding the matching for subsequent inputs.
452
-
453
-
Consider this detailed example:
454
-
455
-
Original JSON:
416
+
The `*` captures `Temperature` first, so the rule looks for both `Temperature.Max` and `Temperature.Min`. Then it captures `Pressure` and looks for `Pressure.Max` and `Pressure.Min`. The output is:
This revised mapping accurately captures the necessary fields. It correctly specifies the paths to include the nested `Mid` object, which ensures that the asterisks work effectively across different levels of the JSON structure.
424
+
If any input can't resolve for a captured segment (for example, `*.Mid.Avg` when the field is nested differently), that segment is skipped. Make sure the paths in all inputs reflect the actual structure of the data.
510
425
511
-
### Specialization and second rules
426
+
### Override a wildcard for specific fields
512
427
513
-
When you use the previous example from multi-input wildcards, consider the following mappings that generate two derived values for each property:
428
+
You can combine a wildcard rule with specific rules. Specific rules take precedence when they have a **lower coverage** (fewer segments matched by `*`). This is called **specialization**.
This mapping is intended to create two separate calculations (`Avg` and `Diff`) for each property under `ColorProperties`. This example shows the result:
Here, the second mapping definition on the same inputs acts as a *second rule* for mapping.
435
+
The first rule applies to all fields. The second rule overrides it for `Pressure` only, because `Pressure.Max` is more specific than `*.Max` (coverage 0 vs. coverage 1).
542
436
543
-
Now, consider a scenario where a specific field needs a different calculation:
An empty output field in the second definition implies not writing the fields in the output record (effectively removing `Opacity`). This setup is more of a `Specialization` than a `Second Rule`.
* The evaluation progresses from the top rule in the mapping definition.
567
-
* If a new mapping resolves to the same fields as a previous rule, the following conditions apply:
568
-
* A `Rank` is calculated for each resolved input based on the number of segments the wildcard captures. For instance, if the `Captured Segments` are `Properties.Opacity`, the `Rank` is 2. If it's only `Opacity`, the `Rank` is 1. A mapping without wildcards has a `Rank` of 0.
569
-
* If the `Rank` of the latter rule is equal to or higher than the previous rule, a data flow treats it as a `Second Rule`.
570
-
* Otherwise, the data flow treats the configuration as a `Specialization`.
571
-
572
-
For example, the mapping that directs `Opacity.Max` and `Opacity.Min` to an empty output has a `Rank` of 0. Because the second rule has a lower `Rank` than the previous one, it's considered a specialization and overrides the previous rule, which would calculate a value for `Opacity`.
454
+
Both rules execute for each captured segment, producing two output fields per sensor.
573
455
574
456
### Wildcards in contextualization datasets
575
457
576
-
Contextualization datasets can be used with wildcards. Consider a dataset named `position` that contains the following record:
577
-
578
-
```json
579
-
{
580
-
"Position": "Analyst",
581
-
"BaseSalary": 70000,
582
-
"WorkingHours": "Regular"
583
-
}
584
-
```
585
-
586
-
In an earlier example, we used a specific field from this dataset:
This mapping copies `BaseSalary` from the context dataset directly into the `Employment` section of the output record. If you want to automate the process and include all fields from the `position` dataset into the `Employment` section, you can use wildcards:
458
+
You can use wildcards with `$context` references to copy all fields from a dataset:
593
459
594
460
| Input | Output |
595
461
|-------|--------|
596
-
|`$context(position).*`|`Employment.*`|
462
+
|`$context(assetMeta).*`|`Asset.*`|
597
463
598
-
This configuration allows for a dynamic mapping where every field within the `position` dataset is copied into the `Employment` section of the output record:
599
-
600
-
```json
601
-
{
602
-
"Employment": {
603
-
"Position": "Analyst",
604
-
"BaseSalary": 70000,
605
-
"WorkingHours": "Regular"
606
-
}
607
-
}
608
-
```
464
+
This copies every field from the `assetMeta` dataset into the `Asset` section of the output.
0 commit comments