feat(spp_pii_encryption): port PII encryption core from openspp-modules#232
feat(spp_pii_encryption): port PII encryption core from openspp-modules#232gonzalesedwin1123 wants to merge 3 commits into
Conversation
Migrate the field-level PII encryption module to OpenSPP2 (Odoo 19): - spp.encrypted.field.mixin: transparent AES-256-GCM encryption with HMAC-SHA256 blind indexes (exact/partial/phonetic) for searchability - spp.field.encryption.config: UI-based per-field encryption config - spp.pii.audit.log: audit trail of PII field access - masked_char OWL widget for masked display with reveal + audit Integrates with spp_key_management's key manager (get_key/get_salt). Depends on base, spp_key_management, spp_registry, spp_security. The bulk-migration wizard is intentionally deferred to a follow-up PR: it depends on spp_data_classification, which is not yet migrated to OpenSPP2. Realises ADR-011/ADR-012. Verified: 7/7 mixin tests pass, module installs in full UI stack, backend asset bundle compiles with the masked widget included.
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces the spp_pii_encryption module for Odoo, which implements field-level AES-256-GCM encryption for PII data, searchable blind indexes, UI-based configuration, and access audit logging. The review feedback highlights several critical and high-severity issues: a crash on module load due to the incorrect use of models.Constraint instead of _sql_constraints, stale blind indexes left in the database when encrypted fields are cleared during creation or modification, timezone inconsistencies from using datetime.now() instead of fields.Datetime.now(), and a misleading masking behavior for short values in the frontend widget.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| _unique_model_field = models.Constraint( | ||
| "UNIQUE(model_id, field_id)", | ||
| "Encryption configuration already exists for this model and field!", | ||
| ) |
There was a problem hiding this comment.
Odoo does not have a models.Constraint class. Defining a constraint this way will raise an AttributeError on module load, preventing the module from starting or installing. SQL constraints in Odoo must be defined using the _sql_constraints list of tuples.
_sql_constraints = [
(
"unique_model_field",
"UNIQUE(model_id, field_id)",
"Encryption configuration already exists for this model and field!",
)
]| def write(self, vals): | ||
| """Encrypt fields before write.""" | ||
| encrypted_fields = self._get_encrypted_fields() | ||
| if not encrypted_fields: | ||
| return super().write(vals) | ||
|
|
||
| for field_name in encrypted_fields: | ||
| if field_name in vals and vals[field_name]: | ||
| plaintext = vals[field_name] | ||
|
|
||
| # Encrypt | ||
| vals[field_name] = self._encrypt_value(plaintext, field_name) | ||
|
|
||
| # Get configured index type | ||
| index_type = self._get_index_type(field_name) | ||
|
|
||
| # Compute blind indexes if fields exist | ||
| index_field = f"{field_name}_index" | ||
| if index_field in self._fields: | ||
| vals[index_field] = self._compute_blind_index(plaintext, field_name, index_type) | ||
|
|
||
| last4_field = f"{field_name}_last4" | ||
| if last4_field in self._fields: | ||
| # SECURITY: Store hashed partial index, not plaintext | ||
| vals[last4_field] = self._compute_blind_index(plaintext, field_name, "partial") | ||
|
|
||
| return super().write(vals) |
There was a problem hiding this comment.
If an encrypted field is cleared (set to False or ""), the current implementation skips the encryption block entirely and does not update the corresponding blind index fields (_index and _last4). This leaves stale hashes in the database, allowing searches to find records that have actually had their PII cleared. We should explicitly clear the index fields when the main field is cleared.
def write(self, vals):
"""Encrypt fields before write."""
encrypted_fields = self._get_encrypted_fields()
if not encrypted_fields:
return super().write(vals)
for field_name in encrypted_fields:
if field_name in vals:
plaintext = vals[field_name]
if plaintext:
# Encrypt
vals[field_name] = self._encrypt_value(plaintext, field_name)
# Get configured index type
index_type = self._get_index_type(field_name)
# Compute blind indexes if fields exist
index_field = f"{field_name}_index"
if index_field in self._fields:
vals[index_field] = self._compute_blind_index(plaintext, field_name, index_type)
last4_field = f"{field_name}_last4"
if last4_field in self._fields:
# SECURITY: Store hashed partial index, not plaintext
vals[last4_field] = self._compute_blind_index(plaintext, field_name, "partial")
else:
# Clear indexes if the field is cleared
index_field = f"{field_name}_index"
if index_field in self._fields:
vals[index_field] = False
last4_field = f"{field_name}_last4"
if last4_field in self._fields:
vals[last4_field] = False
return super().write(vals)| @api.model_create_multi | ||
| def create(self, vals_list): | ||
| """Encrypt fields before create.""" | ||
| encrypted_fields = self._get_encrypted_fields() | ||
| if not encrypted_fields: | ||
| return super().create(vals_list) | ||
|
|
||
| for vals in vals_list: | ||
| for field_name in encrypted_fields: | ||
| if field_name in vals and vals[field_name]: | ||
| plaintext = vals[field_name] | ||
|
|
||
| # Encrypt | ||
| vals[field_name] = self._encrypt_value(plaintext, field_name) | ||
|
|
||
| # Get configured index type | ||
| index_type = self._get_index_type(field_name) | ||
|
|
||
| # Compute blind indexes if fields exist | ||
| index_field = f"{field_name}_index" | ||
| if index_field in self._fields: | ||
| vals[index_field] = self._compute_blind_index(plaintext, field_name, index_type) | ||
|
|
||
| last4_field = f"{field_name}_last4" | ||
| if last4_field in self._fields: | ||
| # SECURITY: Store hashed partial index, not plaintext | ||
| vals[last4_field] = self._compute_blind_index(plaintext, field_name, "partial") | ||
|
|
||
| return super().create(vals_list) |
There was a problem hiding this comment.
For consistency and safety, we should also handle the clearing of index fields during record creation if the field is explicitly passed as falsy or empty.
@api.model_create_multi
def create(self, vals_list):
"""Encrypt fields before create."""
encrypted_fields = self._get_encrypted_fields()
if not encrypted_fields:
return super().create(vals_list)
for vals in vals_list:
for field_name in encrypted_fields:
if field_name in vals:
plaintext = vals[field_name]
if plaintext:
# Encrypt
vals[field_name] = self._encrypt_value(plaintext, field_name)
# Get configured index type
index_type = self._get_index_type(field_name)
# Compute blind indexes if fields exist
index_field = f"{field_name}_index"
if index_field in self._fields:
vals[index_field] = self._compute_blind_index(plaintext, field_name, index_type)
last4_field = f"{field_name}_last4"
if last4_field in self._fields:
# SECURITY: Store hashed partial index, not plaintext
vals[last4_field] = self._compute_blind_index(plaintext, field_name, "partial")
else:
index_field = f"{field_name}_index"
if index_field in self._fields:
vals[index_field] = False
last4_field = f"{field_name}_last4"
if last4_field in self._fields:
vals[last4_field] = False
return super().create(vals_list)| from datetime import datetime, timedelta | ||
|
|
||
| if user_id is None: | ||
| user_id = self.env.user.id | ||
|
|
||
| cutoff = datetime.now() - timedelta(days=days) |
There was a problem hiding this comment.
Using datetime.now() retrieves the server's local time, which can lead to timezone inconsistencies and incorrect query results because Odoo stores datetime values in UTC in the database. Instead, use fields.Datetime.now() to get the current UTC datetime.
| from datetime import datetime, timedelta | |
| if user_id is None: | |
| user_id = self.env.user.id | |
| cutoff = datetime.now() - timedelta(days=days) | |
| from datetime import timedelta | |
| if user_id is None: | |
| user_id = self.env.user.id | |
| cutoff = fields.Datetime.now() - timedelta(days=days) |
| applyMask(value, pattern) { | ||
| if (!value || !pattern) return "****"; | ||
|
|
||
| // Pattern interpretation: | ||
| // * = mask this character | ||
| // # = show this character from the end | ||
| // Other chars = literal | ||
|
|
||
| const valueChars = value.split(""); | ||
| const patternChars = pattern.split(""); | ||
| let result = ""; | ||
| let valueIdx = 0; | ||
| const endIdx = value.length - 1; | ||
|
|
||
| // Count # in pattern to know how many chars to show from end | ||
| const hashCount = patternChars.filter((c) => c === "#").length; | ||
| let showFromEnd = hashCount; | ||
|
|
||
| for (const patternChar of patternChars) { | ||
| if (patternChar === "*") { | ||
| // Mask character | ||
| result += "•"; | ||
| valueIdx++; | ||
| } else if (patternChar === "#") { | ||
| // Show character from end | ||
| const showIdx = value.length - showFromEnd; | ||
| if (showIdx >= 0 && showIdx < value.length) { | ||
| result += value[showIdx]; | ||
| } | ||
| showFromEnd--; | ||
| } else { | ||
| // Literal character (like - or space) | ||
| result += patternChar; | ||
| } | ||
| } | ||
|
|
||
| return result || "••••••••"; | ||
| } |
There was a problem hiding this comment.
The current applyMask implementation pads the output with fake bullets to match the full pattern length even if the actual value is much shorter (e.g., a 3-character value like "123" gets masked as "••••-••••-123"). This is highly misleading as it makes short values look long. Additionally, valueIdx is incremented but never used. We should align the value with the pattern from right to left to respect the actual length of the value.
applyMask(value, pattern) {
if (!value || !pattern) return "****";
const valueChars = value.split("");
const patternChars = pattern.split("");
const resultChars = [];
let valIdx = valueChars.length - 1;
for (let i = patternChars.length - 1; i >= 0; i--) {
if (valIdx < 0) break;
const pChar = patternChars[i];
if (pChar === "#") {
resultChars.unshift(valueChars[valIdx]);
valIdx--;
} else if (pChar === "*") {
resultChars.unshift("•");
valIdx--;
} else {
resultChars.unshift(pChar);
if (valueChars[valIdx] === pChar) {
valIdx--;
}
}
}
while (valIdx >= 0) {
resultChars.unshift("•");
valIdx--;
}
return resultChars.join("") || "••••••••";
}
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## 19.0 #232 +/- ##
========================================
Coverage 73.05% 73.05%
========================================
Files 1069 1075 +6
Lines 62080 62353 +273
========================================
+ Hits 45351 45551 +200
- Misses 16729 16802 +73
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
- masked_field.js: remove unused vars (Component import, valueChars, endIdx, valueIdx, unused catch binding) flagged by eslint no-unused-vars - regenerate README.rst / index.html to match the manifest maintainers (oca-gen-addon-readme was failing on stale maintainer block)
…lpers Raise patch coverage on the migrated module: - test_field_encryption_config: get_encrypted_fields/get_field_config/ is_field_encrypted/get_index_type, index-field-name computation (index vs last4), toggle actions, and the char/text-only constraint - test_audit_log: log_field_access, get_access_history, get_user_access_history, display_name - test_encrypted_field_mixin: _get_encrypted_fields default, _get_index_type default, phonetic/passthrough normalization, graceful decrypt failure, and search helpers when no index column exists Note: the mixin's create/write/read ORM hooks can't be unit-tested in isolation (a tests/-defined concrete model isn't registered in this harness, same limitation spp_approval skips around). They are exercised by consumer modules that apply the mixin to real PII fields.
1 participant
What
Migrates the field-level PII encryption module from
openspp-modulesto OpenSPP2 (Odoo 19). This is PR1 of a planned sequence and ports the runtime encryption core; the bulk-migration wizard is deferred (see below).Realises ADR-011 (Data Classification) / ADR-012 (PII Encryption Strategy).
What's included
spp.encrypted.field.mixinspp.field.encryption.configspp.pii.audit.logmasked_char(OWL)Integrates with
spp_key_management's key manager (get_key/get_salt).Depends on:
base,spp_key_management,spp_registry,spp_security. External python:cryptography.What's intentionally deferred
The bulk-migration wizard (scan / dry-run / migrate / backup / rollback of existing plaintext) is not in this PR. Its scan step depends on
spp.field.classificationfromspp_data_classification, which is not yet migrated to OpenSPP2. Plan:spp_data_classificationAdaptations from the source module
category→OpenSPP/Configuration,website→ OpenSPP2,auto_install: False(admin opt-in tool), dependency list trimmed (droppedspp_data_classificationand the unusedspp_encryption).ValidationErrormessage in_()for translation.Verification
./spp t spp_pii_encryption→ 7/7 tests pass, module installsweb.assets_backendbundle compiles with the masked widget JS + SCSS includedNotes
string=params; oneexcept: passinaudit_log.py) were carried over unchanged from the source module.