diff --git a/documentation/docs/admin/security/data_encryption.md b/documentation/docs/admin/security/data_encryption.md index 5a4701368df..a4ffbe7486f 100644 --- a/documentation/docs/admin/security/data_encryption.md +++ b/documentation/docs/admin/security/data_encryption.md @@ -12,17 +12,33 @@ For enhanced security control, PMM supports custom encryption keys. **Key format requirements:** -- The key must be a 32-byte (256-bit) random value, suitable for AES-256-GCM encryption. -- The file must contain exactly 32 raw bytes (not a hex-encoded or base64-encoded string). +The key file must contain a base64-encoded Tink keyset created from the `AES256GCMKeyTemplate`. This is not a raw 32-byte value, so a key produced with a general-purpose tool such as `openssl rand` cannot be used: PMM fails to start if it cannot parse the keyset. +Generate a key in the correct format with the Encryption Rotation Tool, which prints a new key to stdout without touching the database: -PMM uses this key with the TINK `AES256GCMKeyTemplate` output prefix type. +```bash +pmm-encryption-rotation --generate-key +``` -To set up a custom key, configure the `PMM_ENCRYPTION_KEY_PATH` environment variable to point to your custom key file. +To set up a custom key, write that value to a file and point the `PMM_ENCRYPTION_KEY_PATH` environment variable at it. !!! hint alert alert-success "Important" Configure this **before** any data encryption occurs: either before upgrading to PMM 3 or before initially starting a new PMM 3.x instance. +### High availability deployments + +All PMM Server nodes in a [highly available deployment](../../install-pmm/install-HA-clustered.md) share one PostgreSQL database, but each node reads its encryption key from its own local file. Every node must therefore use the **same** encryption key. + +A node holding a different key cannot decrypt credentials written by the other nodes. PMM detects this and refuses to start the affected node, because otherwise it would hand unusable credentials to PMM Clients and monitoring would stop for the affected services. + +Generate the key once, place it on every node before starting them, and back it up with the rest of your cluster configuration: + +```bash +pmm-encryption-rotation --generate-key > pmm-encryption.key +``` + +To rotate the key in an HA cluster, run the [rotation procedure](#rotating-the-encryption-key) on a single node, then copy the resulting key file to all the other nodes and restart them. + ### Key management requirements Once configured, PMM will use the custom key to encrypt and decrypt all sensitive data stored within the system. diff --git a/documentation/docs/install-pmm/install-HA-clustered.md b/documentation/docs/install-pmm/install-HA-clustered.md index 704fe512ed2..bcd7748267d 100644 --- a/documentation/docs/install-pmm/install-HA-clustered.md +++ b/documentation/docs/install-pmm/install-HA-clustered.md @@ -617,6 +617,35 @@ certs: -----END DH PARAMETERS----- ``` +### Manage the encryption key + +PMM encrypts the credentials it stores for monitored services. All replicas share one PostgreSQL database, so they must all use the **same** encryption key: a replica holding a different key cannot decrypt credentials written by the others, and the affected services stop being monitored. + +The chart handles this for you. On installation it generates one key, stores it in a Kubernetes secret named `pg-encryption-key`, and mounts it into every replica at the path given by `PMM_ENCRYPTION_KEY_PATH`. Upgrades and rescaling reuse the existing key, so you do not need to configure anything. + +Two things follow from this: + +- **Back up the secret.** It is the only copy of the key. Without it, the credentials in a restored database cannot be decrypted: + + ```sh + kubectl get secret pg-encryption-key -n pmm -o yaml > pg-encryption-key-backup.yaml + ``` + +- **Keep the secret when reinstalling against existing data.** The secret is not owned by the Helm release and survives `helm uninstall`. If you delete it but keep the PostgreSQL data, a fresh installation generates a new key that cannot read the existing rows. Restore the backup before reinstalling: + + ```sh + kubectl apply -f pg-encryption-key-backup.yaml + ``` + +To supply your own key instead, create the secret before installing the chart: + +```sh +kubectl create secret generic pg-encryption-key -n pmm \ + --from-literal=key="$(pmm-encryption-rotation --generate-key)" +``` + +See [PMM data encryption](../admin/security/data_encryption.md) for the key format and rotation. + ### Configure storage PMM HA stores data in distributed databases, not on the PMM server pods themselves. To increase storage capacity, configure the ClickHouse and VictoriaMetrics clusters. diff --git a/documentation/docs/install-pmm/install-pmm-server/deployment-options/docker/preview_env_var.md b/documentation/docs/install-pmm/install-pmm-server/deployment-options/docker/preview_env_var.md index a776bbcd417..69bb96abd7f 100644 --- a/documentation/docs/install-pmm/install-pmm-server/deployment-options/docker/preview_env_var.md +++ b/documentation/docs/install-pmm/install-pmm-server/deployment-options/docker/preview_env_var.md @@ -14,6 +14,13 @@ | `PMM_HA_GRAFANA_GOSSIP_PORT` | HA Grafana gossip port. | `PMM_HA_PEERS` | HA Peers. +!!! caution alert alert-warning "All HA nodes must share one encryption key" + HA nodes share one PostgreSQL database, but each node reads its encryption key from its own `/srv/pmm-encryption.key`. A node that generated its own key cannot decrypt the credentials stored by the other nodes, and the services those credentials belong to stop being monitored. + + Generate the key once with `pmm-encryption-rotation --generate-key`, place it at `/srv/pmm-encryption.key` on every node, and only then start them. With `PMM_HA_ENABLE` set, a node without a key file refuses to start rather than generating one of its own. + + See [PMM data encryption](../../../../admin/security/data_encryption.md) for details. + ## Available preview variables | Variable | Description diff --git a/managed/cmd/pmm-managed-init/main.go b/managed/cmd/pmm-managed-init/main.go index 8cff42b19d5..111056b0296 100644 --- a/managed/cmd/pmm-managed-init/main.go +++ b/managed/cmd/pmm-managed-init/main.go @@ -16,6 +16,7 @@ package main import ( + "fmt" "os" "strconv" @@ -24,6 +25,7 @@ import ( "github.com/percona/pmm/managed/models" "github.com/percona/pmm/managed/services/clickhouse" "github.com/percona/pmm/managed/services/supervisord" + "github.com/percona/pmm/managed/utils/encryption" "github.com/percona/pmm/managed/utils/env" "github.com/percona/pmm/managed/utils/envvars" "github.com/percona/pmm/utils/logger" @@ -67,6 +69,12 @@ func main() { isHAEnabled, _ := strconv.ParseBool(os.Getenv("PMM_HA_ENABLE")) if isHAEnabled { pmmConfigParams["AgentConfigFilePath"] = "/srv/pmm-agent/config/pmm-agent.yaml" + + err = checkHAEncryptionKey() + if err != nil { + logrus.Errorf("Configuration error: %s", err) + os.Exit(1) + } } err = supervisord.SavePMMConfig(pmmConfigParams) @@ -75,3 +83,24 @@ func main() { os.Exit(1) } } + +// checkHAEncryptionKey refuses to start an HA node that has no encryption key yet. +// +// All nodes of an HA cluster share one PostgreSQL database but keep their own key file, so +// letting a node generate its own key leaves it unable to decrypt rows written by the others. +func checkHAEncryptionKey() error { + path := encryption.KeyPath() + + _, err := os.Stat(path) + switch { + case err == nil: + return nil + case os.IsNotExist(err): + return fmt.Errorf("encryption key %s not found. In HA mode all PMM Server nodes must share "+ + "one encryption key, so it is never generated automatically. Generate it once with "+ + "`pmm-encryption-rotation --generate-key`, place the output at %s on every node, then start them", + path, path) + default: + return fmt.Errorf("cannot read encryption key %s: %w", path, err) + } +} diff --git a/managed/cmd/pmm-managed-init/main_test.go b/managed/cmd/pmm-managed-init/main_test.go new file mode 100644 index 00000000000..fc1db3e54ee --- /dev/null +++ b/managed/cmd/pmm-managed-init/main_test.go @@ -0,0 +1,47 @@ +// Copyright (C) 2023 Percona LLC +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . + +package main + +import ( + "os" + "path/filepath" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "github.com/percona/pmm/managed/utils/encryption" +) + +func TestCheckHAEncryptionKey(t *testing.T) { + t.Run("missing key is rejected", func(t *testing.T) { + path := filepath.Join(t.TempDir(), "pmm-encryption.key") + t.Setenv(encryption.CustomEncryptionKeyPathEnvVar, path) + + err := checkHAEncryptionKey() + require.Error(t, err) + assert.Contains(t, err.Error(), path) + assert.Contains(t, err.Error(), "--generate-key") + }) + + t.Run("existing key is accepted", func(t *testing.T) { + path := filepath.Join(t.TempDir(), "pmm-encryption.key") + require.NoError(t, os.WriteFile(path, []byte("key"), 0o600)) + t.Setenv(encryption.CustomEncryptionKeyPathEnvVar, path) + + assert.NoError(t, checkHAEncryptionKey()) + }) +} diff --git a/managed/cmd/pmm-managed/main.go b/managed/cmd/pmm-managed/main.go index 2468e9778a0..8d4d373db9d 100644 --- a/managed/cmd/pmm-managed/main.go +++ b/managed/cmd/pmm-managed/main.go @@ -107,6 +107,7 @@ import ( "github.com/percona/pmm/managed/services/vmalert" "github.com/percona/pmm/managed/utils/clean" "github.com/percona/pmm/managed/utils/distribution" + "github.com/percona/pmm/managed/utils/encryption" "github.com/percona/pmm/managed/utils/envvars" "github.com/percona/pmm/managed/utils/interceptors" platformClient "github.com/percona/pmm/managed/utils/platform" @@ -146,6 +147,14 @@ const ( var pprofSemaphore = semaphore.NewWeighted(1) +// mEncryptionKeyMismatch makes the condition visible to monitoring, since standalone PMM keeps +// running with a mismatched key. +var mEncryptionKeyMismatch = prom.NewGauge(prom.GaugeOpts{ + Namespace: "pmm_managed", + Name: "encryption_key_mismatch", + Help: "1 if the local encryption key does not match the key the database was encrypted with, 0 otherwise.", +}) + func addLogsHandler(mux *http.ServeMux, logs *server.Logs) { l := logrus.WithField("component", "logs.zip") @@ -652,6 +661,30 @@ func migrateDB(ctx context.Context, sqlDB *sql.DB, params models.SetupDBParams) } } +// verifyEncryptionKey checks that this node holds the encryption key the database was encrypted +// with. +// +// A mismatch is fatal in HA, where the node would otherwise write rows its peers cannot read. A +// standalone node only logs it, so that an upgrade cannot turn an installation whose key went +// missing into one that no longer boots. +func verifyEncryptionKey(l *logrus.Entry, db *reform.DB, haEnabled bool) { + err := models.VerifyEncryptionKey(db) + if err == nil { + return + } + if !errors.Is(err, models.ErrEncryptionKeyMismatch) { + l.Panicf("Failed to verify encryption key: %+v", err) + } + + mEncryptionKeyMismatch.Set(1) + if haEnabled { + l.Fatalf("%s. Every PMM Server node in an HA cluster must use the same encryption key: "+ + "copy %s from a node that works and restart this one.", err, encryption.KeyPath()) + } + l.Errorf("%s. Stored credentials cannot be decrypted, so monitoring will not work until the "+ + "matching key is restored to %s.", err, encryption.KeyPath()) +} + // newClickhouseDB return a new Clickhouse db. func newClickhouseDB(dsn string, maxIdleConns, maxOpenConns int) (*sql.DB, error) { db, err := sql.Open("clickhouse", dsn) @@ -900,6 +933,9 @@ func main() { //nolint:gocognit,maintidx,cyclop prom.MustRegister(reformL) db := reform.NewDB(sqlDB, postgresql.Dialect, reformL) + prom.MustRegister(mEncryptionKeyMismatch) + verifyEncryptionKey(l, db, *haEnabled) + // Generate unique PMM Server ID if it's not already. err = models.SetPMMServerID(db) if err != nil { diff --git a/managed/models/agent_helpers.go b/managed/models/agent_helpers.go index 20da2a24104..b2ae2f26dca 100644 --- a/managed/models/agent_helpers.go +++ b/managed/models/agent_helpers.go @@ -232,6 +232,40 @@ type AgentFilters struct { Disabled *bool } +// decryptAgents decrypts Agent rows as returned by reform. +func decryptAgents(structs []reform.Struct) ([]*Agent, error) { + agents := make([]*Agent, len(structs)) + for i, s := range structs { + decryptedAgent, err := DecryptAgent(*s.(*Agent)) //nolint:forcetypeassert + if err != nil { + return nil, err + } + agents[i] = &decryptedAgent + } + + return agents, nil +} + +// insertAgent encrypts the Agent, inserts it and returns it decrypted again. +func insertAgent(q *reform.Querier, agent Agent) (*Agent, error) { + encryptedAgent, err := EncryptAgent(agent) + if err != nil { + return nil, err + } + + err = q.Insert(&encryptedAgent) + if err != nil { + return nil, err + } + + decryptedAgent, err := DecryptAgent(encryptedAgent) + if err != nil { + return nil, err + } + + return &decryptedAgent, nil +} + // FindAgents returns Agents by filters. func FindAgents(q *reform.Querier, filters AgentFilters) ([]*Agent, error) { var conditions []string @@ -294,13 +328,7 @@ func FindAgents(q *reform.Querier, filters AgentFilters) ([]*Agent, error) { return nil, err } - agents := make([]*Agent, len(structs)) - for i, s := range structs { - decryptedAgent := DecryptAgent(*s.(*Agent)) //nolint:forcetypeassert - agents[i] = &decryptedAgent - } - - return agents, nil + return decryptAgents(structs) } // FindAgentByID finds Agent by ID. @@ -317,7 +345,13 @@ func FindAgentByID(q *reform.Querier, id string) (*Agent, error) { } return nil, err } - return new(DecryptAgent(*agent)), nil + + decryptedAgent, err := DecryptAgent(*agent) + if err != nil { + return nil, err + } + + return new(decryptedAgent), nil } // FindAgentsByIDs finds Agents by IDs. @@ -337,12 +371,7 @@ func FindAgentsByIDs(q *reform.Querier, ids []string) ([]*Agent, error) { return nil, err } - res := make([]*Agent, len(structs)) - for i, s := range structs { - decryptedAgent := DecryptAgent(*s.(*Agent)) //nolint:forcetypeassert - res[i] = &decryptedAgent - } - return res, nil + return decryptAgents(structs) } // FindDBConfigForService find DB config from agents running on service specified by serviceID. @@ -390,10 +419,9 @@ func FindDBConfigForService(q *reform.Querier, serviceID string) (*DBConfig, err return nil, err } - res := make([]*Agent, len(structs)) - for i, s := range structs { - decryptedAgent := DecryptAgent(*s.(*Agent)) //nolint:forcetypeassert - res[i] = &decryptedAgent + res, err := decryptAgents(structs) + if err != nil { + return nil, err } if len(res) == 0 { @@ -418,13 +446,7 @@ func FindPMMAgentsRunningOnNode(q *reform.Querier, nodeID string) ([]*Agent, err return nil, status.Errorf(codes.FailedPrecondition, "Couldn't get agents by runs_on_node_id, %s", nodeID) } - res := make([]*Agent, 0, len(structs)) - for _, str := range structs { - decryptedAgent := DecryptAgent(*str.(*Agent)) //nolint:forcetypeassert - res = append(res, &decryptedAgent) - } - - return res, nil + return decryptAgents(structs) } // FindPMMAgentsForService gets pmm-agents for service. @@ -463,13 +485,7 @@ func FindPMMAgentsForService(q *reform.Querier, serviceID string) ([]*Agent, err if err != nil { return nil, status.Errorf(codes.FailedPrecondition, "Couldn't get pmm-agents for service %s", serviceID) } - res := make([]*Agent, 0, len(pmmAgentRecords)) - for _, str := range pmmAgentRecords { - decryptedAgent := DecryptAgent(*str.(*Agent)) //nolint:forcetypeassert - res = append(res, &decryptedAgent) - } - - return res, nil + return decryptAgents(pmmAgentRecords) } // FindPMMAgentsForServicesOnNode gets pmm-agents for Services running on Node. @@ -545,12 +561,7 @@ func FindAgentsForScrapeConfig(q *reform.Querier, pmmAgentID *string, pushMetric return nil, err } - res := make([]*Agent, len(allAgents)) - for i, s := range allAgents { - decryptedAgent := DecryptAgent(*s.(*Agent)) //nolint:forcetypeassert - res[i] = &decryptedAgent - } - return res, nil + return decryptAgents(allAgents) } // FindAllPMMAgentsIDs returns pmm-agents-ids with agents. @@ -592,7 +603,12 @@ func FindPmmAgentIDToRunActionOrJob(pmmAgentID string, agents []*Agent) (string, // UpdateAgent updates the Agent in the database. func UpdateAgent(q *reform.Querier, agent *Agent) error { - err := q.Update(new(EncryptAgent(*agent))) + encryptedAgent, err := EncryptAgent(*agent) + if err != nil { + return err + } + + err = q.Update(new(encryptedAgent)) if err != nil { return fmt.Errorf("failed to update Agent: %w", err) } @@ -706,12 +722,7 @@ func CreateNodeExporter(q *reform.Querier, return nil, err } - encryptedAgent := EncryptAgent(*row) - err = q.Insert(&encryptedAgent) - if err != nil { - return nil, err - } - return new(DecryptAgent(encryptedAgent)), nil + return insertAgent(q, *row) } // CreateExternalExporterParams params for add external exporter. @@ -796,12 +807,7 @@ func CreateExternalExporter(q *reform.Querier, params *CreateExternalExporterPar return nil, err } - encryptedAgent := EncryptAgent(*row) - err = q.Insert(&encryptedAgent) - if err != nil { - return nil, err - } - return new(DecryptAgent(encryptedAgent)), nil + return insertAgent(q, *row) } // CreateAgentParams params for add common exporter. @@ -998,12 +1004,7 @@ func CreateAgent(q *reform.Querier, agentType AgentType, params *CreateAgentPara // do nothing } - encryptedAgent := EncryptAgent(trimUnicodeNilsInCertFiles(*row)) - err = q.Insert(&encryptedAgent) - if err != nil { - return nil, err - } - return new(DecryptAgent(encryptedAgent)), nil + return insertAgent(q, trimUnicodeNilsInCertFiles(*row)) } func trimUnicodeNilsInCertFiles(agent Agent) Agent { @@ -1431,13 +1432,23 @@ func ChangeAgent(q *reform.Querier, agentID string, params *ChangeAgentParams) ( row.RTAOptions.Merge(params.RTAOptions) // need to encrypt Agent's sensitive data before update - row = new(EncryptAgent(*row)) + encryptedAgent, err := EncryptAgent(*row) + if err != nil { + return nil, err + } + + row = new(encryptedAgent) err = q.Update(row) if err != nil { return nil, err } - return new(DecryptAgent(*row)), nil + decryptedAgent, err := DecryptAgent(*row) + if err != nil { + return nil, err + } + + return new(decryptedAgent), nil } // RemoveAgent removes Agent by ID. diff --git a/managed/models/agent_helpers_test.go b/managed/models/agent_helpers_test.go index 462b0420790..38e54dec066 100644 --- a/managed/models/agent_helpers_test.go +++ b/managed/models/agent_helpers_test.go @@ -205,7 +205,9 @@ func TestAgentHelpers(t *testing.T) { }, } { if v, ok := str.(*models.Agent); ok { - str = new(models.EncryptAgent(*v)) + encrypted, err := models.EncryptAgent(*v) + require.NoError(t, err) + str = new(encrypted) } require.NoError(t, q.Insert(str)) } @@ -926,7 +928,9 @@ func TestAgentHelpers(t *testing.T) { CreatedAt: now, UpdatedAt: now, } - err := q.Insert(awsAgent) + encryptedAgent, err := models.EncryptAgent(*awsAgent) + require.NoError(t, err) + err = q.Insert(&encryptedAgent) require.NoError(t, err) // Test changing AWS options @@ -973,7 +977,9 @@ func TestAgentHelpers(t *testing.T) { CreatedAt: now, UpdatedAt: now, } - err := q.Insert(mysqlAgent) + encryptedAgent, err := models.EncryptAgent(*mysqlAgent) + require.NoError(t, err) + err = q.Insert(&encryptedAgent) require.NoError(t, err) // Test changing MySQL options @@ -1129,7 +1135,9 @@ func TestAgentHelpers(t *testing.T) { CreatedAt: now, UpdatedAt: now, } - err := q.Insert(azureAgent) + encryptedAgent, err := models.EncryptAgent(*azureAgent) + require.NoError(t, err) + err = q.Insert(&encryptedAgent) require.NoError(t, err) // Test changing Azure options diff --git a/managed/models/agent_model_test.go b/managed/models/agent_model_test.go index c5b012975ba..eb1a2f7d819 100644 --- a/managed/models/agent_model_test.go +++ b/managed/models/agent_model_test.go @@ -588,7 +588,9 @@ func TestExporterURL(t *testing.T) { }, } { if v, ok := str.(*models.Agent); ok { - str = new(models.EncryptAgent(*v)) + encrypted, err := models.EncryptAgent(*v) + require.NoError(t, err) + str = new(encrypted) } require.NoError(t, q.Insert(str), "failed to INSERT %+v", str) } diff --git a/managed/models/database.go b/managed/models/database.go index 7a3895d5aab..b26eee04cf6 100644 --- a/managed/models/database.go +++ b/managed/models/database.go @@ -1356,13 +1356,23 @@ func dbEncryption(tx *reform.TX, database string, items []encryption.Table, return err } + // The fingerprint is recorded in the same transaction as the encrypted column list, so a + // concurrently starting node cannot observe encrypted data with no fingerprint to check its + // own key against. encryptedItems := []string{} + fingerprint := "" if expectedState { encryptedItems = prepared + + fingerprint, err = encryption.Fingerprint() + if err != nil { + return err + } } _, err = UpdateSettings(tx, &ChangeSettingsParams{ - EncryptedItems: encryptedItems, + EncryptedItems: encryptedItems, + EncryptionKeyFingerprint: &fingerprint, }) if err != nil { return err diff --git a/managed/models/dsn_helpers_test.go b/managed/models/dsn_helpers_test.go index 7fed57d57d7..71f4d72e21e 100644 --- a/managed/models/dsn_helpers_test.go +++ b/managed/models/dsn_helpers_test.go @@ -163,7 +163,9 @@ func TestFindDSNByServiceID(t *testing.T) { }, } { if v, ok := str.(*models.Agent); ok { - str = new(models.EncryptAgent(*v)) + encrypted, err := models.EncryptAgent(*v) + require.NoError(t, err) + str = new(encrypted) } require.NoError(t, q.Insert(str)) } diff --git a/managed/models/encryption_helpers.go b/managed/models/encryption_helpers.go index ba9aea666a0..a24c3a14c31 100644 --- a/managed/models/encryption_helpers.go +++ b/managed/models/encryption_helpers.go @@ -18,113 +18,171 @@ package models import ( "database/sql" "encoding/json" + "errors" + "fmt" + "slices" + "strings" - "github.com/sirupsen/logrus" + "gopkg.in/reform.v1" "github.com/percona/pmm/managed/utils/encryption" ) // EncryptAgent encrypt agent. -func EncryptAgent(agent Agent) Agent { +func EncryptAgent(agent Agent) (Agent, error) { return agentEncryption(agent, encryption.Encrypt) } // DecryptAgent decrypt agent. -func DecryptAgent(agent Agent) Agent { +// On error the returned Agent is only partially decrypted and must not be used: its remaining +// fields still hold ciphertext. +func DecryptAgent(agent Agent) (Agent, error) { return agentEncryption(agent, encryption.Decrypt) } -func agentEncryption(agent Agent, handler func(string) (string, error)) Agent { //nolint:gocognit - if agent.Username != nil { - username, err := handler(*agent.Username) - if err != nil { - logrus.Warning(err) - } - agent.Username = &username +// ErrEncryptionKeyMismatch is returned when this node's encryption key is not the key the data +// in the database was encrypted with. +var ErrEncryptionKeyMismatch = errors.New("encryption key does not match the database") + +// VerifyEncryptionKey reports whether this node holds the encryption key the database was +// encrypted with, and records the key's fingerprint when none is stored yet. +// +// Every node of an HA cluster shares one database but keeps its own key file, so a node that +// generated its own key cannot decrypt the credentials written by the others. +func VerifyEncryptionKey(q reform.DBTX) error { + fingerprint, err := encryption.Fingerprint() + if err != nil { + return err } - if agent.Password != nil { - password, err := handler(*agent.Password) - if err != nil { - logrus.Warning(err) - } - agent.Password = &password + settings, err := GetSettings(q) + if err != nil { + return err } - if agent.AgentPassword != nil { - agentPassword, err := handler(*agent.AgentPassword) + if settings.EncryptionKeyFingerprint == "" { + // Either a fresh install or an upgrade from a version that did not record the + // fingerprint. This node's key is adopted only if it can read what is already stored. + err = checkStoredSecretsReadable(q, settings) if err != nil { - logrus.Warning(err) + return err } - agent.AgentPassword = &agentPassword + + settings.EncryptionKeyFingerprint = fingerprint + + return SaveSettings(q, settings) } - var err error - if !agent.AWSOptions.IsEmpty() { - agent.AWSOptions.AWSAccessKey, err = handler(agent.AWSOptions.AWSAccessKey) - if err != nil { - logrus.Warning(err) - } + if settings.EncryptionKeyFingerprint != fingerprint { + return fmt.Errorf("%w: this node's key fingerprint is %s, the database was encrypted with %s", + ErrEncryptionKeyMismatch, fingerprint, settings.EncryptionKeyFingerprint) + } - agent.AWSOptions.AWSSecretKey, err = handler(agent.AWSOptions.AWSSecretKey) - if err != nil { - logrus.Warning(err) - } + return nil +} + +// checkStoredSecretsReadable decrypts one stored agent username to tell a matching key from a +// foreign one on databases that carry no fingerprint yet. +func checkStoredSecretsReadable(q reform.DBTX, settings *Settings) error { + encrypted := slices.ContainsFunc(settings.EncryptedItems, func(item string) bool { + return strings.HasSuffix(item, ".agents.username") + }) + if !encrypted { + // The probed column holds plaintext, so nothing there can contradict this key. + return nil } - if !agent.AzureOptions.IsEmpty() { - agent.AzureOptions.ClientID, err = handler(agent.AzureOptions.ClientID) - if err != nil { - logrus.Warning(err) - } - agent.AzureOptions.ClientSecret, err = handler(agent.AzureOptions.ClientSecret) - if err != nil { - logrus.Warning(err) - } - agent.AzureOptions.SubscriptionID, err = handler(agent.AzureOptions.SubscriptionID) - if err != nil { - logrus.Warning(err) - } - agent.AzureOptions.TenantID, err = handler(agent.AzureOptions.TenantID) - if err != nil { - logrus.Warning(err) - } + var username string + err := q.QueryRow("SELECT username FROM agents WHERE username IS NOT NULL AND username != '' LIMIT 1").Scan(&username) + if errors.Is(err, sql.ErrNoRows) { + return nil + } + if err != nil { + return fmt.Errorf("failed to read stored agent credentials: %w", err) } - if !agent.MongoDBOptions.IsEmpty() { - agent.MongoDBOptions.TLSCertificateKey, err = handler(agent.MongoDBOptions.TLSCertificateKey) - if err != nil { - logrus.Warning(err) - } - agent.MongoDBOptions.TLSCertificateKeyFilePassword, err = handler(agent.MongoDBOptions.TLSCertificateKeyFilePassword) + _, err = encryption.Decrypt(username) + if err != nil { + return fmt.Errorf("%w: stored agent credentials cannot be decrypted with this node's key: %w", + ErrEncryptionKeyMismatch, err) + } + + return nil +} + +func agentEncryption(agent Agent, handler func(string) (string, error)) (Agent, error) { + apply := func(name string, val *string) error { + res, err := handler(*val) if err != nil { - logrus.Warning(err) + return fmt.Errorf("agent %s: %s: %w", agent.AgentID, name, err) } + *val = res + + return nil } - if !agent.MySQLOptions.IsEmpty() { - agent.MySQLOptions.TLSCert, err = handler(agent.MySQLOptions.TLSCert) - if err != nil { - logrus.Warning(err) + // The *string fields are shared with the caller's Agent, so a copy is assigned instead of + // writing through the existing pointer. + ptr := func(name string, val *string) (*string, error) { + if val == nil { + return nil, nil //nolint:nilnil } - agent.MySQLOptions.TLSKey, err = handler(agent.MySQLOptions.TLSKey) + + res := *val + err := apply(name, &res) if err != nil { - logrus.Warning(err) + return nil, err } + + return &res, nil } - if !agent.PostgreSQLOptions.IsEmpty() { - agent.PostgreSQLOptions.SSLCert, err = handler(agent.PostgreSQLOptions.SSLCert) - if err != nil { - logrus.Warning(err) + var err error + + agent.Username, err = ptr("username", agent.Username) + if err != nil { + return agent, err + } + + agent.Password, err = ptr("password", agent.Password) + if err != nil { + return agent, err + } + + agent.AgentPassword, err = ptr("agent_password", agent.AgentPassword) + if err != nil { + return agent, err + } + + for _, f := range []struct { + nonEmpty bool + name string + val *string + }{ + {!agent.AWSOptions.IsEmpty(), "aws_options.access_key", &agent.AWSOptions.AWSAccessKey}, + {!agent.AWSOptions.IsEmpty(), "aws_options.secret_key", &agent.AWSOptions.AWSSecretKey}, + {!agent.AzureOptions.IsEmpty(), "azure_options.client_id", &agent.AzureOptions.ClientID}, + {!agent.AzureOptions.IsEmpty(), "azure_options.client_secret", &agent.AzureOptions.ClientSecret}, + {!agent.AzureOptions.IsEmpty(), "azure_options.subscription_id", &agent.AzureOptions.SubscriptionID}, + {!agent.AzureOptions.IsEmpty(), "azure_options.tenant_id", &agent.AzureOptions.TenantID}, + {!agent.MongoDBOptions.IsEmpty(), "mongo_options.tls_certificate_key", &agent.MongoDBOptions.TLSCertificateKey}, + {!agent.MongoDBOptions.IsEmpty(), "mongo_options.tls_certificate_key_file_password", &agent.MongoDBOptions.TLSCertificateKeyFilePassword}, + {!agent.MySQLOptions.IsEmpty(), "mysql_options.tls_cert", &agent.MySQLOptions.TLSCert}, + {!agent.MySQLOptions.IsEmpty(), "mysql_options.tls_key", &agent.MySQLOptions.TLSKey}, + {!agent.PostgreSQLOptions.IsEmpty(), "postgresql_options.ssl_cert", &agent.PostgreSQLOptions.SSLCert}, + {!agent.PostgreSQLOptions.IsEmpty(), "postgresql_options.ssl_key", &agent.PostgreSQLOptions.SSLKey}, + } { + if !f.nonEmpty { + continue } - agent.PostgreSQLOptions.SSLKey, err = handler(agent.PostgreSQLOptions.SSLKey) + + err = apply(f.name, f.val) if err != nil { - logrus.Warning(err) + return agent, err } } - return agent + return agent, nil } // EncryptAWSOptionsHandler returns encrypted AWS Options. diff --git a/managed/models/encryption_helpers_test.go b/managed/models/encryption_helpers_test.go index 6bdf2ec854f..ab1b0d527e1 100644 --- a/managed/models/encryption_helpers_test.go +++ b/managed/models/encryption_helpers_test.go @@ -18,6 +18,7 @@ package models_test import ( "context" "database/sql" + "encoding/base64" "encoding/json" "path/filepath" "testing" @@ -90,6 +91,61 @@ func TestDefaultAgentEncryptionColumnsRoundTrip(t *testing.T) { assert.Equal(t, original, readAgentSecrets(ctx, t, sqlDB)) } +// TestEncryptDecryptAgentRoundTrip covers the happy path and the invariant that the caller's +// Agent is left untouched: the *string fields are shared with the caller, so the handlers must +// replace the pointers instead of writing through them. +func TestEncryptDecryptAgentRoundTrip(t *testing.T) { + agent := models.Agent{ + AgentID: "/agent_id/1", + Username: new("username"), + Password: new("password"), + MySQLOptions: models.MySQLOptions{ + TLSCert: "mysql-tls-cert", + TLSKey: "mysql-tls-key", + }, + } + + encrypted, err := models.EncryptAgent(agent) + require.NoError(t, err) + require.NotNil(t, encrypted.Username) + assert.NotEqual(t, "username", *encrypted.Username) + assert.NotEqual(t, "mysql-tls-cert", encrypted.MySQLOptions.TLSCert) + + require.NotNil(t, agent.Username) + assert.Equal(t, "username", *agent.Username, "input agent must not be mutated") + + decrypted, err := models.DecryptAgent(encrypted) + require.NoError(t, err) + require.NotNil(t, decrypted.Username) + require.NotNil(t, decrypted.Password) + assert.Equal(t, "username", *decrypted.Username) + assert.Equal(t, "password", *decrypted.Password) + assert.Equal(t, "mysql-tls-cert", decrypted.MySQLOptions.TLSCert) + assert.Equal(t, "mysql-tls-key", decrypted.MySQLOptions.TLSKey) +} + +// TestDecryptAgentDoesNotReturnCiphertext guards the fix for +// https://perconadev.atlassian.net/browse/PMM-14979: a value that this node's key cannot decrypt +// must surface as an error, and the ciphertext must not be handed back to the caller as if it +// were the decrypted value. +func TestDecryptAgentDoesNotReturnCiphertext(t *testing.T) { + // Valid base64 but not ciphertext produced by this node's key, which is what an HA + // follower reads when the row was encrypted with another node's key. + foreignCiphertext := base64.StdEncoding.EncodeToString([]byte("encrypted-with-another-key")) + + agent := models.Agent{ + AgentID: "/agent_id/1", + Username: new(foreignCiphertext), + Password: new(foreignCiphertext), + } + + decrypted, err := models.DecryptAgent(agent) + require.Error(t, err) + assert.Contains(t, err.Error(), "/agent_id/1", "error should identify the agent") + assert.Contains(t, err.Error(), "username", "error should identify the field") + assert.Nil(t, decrypted.Username, "ciphertext must not be returned as the decrypted value") +} + //nolint:dupword func insertAgentWithSecrets(ctx context.Context, t *testing.T, db *sql.DB) { t.Helper() diff --git a/managed/models/encryption_key_test.go b/managed/models/encryption_key_test.go new file mode 100644 index 00000000000..fa1a6258bf4 --- /dev/null +++ b/managed/models/encryption_key_test.go @@ -0,0 +1,122 @@ +// Copyright (C) 2023 Percona LLC +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . + +package models + +import ( + "encoding/base64" + "encoding/json" + "testing" + + sqlmock "github.com/DATA-DOG/go-sqlmock" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + "gopkg.in/reform.v1" + "gopkg.in/reform.v1/dialects/postgresql" + + "github.com/percona/pmm/managed/utils/encryption" +) + +// TestVerifyEncryptionKey covers detection of the HA misconfiguration behind +// https://perconadev.atlassian.net/browse/PMM-14979, where each node generated its own +// encryption key while sharing one database. +func TestVerifyEncryptionKey(t *testing.T) { + localFingerprint, err := encryption.Fingerprint() + require.NoError(t, err) + require.NotEmpty(t, localFingerprint) + + foreignCiphertext := base64.StdEncoding.EncodeToString([]byte("encrypted-with-another-key")) + + readableCiphertext, err := encryption.Encrypt("pmm-managed") + require.NoError(t, err) + + settingsJSON := func(t *testing.T, s Settings) []byte { + t.Helper() + b, err := json.Marshal(s) //nolint:musttag + require.NoError(t, err) + + return b + } + + newMock := func(t *testing.T) (*reform.DB, sqlmock.Sqlmock) { + t.Helper() + + sqlDB, mock, err := sqlmock.New() + require.NoError(t, err) + t.Cleanup(func() { + assert.NoError(t, mock.ExpectationsWereMet()) + _ = mock.ExpectClose() + assert.NoError(t, sqlDB.Close()) + }) + + return reform.NewDB(sqlDB, postgresql.Dialect, nil), mock + } + + expectSettings := func(t *testing.T, mock sqlmock.Sqlmock, s Settings) { + t.Helper() + mock.ExpectQuery("SELECT settings FROM settings"). + WillReturnRows(sqlmock.NewRows([]string{"settings"}).AddRow(settingsJSON(t, s))) + } + + t.Run("matching fingerprint is accepted and not rewritten", func(t *testing.T) { + db, mock := newMock(t) + expectSettings(t, mock, Settings{EncryptionKeyFingerprint: localFingerprint}) + + assert.NoError(t, VerifyEncryptionKey(db)) + }) + + t.Run("foreign fingerprint is reported as a mismatch", func(t *testing.T) { + db, mock := newMock(t) + expectSettings(t, mock, Settings{EncryptionKeyFingerprint: "0123456789abcdef"}) + + err := VerifyEncryptionKey(db) + require.ErrorIs(t, err, ErrEncryptionKeyMismatch) + assert.Contains(t, err.Error(), localFingerprint) + assert.Contains(t, err.Error(), "0123456789abcdef") + }) + + t.Run("fingerprint is recorded when nothing is encrypted yet", func(t *testing.T) { + db, mock := newMock(t) + expectSettings(t, mock, Settings{}) + mock.ExpectExec("UPDATE settings SET settings"). + WithArgs(sqlmock.AnyArg()). + WillReturnResult(sqlmock.NewResult(0, 1)) + + assert.NoError(t, VerifyEncryptionKey(db)) + }) + + t.Run("fingerprint is adopted when stored data decrypts", func(t *testing.T) { + db, mock := newMock(t) + expectSettings(t, mock, Settings{EncryptedItems: []string{"pmm-managed.agents.username"}}) + mock.ExpectQuery("SELECT username FROM agents"). + WillReturnRows(sqlmock.NewRows([]string{"username"}).AddRow(readableCiphertext)) + mock.ExpectExec("UPDATE settings SET settings"). + WithArgs(sqlmock.AnyArg()). + WillReturnResult(sqlmock.NewResult(0, 1)) + + assert.NoError(t, VerifyEncryptionKey(db)) + }) + + // The upgrade path for the reported cluster: a follower with its own key, against a database + // whose rows were encrypted by another node and which carries no fingerprint yet. + t.Run("fingerprint is not adopted when stored data cannot be decrypted", func(t *testing.T) { + db, mock := newMock(t) + expectSettings(t, mock, Settings{EncryptedItems: []string{"pmm-managed.agents.username"}}) + mock.ExpectQuery("SELECT username FROM agents"). + WillReturnRows(sqlmock.NewRows([]string{"username"}).AddRow(foreignCiphertext)) + + require.ErrorIs(t, VerifyEncryptionKey(db), ErrEncryptionKeyMismatch) + }) +} diff --git a/managed/models/settings.go b/managed/models/settings.go index d76fc4552cf..6836010f022 100644 --- a/managed/models/settings.go +++ b/managed/models/settings.go @@ -118,6 +118,11 @@ type Settings struct { // Contains all encrypted tables in format 'db.table.column'. EncryptedItems []string `json:"encrypted_items"` + + // EncryptionKeyFingerprint identifies the encryption key the data in this database was + // encrypted with. In HA all nodes share the database but keep their own key file, so a node + // compares this against its own key to detect that it cannot read the stored credentials. + EncryptionKeyFingerprint string `json:"encryption_key_fingerprint"` } // IsAlertingEnabled returns true if alerting is enabled. diff --git a/managed/models/settings_helpers.go b/managed/models/settings_helpers.go index a48c5d74510..d15e704a645 100644 --- a/managed/models/settings_helpers.go +++ b/managed/models/settings_helpers.go @@ -102,6 +102,9 @@ type ChangeSettingsParams struct { // List of items in format 'db.table.column' to be encrypted. EncryptedItems []string + + // EncryptionKeyFingerprint identifies the key the data was encrypted with. Empty clears it. + EncryptionKeyFingerprint *string } // SetPMMServerID should be run on start up to generate unique PMM Server ID. @@ -243,6 +246,10 @@ func UpdateSettings(q reform.DBTX, params *ChangeSettingsParams) (*Settings, err settings.EncryptedItems = params.EncryptedItems } + if params.EncryptionKeyFingerprint != nil { + settings.EncryptionKeyFingerprint = *params.EncryptionKeyFingerprint + } + err = SaveSettings(q, settings) if err != nil { return nil, err diff --git a/managed/services/agents/service_info_broker.go b/managed/services/agents/service_info_broker.go index d0165990110..f9a9edf2425 100644 --- a/managed/services/agents/service_info_broker.go +++ b/managed/services/agents/service_info_broker.go @@ -194,7 +194,7 @@ func (c *ServiceInfoBroker) GetInfoFromService(ctx context.Context, q *reform.Qu case models.MySQLServiceType: agent.MySQLOptions.TableCount = &sInfo.TableCount l.Debugf("Updating table count: %d.", sInfo.TableCount) - err = q.Update(new(models.EncryptAgent(*agent))) + err = models.UpdateAgent(q, agent) if err != nil { return fmt.Errorf("failed to update table count: %w", err) } @@ -216,7 +216,7 @@ func (c *ServiceInfoBroker) GetInfoFromService(ctx context.Context, q *reform.Qu agent.PostgreSQLOptions.DatabaseCount = int32(databaseCount - excludedDatabaseCount) l.Debugf("Updating PostgreSQL options, database count: %d.", agent.PostgreSQLOptions.DatabaseCount) - err = q.Update(new(models.EncryptAgent(*agent))) + err = models.UpdateAgent(q, agent) if err != nil { return fmt.Errorf("failed to update database count: %w", err) } diff --git a/managed/services/realtimeanalytics/service.go b/managed/services/realtimeanalytics/service.go index 0414f9b6558..f7e9053a51a 100644 --- a/managed/services/realtimeanalytics/service.go +++ b/managed/services/realtimeanalytics/service.go @@ -268,10 +268,8 @@ func (s *Service) StartSession(ctx context.Context, req *rtav1.StartSessionReque rtaAgent.Disabled = false // Need to update CreatedAt to reflect the new session start time. rtaAgent.CreatedAt = time.Now() - // Encrypt agent's sensitive data before updating it in the database. - rtaAgent = new(models.EncryptAgent(*rtaAgent)) - err = tx.Update(rtaAgent) + err = models.UpdateAgent(tx.Querier, rtaAgent) if err != nil { return status.Errorf(codes.Internal, "Failed to update Real-Time Analytics agent %s: %v", rtaAgent.AgentID, err) } @@ -433,10 +431,8 @@ func (s *Service) StopSession(ctx context.Context, req *rtav1.StopSessionRequest // RTA Agent exists - update its state rtaAgent := existingRTAAgents[0] rtaAgent.Disabled = true - // Encrypt agent's sensitive data before updating it in the database. - rtaAgent = new(models.EncryptAgent(*rtaAgent)) - err = tx.Update(rtaAgent) + err = models.UpdateAgent(tx.Querier, rtaAgent) if err != nil { return status.Errorf(codes.Internal, "Failed to update Real-Time Analytics agent %s: %v", rtaAgent.AgentID, err) } diff --git a/managed/services/realtimeanalytics/service_test.go b/managed/services/realtimeanalytics/service_test.go index be12458c710..6e675136161 100644 --- a/managed/services/realtimeanalytics/service_test.go +++ b/managed/services/realtimeanalytics/service_test.go @@ -238,7 +238,8 @@ func TestListSessions(t *testing.T) { svc := NewService(db, registry, stateUpdater, store) rtaAgent.Status = inventoryv1.AgentStatus_name[int32(inventoryv1.AgentStatus_AGENT_STATUS_RUNNING)] - err = db.Update(rtaAgent) + err = models.UpdateAgent(db.Querier, rtaAgent) + require.NoError(t, err) resp, err := svc.ListSessions(t.Context(), &rtav1.ListSessionsRequest{}) require.NoError(t, err) diff --git a/managed/services/victoriametrics/victoriametrics_test.go b/managed/services/victoriametrics/victoriametrics_test.go index 77f67e2452e..3e0c7f245f4 100644 --- a/managed/services/victoriametrics/victoriametrics_test.go +++ b/managed/services/victoriametrics/victoriametrics_test.go @@ -287,7 +287,9 @@ func TestVictoriaMetrics(t *testing.T) { }, } { if str, ok := str.(*models.Agent); ok { - *str = models.EncryptAgent(*str) + encrypted, err := models.EncryptAgent(*str) + check.NoError(err) + *str = encrypted } err := db.Insert(str) diff --git a/managed/utils/encryption/encryption.go b/managed/utils/encryption/encryption.go index 9fd9fc090bb..473a0f00ea3 100644 --- a/managed/utils/encryption/encryption.go +++ b/managed/utils/encryption/encryption.go @@ -18,7 +18,9 @@ package encryption import ( "bytes" + "crypto/sha256" "encoding/base64" + "encoding/hex" "errors" "fmt" "os" @@ -152,7 +154,7 @@ func RotateEncryptionKey() error { // RestoreOldEncryptionKey is a wrapper around DefaultEncryption.RestoreOldEncryptionKey. func RestoreOldEncryptionKey() error { - err := os.Rename(strings.TrimSuffix(encryptionKeyPath(), ".key")+"_old.key", encryptionKeyPath()) + err := os.Rename(strings.TrimSuffix(KeyPath(), ".key")+"_old.key", KeyPath()) if err != nil { return fmt.Errorf("could not restore old encryption key: %w", err) } @@ -161,7 +163,7 @@ func RestoreOldEncryptionKey() error { } func backupOldEncryptionKey() error { - err := os.Rename(encryptionKeyPath(), strings.TrimSuffix(encryptionKeyPath(), ".key")+"_old.key") + err := os.Rename(KeyPath(), strings.TrimSuffix(KeyPath(), ".key")+"_old.key") if err != nil { return fmt.Errorf("failed to backup old encryption key: %w", err) } @@ -185,6 +187,29 @@ func (e *Encryption) GenerateKey() (string, error) { return base64.StdEncoding.EncodeToString(buff.Bytes()), nil } +// Fingerprint is a wrapper around DefaultEncryption.Fingerprint. +func Fingerprint() (string, error) { + return getDefaultEncryption().Fingerprint() +} + +// Fingerprint returns a stable identifier of the encryption key: a digest, so it cannot be used +// to reconstruct the key. It is recorded next to the encrypted data so that a node can tell +// whether it holds the right key without having an encrypted row to test against. +func (e *Encryption) Fingerprint() (string, error) { + if e == nil || e.Key == "" { + return "", ErrEncryptionNotInitialized + } + + serializedKeyset, err := base64.StdEncoding.DecodeString(e.Key) + if err != nil { + return "", fmt.Errorf("failed to decode keyset: %w", err) + } + + sum := sha256.Sum256(serializedKeyset) + + return hex.EncodeToString(sum[:]), nil +} + func (e *Encryption) generateAndPersistKey() error { key, err := e.GenerateKey() if err != nil { @@ -204,16 +229,18 @@ func Encrypt(secret string) (string, error) { } // Encrypt returns input string encrypted. +// On failure it returns an empty string rather than the plaintext, so that a caller which +// ignores the error cannot persist an unencrypted secret. func (e *Encryption) Encrypt(secret string) (string, error) { if e == nil || e.Primitive == nil { - return secret, ErrEncryptionNotInitialized + return "", ErrEncryptionNotInitialized } if secret == "" { return secret, nil } cipherText, err := e.Primitive.Encrypt([]byte(secret), []byte("")) if err != nil { - return secret, fmt.Errorf("encryption: %w", err) + return "", fmt.Errorf("encryption: %w", err) } return base64.StdEncoding.EncodeToString(cipherText), nil @@ -269,20 +296,22 @@ func Decrypt(cipherText string) (string, error) { } // Decrypt returns input string decrypted. +// On failure it returns an empty string rather than the input ciphertext, so that a caller which +// ignores the error cannot pass ciphertext on as if it were the decrypted value. func (e *Encryption) Decrypt(cipherText string) (string, error) { if e == nil || e.Primitive == nil { - return cipherText, ErrEncryptionNotInitialized + return "", ErrEncryptionNotInitialized } if cipherText == "" { return cipherText, nil } decoded, err := base64.StdEncoding.DecodeString(cipherText) if err != nil { - return cipherText, fmt.Errorf("decryption: %w, %s", err, cipherText) + return "", fmt.Errorf("decryption: %w", err) } secret, err := e.Primitive.Decrypt(decoded, []byte("")) if err != nil { - return cipherText, fmt.Errorf("decryption: %w", err) + return "", fmt.Errorf("decryption: %w", err) } return string(secret), nil diff --git a/managed/utils/encryption/helpers.go b/managed/utils/encryption/helpers.go index 0cbbd2c3728..9fb6a5bf564 100644 --- a/managed/utils/encryption/helpers.go +++ b/managed/utils/encryption/helpers.go @@ -25,8 +25,9 @@ import ( "gopkg.in/reform.v1" ) -func encryptionKeyPath() string { - customKeyPath := os.Getenv("PMM_ENCRYPTION_KEY_PATH") +// KeyPath returns the path PMM reads the encryption key from. +func KeyPath() string { + customKeyPath := os.Getenv(CustomEncryptionKeyPathEnvVar) if customKeyPath != "" { return customKeyPath }