MacOS Certificate Generator Fix#5941
Open
afifi-ins wants to merge 34 commits into
Open
Conversation
Remove the custom SafeKeychainHandle-based keychain approach that was causing 'The X509 certificate store has not been opened' errors on macOS. Changes: - CertificateHelper.GetX509Store: macOS now uses standard X509Store with StoreLocation.CurrentUser (same as Linux) instead of custom keychain - Remove GetMacOSX509Store, OSXCustomKeychainFilePath, OSXCustomKeychainPassword, and EnsureStoreIsOpened - Remove !IsMacOS() guards that skipped store.Open(ReadWrite) in CertificateManager and CertificateGeneratorLibrary - Remove SafeKeychainHandle.cs linked file from csproj Co-authored-by: Copilot <[email protected]>
On macOS, the Root and TrustedPeople certificate stores use Apple's trust infrastructure and cannot be opened with ReadWrite via the .NET X509Store API. Instead of failing and catching exceptions, handle this upfront by checking the platform and store type before attempting operations. Changes: - CertificateHelper: Add AddTrustedCertOnMacOS/RemoveTrustedCertOnMacOS helpers that use the macOS 'security' CLI to manage certificate trust - CertificateManager.AddToStoreIfNeeded: Route macOS Root/TrustedPeople operations directly to the security CLI without attempting X509Store - CertificateGeneratorLibrary.RemoveCertificatesFromStore: Check macOS and store type upfront, use security CLI for read-only stores Co-authored-by: Copilot <[email protected]>
The macOS login keychain requires user interaction ('User interaction
is not allowed') when adding certificates via the .NET X509Store API,
which fails in CI/headless environments.
Replace all X509Store operations on macOS with a custom unlocked
keychain managed via the 'security' CLI:
- CertificateHelper: Create/manage a dedicated 'wcf-test.keychain-db'
that is unlocked and added to the search list. All cert imports and
trust operations target this keychain.
- CertificateManager.AddToStoreIfNeeded: On macOS, route My store
operations to ImportCertToMacOSKeychain (security import) and
Root/TrustedPeople to AddTrustedCertOnMacOS (security add-trusted-cert)
- CertificateGeneratorLibrary: On macOS, cleanup deletes the entire
custom keychain instead of iterating per-store
Co-authored-by: Copilot <[email protected]>
BouncyCastle's default Pkcs12StoreBuilder uses RC2-40-CBC for cert encryption, which is not supported by macOS's Security framework, causing 'Import/Export format unsupported' when loading PFX bytes into X509Certificate2. Configure the PKCS12 builder to use 3DES (PbeWithShaAnd3KeyTripleDesCbc) for both key and cert encryption, which is supported across all platforms. Co-authored-by: Copilot <[email protected]>
…mport On macOS, the .NET X509Certificate2 constructor cannot load BouncyCastle-generated PKCS12 due to format incompatibilities with the Apple Security framework (Import/Export format unsupported error). Instead of loading PFX into X509Certificate2 on macOS: - Create X509Certificate2 from DER-encoded public cert only (for metadata) - Pass raw PFX bytes directly to the security CLI import command - Updated AddToStoreIfNeeded and InstallCertificateToMyStore to accept optional PFX bytes for macOS keychain import - Changed ImportCertToMacOSKeychain to accept byte[] instead of X509Certificate2 Co-authored-by: Copilot <[email protected]>
…certs On macOS, new X509Certificate2(cert.GetEncoded()) also fails with 'Unknown format in import' for non-authority certs (machine/user certs with SANs and other extensions). The Apple Security framework cannot parse the DER encoding from BouncyCastle for these certs. Fix: On macOS for non-authority certs, skip X509Certificate2 creation entirely. Compute thumbprint (SHA1 of DER) directly from BouncyCastle cert. Container.Certificate is null on macOS — all store operations already use PFX bytes (My store) or DER bytes (Root/TrustedPeople) via the security CLI. Changes: - CertificateGenerator: compute thumbprint from SHA1 on macOS, null cert - CertificateHelper: added AddTrustedCertOnMacOS(byte[]) overload - CertificateManager: added certDerBytes parameter for trust stores, handle null certificate gracefully on macOS Co-authored-by: Copilot <[email protected]>
certificate is null on macOS (we skip X509Certificate2 creation). Use null-conditional operator for Thumbprint access. Co-authored-by: Copilot <[email protected]>
…ve cert After importing PFX to the custom keychain via CLI, the cert needs to be findable by .NET's X509Store API so Kestrel can use it for HTTPS and TestHost.CertificateFromFriendlyName can locate it. Changes: - Set custom keychain as default keychain (security default-keychain -s) so X509Store(My, CurrentUser) searches it - After importing host cert to keychain, retrieve it via X509Store by thumbprint to populate s_localCertificate on macOS - Added FindCertificateInStore helper method Co-authored-by: Copilot <[email protected]>
The root cause of all macOS cert issues is that BouncyCastle's PKCS12 and X.509 DER encodings are incompatible with Apple's Security framework. Both the .NET X509Certificate2 constructor and the 'security import' CLI use the same Apple APIs which reject BouncyCastle's output. Fix: On macOS, after BouncyCastle generates the cert and key, export them as PEM and use openssl (available on macOS) to create a compatible PFX. The openssl-generated PFX loads correctly with X509Certificate2, and imports correctly into the macOS keychain. This means container.Certificate is now a real X509Certificate2 with private key on macOS, so all downstream code (ServiceCredentials, Kestrel HTTPS, CertificateFromFriendlyName lookups) works correctly. Co-authored-by: Copilot <[email protected]>
The macOS issue stemmed from Apple's Security framework rejecting BouncyCastle's X.509 cert DER encoding (LoadX509Der fails with 'Import/Export format unsupported'). Workarounds via custom keychain, 3DES PKCS12, and openssl repackaging all preserve the offending DER bytes and fail. This commit replaces BouncyCastle entirely with .NET's built-in X.509 APIs which produce platform-native, Apple-compatible DER encodings: - CertificateRequest + CreateSelfSigned/Create for cert generation - Built-in extensions: BasicConstraints, KeyUsage, SKI, EKU, SAN - AuthorityKeyIdentifier and CRL Distribution Points built via System.Formats.Asn1.AsnWriter (no built-in extension class for AKI prior to .NET 7; CRL DPs has no built-in) - UPN OtherName SAN built via AsnWriter - CRL generated via AsnWriter and signed with RSA.SignData - PKCS12 export via cert.Export(Pkcs12, password) CertificateCreationSettings.EKU changes from List<KeyPurposeID> to List<string> (OID strings). Library and EXE retargeted to net10.0 (CertificateRequest is netstandard2.1+ and CopyWithPrivateKey is .NET Core+; SelfHostedCoreWcfService consumer is already net10.0). CertificateManager simplified: drop pfxBytes/certDerBytes plumbing — on macOS, export PFX from cert at point of use (now possible since cert is Apple-compatible). Custom macOS keychain infrastructure (CertificateHelper) is retained because Root/TrustedPeople stores still cannot be modified via X509Store on macOS, and login keychain may be locked in CI. Co-authored-by: Copilot <[email protected]>
…ation Two issues caught by Helix CI on the BouncyCastle removal: 1. macOS: 'The specified keychain could not be found' from X509CertificateLoader.LoadPkcs12 -> AppleCertificatePal.MoveToKeychain. The PFX round-trip after CreateCertificate is unnecessary now that we build the cert in memory; on macOS, LoadPkcs12 needs a keychain handle. Use the in-memory CopyWithPrivateKey result directly. 2. Linux/all: 'notBefore is earlier than issuerCertificate.NotBefore' from CertificateRequest.Create. BouncyCastle didn't validate child window vs issuer window. The expired-cert test case sets NotBefore = UtcNow-4d, but the authority used UtcNow-1h. Widen the authority validity to +/-10 years so all child certs (including intentionally expired ones) fit. Also clamp child windows defensively if a caller supplies dates outside issuer range. Co-authored-by: Copilot <[email protected]>
The previous iteration removed the PFX round-trip universally to fix macOS, but this broke Windows and Linux: on those platforms, the ephemeral key produced by CopyWithPrivateKey isn't persisted into a key container when the cert is added to an X509Store. Later lookups by thumbprint return a cert without a usable private key, surfacing as: - Windows: ArgumentNullException 'serverCertificate' in Kestrel UseHttps - macOS (now passing the same path): The service certificate is not provided in CoreWCF ServiceCredentials Make the round-trip platform-conditional: - macOS: use the in-memory CopyWithPrivateKey result directly (LoadPkcs12 fails there with 'keychain could not be found') - Windows/Linux: round-trip through X509CertificateLoader.LoadPkcs12 with PersistKeySet so the private key is persisted Also tightened disposal: on Windows/Linux the round-tripped outputCert is a separate instance, so the in-memory certWithKey must be disposed. Co-authored-by: Copilot <[email protected]>
…on Windows Comparing with the original BouncyCastle code (commit 02ddc7b) revealed two regressions in the .NET CertificateRequest port: 1. PFX round-trip on Windows/Linux was missing the MachineKeySet storage flag. The original used MachineKeySet | Exportable | PersistKeySet. Without MachineKeySet the private key lands in the user's CNG container while the cert is added to LocalMachine\My, so subsequent X509Store lookups return a cert with HasPrivateKey=false and Kestrel.UseHttps throws ArgumentNullException. 2. The original BouncyCastle Pkcs12Store.SetKeyEntry set the bag alias to the friendly name, which Windows surfaces as cert.FriendlyName when the PFX is loaded. .NET's cert.Export(Pkcs12) does not set an alias, so explicitly set outputCert.FriendlyName on Windows after loading. This restores TestHost.CertificateFromFriendlyName lookups. Added diagnostic Console.WriteLine in CertificateHelper.ImportCertToMacOSKeychain, ImportPublicCertToMacOSKeychain, and TestHost.CertificateFromFriendlyName so the next macOS CI run shows what's installed and what the lookup is matching against. Co-authored-by: Copilot <[email protected]>
The setter is wrapped in IsWindows() but the analyzer can't see through the helper method. Repo treats CA1416 as error in CI. Co-authored-by: Copilot <[email protected]>
…okups converge The TrustedPeople lookup on macOS was returning 0 candidates even though certs had been imported into the custom keychain. macOS does not have proper per-store separation like Windows: .NET's X509Store(TrustedPeople|Root, CurrentUser) does not enumerate certs imported via the 'security' CLI into the user's default keychain. Route all storeName values through StoreName.My on macOS so: - CertificateManager imports targeting My/Root/TrustedPeople all land in the custom keychain (already the case for My; now also for Root/TrustedPeople via ImportPublicCertToMacOSKeychain). - TestHost.CertificateFromFriendlyName looking up in TrustedPeople finds the imported certs in the same keychain. For Root specifically, also call AddTrustedCertOnMacOS so chain validation still works via OS-level trust settings. Co-authored-by: Copilot <[email protected]>
The TrustedPeople branch was importing the cert as public-only, but the peer cert used by CoreWCF for service credentials needs a private key for key exchange. CoreWCF.SecurityUtils.EnsureCertificateCanDoKeyExchange threw: 'It is likely that certificate ... may not have a private key that is capable of key exchange or the process may not have access rights for the private key' Now always import the PFX (with key) when certificate.HasPrivateKey is true, regardless of target store. Still call AddTrustedCertOnMacOS for Root certs so OS-level chain validation works. Co-authored-by: Copilot <[email protected]>
On macOS the self-signed root cert in our custom test keychain is not seen as trusted by .NET's chain builder, so Find(..., validOnly:true) filters it out and /TestHost.svc/RootCert returns 500. The companion CertificateFromFriendlyName already uses validOnly:false; align the two helpers. Co-authored-by: Copilot <[email protected]>
… on macOS (issue dotnet#2870) These two tests trigger HTTPS handshakes that require the test root cert to be trusted by the OS chain validator. On macOS, .NET defers SSL chain validation to the OS keychain trust store, which is not populated by the in-process .NET AddToStore calls (and the InstallRootCertificate.sh sudo path is not run by Helix). All sibling tests in these files already carry [Issue(2870, OS=OSX)]; add the same attribute to these two so they skip on macOS instead of failing with the unavoidable PartialChain error. Co-authored-by: Copilot <[email protected]>
- Remove unused FindCertificateInStore helper from CertificateManager. - Remove unused RemoveTrustedCertOnMacOS and RemoveCertsFromMacOSKeychain helpers from CertificateHelper (DeleteMacOSKeychain covers cleanup). - Drop the now-resolved diagnostic Console.WriteLine blocks in TestHost.CertificateFromFriendlyName and CertificateHelper imports. Trace.WriteLine equivalents remain for log capture. Cert configuration verified against the original BouncyCastle generator: KeyUsage, BasicConstraints, EKU, SAN, CRL, PKCS12 load flags, and FriendlyName are equivalent across platforms. Co-authored-by: Copilot <[email protected]>
The CertificateGenerator project was retargeted from net471 to net10.0 as part of the cert-generator port to native .NET APIs, so its build output now lands at artifacts\bin\CertificateGenerator\Release\net10.0. Update all .cmd scripts that invoke CertificateGenerator.exe accordingly: - CleanUpWCFSelfHostedSvc.cmd (existence check + -Uninstall calls) - RefreshServerCertificates.cmd - SetupWcfIISHostedService.cmd - StartWCFSelfHostedSvcDoWork.cmd Co-authored-by: Copilot <[email protected]>
Replaces the loose 'private const string Oid...' constants with named static readonly Oid instances that carry a friendly name, which renders helpfully in certificate viewers and improves call-site readability: ekuOids.Add(ServerAuthEkuOid); new X509Extension(CrlDistributionPointsExtensionOid, ..., critical: false); WriteExtension(w, CrlNumberExtensionOid, ..., value); WriteExtension now takes an Oid; AsnWriter.WriteObjectIdentifier still requires a string and is fed via Oid.Value. Drops the unused OidExtAuthorityInfoAccess constant. Co-authored-by: Copilot <[email protected]>
…veNameBuilder.AddUserPrincipalName Replaces hand-rolled AKI and UPN-SAN extension construction with .NET built-in helpers (X509AuthorityKeyIdentifierExtension.CreateFromSubjectKeyIdentifier from .NET 7+, SubjectAlternativeNameBuilder.AddUserPrincipalName from .NET 9+). The CRL path also uses the AKI helper's RawData to obtain the encoded extnValue, eliminating the local BuildAuthorityKeyIdentifierValue helper. Drops the now-unused SAN and UPN OID constants. Co-authored-by: Copilot <[email protected]>
Replaces hand-rolled TBSCertList ASN.1 encoding with the built-in CertificateRevocationListBuilder (.NET 9+), which handles signing, CRL Number, AKI, time encoding (UTCTime/GeneralizedTime), and DER layout. Drops ~140 lines of helpers (BuildTbsCertList, BuildCrlNumberValue, WriteExtension, WriteSha256RsaAlgorithmIdentifier, WriteX509Time) and the AKI/CrlNumber/Sha256-RSA OID constants that were only used by the hand-rolled CRL path. Co-authored-by: Copilot <[email protected]>
- ComputeSerialNumber: use BigInteger ctor + ToByteArray with isUnsigned/isBigEndian instead of manual sign/reverse logic - SerialToHex: use Convert.ToHexString - HexToBytes: use Convert.FromHexString (drop unused whitespace/hyphen stripping) - Remove unused 'using System.IO' Co-authored-by: Copilot <[email protected]>
The fedora-41 Helix image (mcr.microsoft.com/dotnet-buildtools/prereqs:fedora-41-helix) does not ship the 'which' utility, causing every lookup in the script to fail and the root CA install to bail with 'Could not find update-ca-trust'. Replace the three 'which' invocations with the POSIX-portable 'command -v', which is a shell builtin and always available. Co-authored-by: Copilot <[email protected]>
CertificateRevocationListBuilder.AddEntry(byte[]) writes the supplied bytes verbatim as the INTEGER content of the revocation entry; it only rejects redundant 0x00/0xFF padding. ComputeSerialNumber returns the minimal unsigned big-endian encoding (no sign byte), which means that when the high bit of the first byte is set the CRL ends up with an INTEGER that DER interprets as negative, while CertificateRequest.Create encodes the certificate's serial with a leading 0x00 sign byte and ends up positive. The two encodings never match, so the OS revocation check never fires (TCP_ServiceCertRevoked_Throw_SecurityNegotiationException failed because the chain reported the cert as valid). Prepend a 0x00 sign byte when the high bit is set so the CRL serial INTEGER matches the certificate's serial INTEGER for all values. Co-authored-by: Copilot <[email protected]>
Add MacOSKeychain helper that wraps the 'security' CLI: create + unlock a dedicated custom keychain, make it the default user keychain, import certs with -A, and apply 'add-trusted-cert -r trustRoot -p ssl' so the WCF test root CA validates as fully trusted (was landing in the user keychain without an SSL policy, i.e. partial trust, which broke TLS handshakes on macOS). CertificateManager.AddToStoreIfNeeded routes through the helper on macOS before falling through to X509Store.Add, so managed thumbprint lookups keep working. InstallRootCertificate.sh: macOS branch now uses a user-domain custom keychain (no sudo) with the same trust policy. Centralize OIDs in a new Oids.cs constants file for CertificateGenerator. Remove all 25 [Issue(2870, OS = OSID.OSX)] skip attributes across HTTPS/TCP/UDS/WSFederation/WSHttp/WS2007Http/WSNetTcp/BasicHttp(s) tests. Co-authored-by: Copilot <[email protected]>
The previous attempt over-reached: it created a user-domain custom keychain and called add-trusted-cert from C#. Helix runs InstallRootCertificate.sh under 'sudo -E -n', so the user-domain keychain dance fails ('UID=0 does not own /Users/helix-runner'), and add-trusted-cert to a user keychain always requires a GUI TCC prompt, even as root ('SecTrustSettingsSetTrustSettings: authorization denied').
Real root cause is simpler: the original 'security add-trusted-cert -d -r trustRoot -k System.keychain' call omitted '-p ssl', so the root CA landed without an SSL trust policy and macOS reported the chain as partial trust, breaking TLS.
Revert the keychain-juggling. Just add '-p ssl' to the existing System.keychain install. Drop MacOSKeychain.cs and the matching CertificateManager hook (they could not work non-interactively for the test user).
Co-authored-by: Copilot <[email protected]>
The Helix pre-command invoked InstallRootCertificate.sh with `--service-host $(ServiceHost)` but the ServiceHost MSBuild property was never defined, so the call expanded to `--service-host --cert-file ...`. The install script then parsed `--cert-file` as the service host and curl failed with `Could not resolve host: --cert-file`, meaning the root CA was never downloaded or trusted on the Helix macOS machine. Every TLS test then failed with PartialChain (dotnet#2870). Default ServiceHost to localhost so the script can fetch and install the root CA, completing the partial-trust fix together with the existing `-p ssl` change in InstallRootCertificate.sh. Co-authored-by: Copilot <[email protected]>
The previous commit added an XML comment containing `--` which is illegal inside XML comments. MSBuild failed to load the project on all platforms with MSB4025. Reword the comment to avoid `--` sequences; no functional change. Co-authored-by: Copilot <[email protected]>
Roll back the ServiceHost=localhost default from b7b29cc/9769ee02 (the InstallRootCertificate.sh invocation is still needed for the Linux+CoreWCF leg as-is) and instead address the second macOS partial-trust symptom: client certificate installation fails with errSecNoSuchKeychain on Helix because the non-interactive 'helix-runner' user has no login keychain. Add a macOS-only HelixPreCommands block that creates, unlocks, and registers ~/Library/Keychains/login.keychain-db before any tests run so that X509Store(My, CurrentUser) can be opened, completing the dotnet#2870 macOS cert install fix together with the `-p ssl` change in InstallRootCertificate.sh. Co-authored-by: Copilot <[email protected]>
The previous attempt failed because Helix macOS workers already have a login.keychain-db with an unknown password set by the infrastructure; `create-keychain` was a no-op, then `set-keychain-settings` / `unlock-keychain` failed with `user interaction is not allowed` and `passphrase ... is not correct`, leaving the keychain locked and X509Store(My, CurrentUser) still failing on the un-skipped dotnet#2870 tests. Delete any pre-existing login keychain first, then create a fresh one with an empty password we control. Co-authored-by: Copilot <[email protected]>
…sudo Move AddTrustedCertOnMacOS from a custom user keychain (which macOS's TLS chain evaluator doesn't reliably honor) to /Library/Keychains/System.keychain in the admin trust domain via 'sudo -n security add-trusted-cert -d -r trustRoot -p ssl'. Helix macOS workers have passwordless sudo, so this stays non-interactive. Surface security CLI failures to stderr so they're visible in Helix console logs. Co-authored-by: Copilot <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.