Skip to content

Fips compliance internal #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 15 commits into
base: main
Choose a base branch
from
Open

Fips compliance internal #2

wants to merge 15 commits into from

Conversation

beanuwave
Copy link
Collaborator

@beanuwave beanuwave commented Jul 24, 2024 •
edited
Loading

Description

  • FIPS gradle build script is removed.
  • All BC dependencies are replaces by BCFIPS.
  • Password matcher inside Identity-Shiro that replies on BC to check if hashed passwords matches with OpenBSDBCrypt, is replaced by password4j implementation.
  • Adds support for BCFKS format (*.bks) for Key & Truststores.
  • Refactor parsing private keys with formats EC, PKCS8, PKCS1, DSA, w/wo encryption, w/wo parameters.
  • FIPS approved-only mode can be configured over opensearch.yml file.
  • java security file is added to the build.
  • java policy file is altered to grant neccessary security permissions.

This PR provides FIPS 140-2 support by replacing all BC dependencies with BCFIPS dependencies and making FIPS approved-only mode configurable at launch. Running application in approved-only mode restricts BCFIPS provoder to rely solely on FIPS certified cyphers. Due to replacement of BC libraries, BCrypt password matching and private-key loading from file were replaced by alternative implementations.

Reasons for refactoring PemUtils.java that is used by Reindex API, in case of migrating data from a remote cluster that is TLS protected:

  • PKCS#8 implementation was not supported by BCFIPS library.
  • java type security.
  • Password Based Key Derivation Functions such as PKCS#12 and OpenSSL are not supported in BCFIPS approved-only mode, because only PBKDF2 standard is approved for use in FIPS.
  • generally good idea to let ASN1 annotation parsing be done by external security libraries.

Related Issues

opensearch-project/security#3420

Check List

  • Functionality includes testing.
  • API changes companion pull request created, if applicable.
  • Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@beanuwave beanuwave force-pushed the fips_compliance_internal branch from 7e42d65 to 2badd2f Compare July 24, 2024 08:53
@beanuwave beanuwave force-pushed the fips_compliance_internal branch from 2badd2f to abdabf3 Compare December 10, 2024 14:17
iigonin added 14 commits December 12, 2024 18:15
@iigonin
Draft to allow run in FIPS compliace mode
ab9b5ff 
Signed-off-by: Iwan Igonin <[email protected]>

# Conflicts:
#	server/build.gradle
@iigonin
make tests run without BC (not BCFIPS) libraries.
b5eb7a0 
Signed-off-by: Iwan Igonin <[email protected]>

� Please enter the commit message for your changes. Lines starting
� with '�' will be ignored, and an empty message aborts the commit.
�
� interactive rebase in progress; onto 4b284c5
� Last commands done (2 commands done):
�    pick a47f4e6 Draft to allow run in FIPS compliace mode
�    pick 0bee0a8 make tests run without BC (not BCFIPS) libraries.
� Next commands to do (8 remaining commands):
�    pick 4fc6201 disable approved-only mode for launch configuration of testcluster
�    pick 321929f update all BC libraries to support JAVA 21
� You are currently rebasing branch 'fips_compliance2' on '4b284c54270'.
�
� Changes to be committed:
�	modified:   buildSrc/build.gradle
�	modified:   buildSrc/src/main/java/org/opensearch/gradle/OpenSearchTestBasePlugin.java
�	modified:   buildSrc/src/main/java/org/opensearch/gradle/info/BuildParams.java
�	modified:   client/rest/build.gradle
�	new file:   client/rest/licenses/bc-fips-1.0.2.4.jar.sha1
�	new file:   client/rest/licenses/bctls-fips-1.0.19.jar.sha1
�	new file:   client/rest/licenses/bouncycastle-LICENSE.txt
�	new file:   client/rest/licenses/bouncycastle-NOTICE.txt
�	modified:   client/rest/src/test/java/org/opensearch/client/RestClientBuilderIntegTests.java
�	modified:   distribution/src/config/fips_java.security
�	modified:   distribution/tools/keystore-cli/src/test/java/org/opensearch/common/settings/AddFileKeyStoreCommandTests.java
�	modified:   distribution/tools/keystore-cli/src/test/java/org/opensearch/common/settings/AddStringKeyStoreCommandTests.java
�	modified:   distribution/tools/keystore-cli/src/test/java/org/opensearch/common/settings/ChangeKeyStorePasswordCommandTests.java
�	modified:   distribution/tools/keystore-cli/src/test/java/org/opensearch/common/settings/KeyStoreWrapperTests.java
�	modified:   distribution/tools/keystore-cli/src/test/java/org/opensearch/common/settings/ListKeyStoreCommandTests.java
�	modified:   distribution/tools/keystore-cli/src/test/java/org/opensearch/common/settings/RemoveSettingKeyStoreCommandTests.java
�	modified:   distribution/tools/launchers/src/main/java/org/opensearch/tools/launchers/SystemJvmOptions.java
�	modified:   distribution/tools/plugin-cli/build.gradle
�	modified:   gradle/libs.versions.toml
�	modified:   libs/ssl-config/build.gradle
�	deleted:    libs/ssl-config/licenses/bc-fips-1.0.2.5.jar.sha1
�	new file:   libs/ssl-config/licenses/bouncycastle-LICENSE.txt
�	new file:   libs/ssl-config/licenses/bouncycastle-NOTICE.txt
�	modified:   libs/ssl-config/src/main/java/org/opensearch/common/ssl/DefaultJdkTrustConfig.java
�	modified:   libs/ssl-config/src/main/java/org/opensearch/common/ssl/PemUtils.java
�	modified:   libs/ssl-config/src/test/java/org/opensearch/common/ssl/PemKeyConfigTests.java
�	modified:   libs/ssl-config/src/test/java/org/opensearch/common/ssl/PemTrustConfigTests.java
�	modified:   libs/ssl-config/src/test/java/org/opensearch/common/ssl/PemUtilsTests.java
�	modified:   modules/reindex/src/test/java/org/opensearch/index/reindex/ReindexRestClientSslTests.java
�	modified:   modules/transport-netty4/build.gradle
�	modified:   modules/transport-netty4/src/test/java/org/opensearch/http/netty4/ssl/SecureNetty4HttpServerTransportTests.java
�	modified:   modules/transport-netty4/src/test/java/org/opensearch/transport/netty4/ssl/SimpleSecureNetty4TransportTests.java
�	deleted:    modules/transport-netty4/src/test/resources/netty4-secure.jks
�	new file:   modules/transport-netty4/src/test/resources/netty4-secure.p12
�	modified:   plugins/discovery-azure-classic/src/internalClusterTest/java/org/opensearch/discovery/azure/classic/AzureDiscoveryClusterFormationTests.java
�	deleted:    plugins/identity-shiro/licenses/bcprov-jdk18on-1.78.jar.sha1
�	deleted:    plugins/identity-shiro/licenses/bcprov-jdk18on-LICENSE.txt
�	new file:   plugins/identity-shiro/licenses/password4j-1.8.2.jar.sha1
�	new file:   plugins/identity-shiro/licenses/password4j-LICENSE.txt
�	renamed:    plugins/identity-shiro/licenses/bcprov-jdk18on-NOTICE.txt -> plugins/identity-shiro/licenses/password4j-NOTICE.txt
�	modified:   plugins/identity-shiro/src/main/java/org/opensearch/identity/shiro/realm/BCryptPasswordMatcher.java
�	modified:   plugins/repository-azure/build.gradle
�	modified:   plugins/telemetry-otel/build.gradle
�	modified:   server/build.gradle
�	new file:   server/licenses/bc-fips-1.0.2.4.jar.sha1
�	new file:   server/licenses/bctls-fips-1.0.19.jar.sha1
�	new file:   server/licenses/bouncycastle-LICENSE.txt
�	new file:   server/licenses/bouncycastle-NOTICE.txt
�	modified:   server/src/main/java/org/opensearch/bootstrap/Bootstrap.java
�	modified:   server/src/main/java/org/opensearch/common/settings/FipsSettings.java
�	modified:   server/src/main/java/org/opensearch/common/settings/KeyStoreWrapper.java
�	modified:   server/src/main/resources/org/opensearch/bootstrap/security.policy
�	modified:   server/src/main/resources/org/opensearch/bootstrap/test-framework.policy
�
@iigonin
disable approved-only mode for launch configuration of testcluster
c2503cd 
Signed-off-by: Iwan Igonin <[email protected]>

# Conflicts:
#	buildSrc/version.properties
@iigonin
update all BC libraries to support JAVA 21
fde0cb9 
Signed-off-by: Iwan Igonin <[email protected]>
@iigonin
try to pass bwc staged build by adding expected version
ad2cc1c 
Signed-off-by: Iwan Igonin <[email protected]>
@iigonin
ensure FIPS-mode can be enabled by fixing existent and creating addit…
89620fe 
…ional tests.

Signed-off-by: Iwan Igonin <[email protected]>
@iigonin
fixing additional tests.
cd568e0 
Signed-off-by: Iwan Igonin <[email protected]>
@iigonin
Configure FIPS mode through global ENV VAR instead of thread-based.
8cac484 
Signed-off-by: Iwan Igonin <[email protected]>
@iigonin
write some additional unit tests.
b15d24d 
Signed-off-by: Iwan Igonin <[email protected]>
@iigonin
test SystemJvmOptions.java & replace FIPS-140-2 by FIPS-140-3
97cb892 
Signed-off-by: Iwan Igonin <[email protected]>
@iigonin
categorize JKS keystore as untrusted
20bbc77 
Signed-off-by: Iwan Igonin <[email protected]>
@iigonin
general exclusion of keystore binaries and certificates from fobidden…
00ca592 
…Pattern

Signed-off-by: Iwan Igonin <[email protected]>
@iigonin
Kerberos, forbiddenApis, SecureRandom, SunJCE, AzureTests
21dd2b2 
Summery:
- replace unsecure kerberos crypto algorithms
- add 'java.security.KeyStore' to forbidden-apis
- instantiate and use SecureRandom from BCFIPS library
- exclude SunJCE from security providers list at runtime, when running in FIPS JVM
- exclude Azure tests when running in FIPS JVM

Signed-off-by: Iwan Igonin <[email protected]>
@iigonin
adapt ITs to run on docker from this decade + debug + local test fixes
4c551aa
@beanuwave beanuwave force-pushed the fips_compliance_internal branch from abdabf3 to 4c551aa Compare December 13, 2024 09:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@beanuwave @iigonin