Regression in 3.1.4: customizations to the CuratorFramework instance are now silently ignored

[pantherdd]

Describe the bug

spring-cloud-zookeeper release 3.1.4 introduced a regression. There were ways to customize the CuratorFramework instance that got used, e.g. by injecting a CuratorFrameworkCustomizer via a registry.registerIfAbsent call in a custom BootstrapRegistryInitializer that runs before ZookeeperConfigServerBootstrapper. Such customizations worked flawlessly until version 3.1.3, but in 3.1.4 they are now silently ignored.

It doesn't look like there is an easy way to apply such customizations in the new version, other than copying the old registry.registerIfAbsent(ConfigServerInstanceProvider.Function.class, ...) code from the PR and overriding the new ZookeeperFunction::create implementation with it.

This change should be reverted; even if it makes sense to do this, it most definitely doesn't belong in a patch release.
Thanks.

[spencergibb]

It was part of spring-cloud/spring-cloud-commons#1228 and spring-cloud/spring-cloud-config#1922

@ryanjbaxter any ideas?

[ryanjbaxter]

Can you provide an sample illustrating the problem? I see the problem you are describing, but I would like to have something concrete to work with that I can test a solution against.

[pantherdd]

I don't have anything handy that I could share, it's proprietary code that got broken by the change. If necessary, I can of course put some minimal reproduction together, but it's going to take some time and it won't look like it makes sense (I'd just be injecting some meaningless subclasses of CuratorFrameworkCustomizer, EnsembleProvider, etc. using your library's 3.1.3 release). Would that be helpful?

[ryanjbaxter]

Yeah it doesnt have to do anything useful, just show that it was used in 3.1.3, but is no longer used in 3.1.4 (it could just print something to the console to show that)

[pantherdd]

@ryanjbaxter It wasn't too straightforward, but I managed to build a reproduction project:
https://github.com/pantherdd/spring-cloud-zookeeper-issue-324

How to use:

1. Start a local ZooKeeper instance (I used latest 3.9.1 from ZooKeeper Getting Started Guide)
2. Build both subprojects (gradle installDist)
3. Start config-server (java -jar config-server\build\install\config-server\config-server.jar)
   • You should see a JSON containing "message":"Hello from Spring Cloud Config" at http://localhost:8065/config-client/default
4. Start config-client (java -jar config-client\build\install\config-client\config-client.jar)
   • You should see Hello from Spring Cloud Config at http://localhost:8066/message

Using spring-cloud-dependencies version 2021.0.7, everything works fine. But if you upgrade config-client to 2021.0.8, then rebuild and rerun it, you will see that:

• It "hangs" for a few seconds before the app starts, because the new code is trying to connect to local.zookeeper.invalid
• The log lines written (to System.out for simplicity) by the My* classes only start after the Spring logo
• Spring starts printing a lot of errors, e.g. Error fetching config server instance from Zookeeper and Unable to resolve address: local.zookeeper.invalid/<unresolved>:2181
• Calling the /message REST API results in No message

---

I injected a custom implementation of everything into the bootstrap context that I know were picked up until 3.1.3:

• org.springframework.cloud.zookeeper.ZookeeperProperties
• org.apache.curator.RetryPolicy
• org.springframework.cloud.zookeeper.CuratorFrameworkCustomizer
• org.apache.curator.ensemble.EnsembleProvider
• org.apache.curator.drivers.TracerDriver

[spencergibb]

Thanks @pantherdd for taking the time to make the reproducer!

[ryanjbaxter]

Agree it was super helpful!

[pantherdd]

And thanks to you both for the quick fix. :)

However I'm a bit confused with regards to versions. The changes were merged today, and the target milestone is set as 3.1.5, but that seems to have been released yesterday without this:

• https://mvnrepository.com/artifact/org.springframework.cloud/spring-cloud-dependencies/2021.0.9
• https://mvnrepository.com/artifact/org.springframework.cloud/spring-cloud-zookeeper-dependencies/3.1.5
• v3.1.4...v3.1.5
• https://spring.io/blog/2023/12/20/spring-cloud-2021-0-9-aka-jubilee-is-now-available/

Can you clarify? Thanks 🙇‍♂️

[ryanjbaxter]

That was my mistake it should have been 4.0.5

[pantherdd]

Ah, I see. Is there a plan to backport this to 3.1.x too, given that it's a bug fix for a regression in the penultimate release of this major version? Or if that's not possible because the solution in #325 is dependent on something new in Spring 6 / Boot 3 / Cloud 4, what are our options (assuming we can't upgrade everything to the new major versions just yet)?

[ryanjbaxter]

The 3.1.x branch is no longer under open source support. We did the final release yesterday (even though it was no longer supported) because we had enough fixes in various projects across the 2021.0.x release train to warrant doing a release for a number of projects. At this point it is recommended that you begin to upgrade to 2022.0.x or 2023.0.x.

[pantherdd]

Understood, and that is of course in our plans.

Given the above, the only alternatives I see right now for projects that are affected by this bug and that still use the 2021.0.x release train, is to either stay on the last working version which is 2021.0.7 (or more specifically Spring Cloud Zookeeper 3.1.3), or to implement some local workaround for the regression (e.g. by overriding the ConfigServerInstanceProvider.Function bean). None of these are ideal, but given that this release train is now frozen, there should at least be no risk of missing out on improvements (if choosing to stay on 3.1.3) or incompatible changes breaking the workaround (if choosing to implement one). Obviously, upgrading to the next major version is preferred.

[ryanjbaxter]

Correct. And of course we could consider a release if there was a huge demand for a fix, but as of right now there is not

[pantherdd]

For the record, the final fix for this is in #328.
Some more context can be found in #325 and #327, but the code in them is superseded by #328.

[pantherdd]

@ryanjbaxter, I just noticed that when CuratorFactory.registerCurator is registering the ZookeeperProperties bean into the bootstrap context, it's still trying to use the Binder in there to populate it with the user-configured values. Is that code correct as-is, or should that also be changed to rely on PropertyResolver?

• https://github.com/spring-cloud/spring-cloud-zookeeper/blob/v4.0.2/spring-cloud-zookeeper-core/src/main/java/org/springframework/cloud/zookeeper/CuratorFactory.java#L94-L99
• https://github.com/spring-cloud/spring-cloud-zookeeper/blob/v4.0.2/spring-cloud-zookeeper-core/src/main/java/org/springframework/cloud/zookeeper/CuratorFactory.java#L130-L131

[ryanjbaxter]

@pantherdd I agree. The problem is we cannot use PropertyResolver in CuratorFactory since CuratorFactory resides in a module which does not have a dependency on Spring Cloud Config. I think we can work around this by registering ZookeeperProperties in ZookeeperConfigServerBootstrapper before calling registerCurator. See the PR #331