Extend Builder to support abstract and concrete codegen, and add validation of required attributes during builder().build() (#5228)

* add validation to Builder

* address most review comments from Tomas

* addressing other review comments

Author: trentjeff

bom/pom.xml

@@ -1406,22 +1406,17 @@
             <!-- Pico Builder -->
             <dependency>
                 <groupId>io.helidon.pico.builder</groupId>
-                <artifactId>helidon-pico-builder-api</artifactId>
+                <artifactId>helidon-pico-builder</artifactId>
                 <version>${helidon.version}</version>
             </dependency>
             <dependency>
                 <groupId>io.helidon.pico.builder</groupId>
-                <artifactId>helidon-pico-builder-spi</artifactId>
+                <artifactId>helidon-pico-builder-processor-spi</artifactId>
                 <version>${helidon.version}</version>
             </dependency>
             <dependency>
                 <groupId>io.helidon.pico.builder</groupId>
-                <artifactId>helidon-pico-builder-tools</artifactId>
-                <version>${helidon.version}</version>
-            </dependency>
-            <dependency>
-                <groupId>io.helidon.pico.builder</groupId>
-                <artifactId>helidon-pico-builder-runtime-tools</artifactId>
+                <artifactId>helidon-pico-builder-processor-tools</artifactId>
                 <version>${helidon.version}</version>
             </dependency>
             <dependency>

common/testing/junit5/src/main/java/io/helidon/common/testing/junit5/OptionalMatcher.java

@@ -36,7 +36,7 @@ private OptionalMatcher() {
      * <p>
      * Usage example:
      * <pre>
-     *     assertThat(myOptional, present(is("expected")));
+     *     assertThat(myOptional, optionalValue(is("expected")));
      * </pre>
      * @param matcher matcher to validate the content of the optional
      * @param <T> type of the value
@@ -51,7 +51,7 @@ public static <T> Matcher<Optional<T>> optionalValue(Matcher<? super T> matcher)
      * <p>
      * Usage example:
      * <pre>
-     *     assertThat(myOptional, empty());
+     *     assertThat(myOptional, optionalEmpty());
      * </pre>
      * @param <T> type of the optional
      * @return matcher validating the {@link java.util.Optional} is empty
@@ -66,7 +66,7 @@ public static <T> Matcher<Optional<T>> optionalEmpty() {
      * <p>
      * Usage example:
      * <pre>
-     *     assertThat(myOptional, present());
+     *     assertThat(myOptional, optionalPresent());
      * </pre>
      * @param <T> type of the value
      * @return matcher validating the {@link java.util.Optional} is present

pico/builder/README.md

@@ -1,16 +1,16 @@
 # pico-builder
 
-The <b>Pico Builder</b> provides compile-time code generation for fluent builders. It was inspired by [Lombok]([https://projectlombok.org/), but the implementation here in Helidon is different in a few ways:
+The <b>Helidon Pico Builder</b> provides compile-time code generation for fluent builders. It was inspired by [Lombok]([https://projectlombok.org/), but the implementation here in Helidon is different in a few ways:
 <ol>
-    <li>Pico Builders target interface or annotation types only.</li>
-    <li>Generated classes implement the target interface (or annotation) and provide a fluent builder that will always have an implementation of <i>toString(), hashCode(), and equals().</i></li>
+    <li>The <i>Builder</i> annotation targets interface or annotation types only. Your interface effectively contains the attributes of your getter as well as serving as the contract for your getter methods.</li>
+    <li>Generated classes implement your target interface (or annotation) and provide a fluent builder that will always have an implementation of <i>toString(), hashCode(), and equals().</i> implemented</li>
     <li>Generated classes always behave like a <i>SuperBuilder</i> from Lombok. Basically this means that builders can form
       a hierarchy on the types they target (e.g., Level2 derives from Level1 derives from Level0, etc.).</li>
-    <li>Lombok uses AOP while the Pico Builder generates source code (in conjunction with the proper Builder annotation processor).</li>
-    <li>Pico Builders are extensible - you can provide your own implementation of the SPI to customize the generated classes.</li>
+    <li>Lombok uses AOP while the Pico Builder generates source code. You can use the <i>Builder</i> annotation (as well as other annotations in the package and <i>ConfiguredOption</i>) to control the naming and other features of what and how the implementation classes are generated and behave.</li>
+    <li>Pico Builders are extensible - you can provide your own implementation of the <b>Builder Processor SPI</b> to customize the generated classes for your situation.</li>
 </ol>
 
-Supported annotation types (see [api](./api/src/main/java/io/helidon/pico/builder/api) for further details):
+Supported annotation types (see [builder](./builder/src/main/java/io/helidon/pico/builder) for further details):
 * Builder - similar to Lombok's SuperBuilder.
 * Singular - similar to Lombok's Singular.
 
@@ -20,9 +20,9 @@ are ignored on the target being processed.
 
 The Helidon Pico Builder is completely independent of other parts of Pico. It can therefore be used in a standalone manner. The
 generated implementation class will not require any special module to support those classes - just the types from your interface
-and standard JRE types are used.
+and standard JRE types are used. This is made possible when your <i>Builder</i> has the <i>requireBuilderLibrary=false</i>. See the javadoc for details.
 
-## Usage
+## Getting Started
 1. Write your interface that you want to have a builder for.
 ```java
 public interface MyConfigBean {
@@ -31,28 +31,30 @@ public interface MyConfigBean {
     int getPort();
 }
 ```
-2. Annotate your interface definition with <i>Builder</i>, and optionally use <i>ConfiguredOption</i>, <i>Singular</i>, etc.
+2. Annotate your interface definition with <i>Builder</i>, and optionally use <i>ConfiguredOption</i>, <i>Singular</i>, etc. Remember to review the annotation attributes javadoc for any customizations.
 3. Compile (using the <i>pico-builder-processor</i> in your annotation classpath).
 
-The result of this will create (under ./target/generated-sources):
+The result of this will create (under ./target/generated-sources/annotations):
 * MyConfigBeanImpl (in the same package as MyConfigBean) that will support multi-inheritance builders named MyConfigBeanImpl.Builder.
 * Support for toString(), hashCode(), and equals() are always included.
 * Support for toBuilder().
-* Support for streams (see javadoc for [Builder](./api/src/main/java/io/helidon/pico/builder/Builder.java)).
+* Support for streams (see javadoc for [Builder](./builder/src/main/java/io/helidon/pico/builder/Builder.java)).
+* Support for attribute visitors (see [test-builder](./tests/builder/src/main/java/io/helidon/pico/builder/test/testsubjects/package-info.java)).
+* Support for attribute validation (see ConfiguredOption#required() and [builder](./tests/builder/src/main/java/io/helidon/pico/builder/test/testsubjects/package-info.java)).
 
 The implementation of the processor also allows for a provider-based extensibility mechanism.
 
-## modules
-* [api](./api) - defines the compile-time annotations. Typically this module is only needed at compile time.
-* [spi](./spi) - defines the SPI runtime definitions used by builder tooling. Only needed at compile time.
-* [tools](./tools) - provides the creators / code generators. Only needed at compile time.
-* [runtime-tools](./runtime-tools) - provides optional runtime utility classes the can helpful at compile time as well as runtime.
-* [processor](./processor) - the annotation processor which delegates to the tools module for processing. Only needed at compile time.
-* [test-builder](./test-builder) - internal tests that can also serve as examples for usage.
-
-## provider
-To implement your own provider:
-* Write an implementation for <i>BuilderCreator</i> having a higher-than-default <i>Weighted</i> value as compared to <i>DefaultBuilderCreator</i>.
+## Modules
+* [builder](./builder) - provides the compile-time annotations, as well as optional runtime supporting types.
+* [processor-spi](./processor-spi) - defines the Builder Processor SPI runtime definitions used by builder tooling. This module is only needed at compile time.
+* [processor-tools](./processor-tools) - provides the concrete creators & code generators. This module is only needed at compile time.
+* [processor](./processor) - the annotation processor which delegates to the processor-tools module for the main processing logic. This module is only needed at compile time.
+* [tests/builder](./tests/builder) - internal tests that can also serve as examples on usage.
+
+## Customizations
+To implement your own custom <i>Builder</i>:
+* Write an implementation of <i>BuilderCreator</i> having a higher-than-default <i>Weighted</i> value as compared to <i>DefaultBuilderCreator</i>.
 * Include your module with this creator in your annotation processing path.
 
-See [test-builder](./test-builder) for usage examples.
+## Usage
+See [tests/builder](./tests/builder) for usage examples.

pico/builder/api/README.md

@@ -0,0 +1,5 @@
+# pico-builder
+
+This module can either be used compile-time only or at runtime as well depending upon your usage.
+
+See the [main](../README.md) document for details.

pico/builder/api/src/main/java/module-info.java

@@ -20,6 +20,7 @@
 <project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+
     <parent>
         <groupId>io.helidon.pico.builder</groupId>
         <artifactId>helidon-pico-builder-project</artifactId>
@@ -28,21 +29,23 @@
     </parent>
     <modelVersion>4.0.0</modelVersion>
 
-    <artifactId>helidon-pico-builder-spi</artifactId>
-    <name>Helidon Pico Builder SPI</name>
+    <artifactId>helidon-pico-builder</artifactId>
+    <name>Helidon Pico Builder</name>
 
     <dependencies>
         <dependency>
-            <groupId>io.helidon.common</groupId>
-            <artifactId>helidon-common</artifactId>
+            <groupId>io.helidon.config</groupId>
+            <artifactId>helidon-config-metadata</artifactId>
+            <scope>provided</scope>
         </dependency>
         <dependency>
-            <groupId>io.helidon.pico</groupId>
-            <artifactId>helidon-pico-types</artifactId>
+            <groupId>io.helidon.common</groupId>
+            <artifactId>helidon-common</artifactId>
         </dependency>
         <dependency>
-            <groupId>io.helidon.pico.builder</groupId>
-            <artifactId>helidon-pico-builder-api</artifactId>
+            <groupId>io.helidon.common.testing</groupId>
+            <artifactId>helidon-common-testing-junit5</artifactId>
+            <scope>test</scope>
         </dependency>
         <dependency>
             <groupId>org.hamcrest</groupId>

pico/builder/builder/README.md

@@ -14,7 +14,7 @@
  * limitations under the License.
  */
 
-package io.helidon.pico.builder.api;
+package io.helidon.pico.builder;
 
 import java.lang.annotation.ElementType;
 import java.lang.annotation.Retention;

pico/builder/spi/pom.xml renamed to pico/builder/builder/pom.xml

@@ -0,0 +1,47 @@
+/*
+ * Copyright (c) 2022 Oracle and/or its affiliates.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package io.helidon.pico.builder;
+
+import java.util.Map;
+import java.util.function.Supplier;
+
+/**
+ * A functional interface that can be used to visit all attributes of this type.
+ * <p>
+ * This type is used when {@link io.helidon.pico.builder.Builder#requireLibraryDependencies()} is used.
+ */
+@FunctionalInterface
+public interface AttributeVisitor {
+
+    /**
+     * Visits the attribute named 'attrName'.
+     *
+     * @param attrName          the attribute name
+     * @param valueSupplier     the attribute value supplier
+     * @param meta              the meta information for the attribute
+     * @param userDefinedCtx    a user defined context that can be used for holding an object of your choosing
+     * @param type              the type of the attribute
+     * @param typeArgument      the type arguments (if type is a parameterized / generic type)
+     */
+    void visit(String attrName,
+               Supplier<Object> valueSupplier,
+               Map<String, Object> meta,
+               Object userDefinedCtx,
+               Class<?> type,
+               Class<?>... typeArgument);
+
+}