Feature Test Summary for MicroProfile OpenAPI 4.1

[Azquelt]

• Feature: MicroProfile OpenAPI 4.1 #30717

Test Strategy

This is a very small release. Changes are minor and are adequately tested by the TCK. Our testing is therefore limited to repeating the existing tests against the new version and running the new TCK.

List of FAT projects affected

• Add MP OpenAPI 4.1 bundles and features #31296
  • com.ibm.ws.microprofile.openapi_fat
  • io.openliberty.microprofile.openapi.2.0.internal_fat
  • io.openliberty.microprofile.openapi.4.0.internal_fat
  • io.openliberty.jakartaee10.internal_fat
  • io.openliberty.jakartaee11.internal_fat
• MicroProfile OpenAPI 4.1 Implementation #31368
  • io.openliberty.microprofile.openapi.4.1.internal_fat_tck
• Fix OpenAPI UI for 3.1 documents + update Swagger UI to 5.22.0 #31712
  • io.openliberty.microprofile.openapi.ui.internal_fat

Test strategy

• New functionality
  • Addition of getJsonSchemaDialect, setJsonSchemaDialect and jsonSchemaDialect to the OpenAPI model interface
    • Tested reflectively in ModelConstructionTest
      • Test that getJsonSchemaDialect returns the same instance passed to setJsonSchemaDialect
      • Test that jsonSchemaDialect returns the model object
      • Test that getJsonSchemaDialect returns the same instance passed to jsonSchemaDialect
    • Test that the jsonSchemaDialect property can be set in an OASModelReader and is reflected in the output OpenAPI document
      • Tested in Add json schema dialect property microprofile/microprofile-open-api#675
  • Addition of getExtension and hasExtension to the Extensible model interface
    • For every model object which implements Extensible
      • Test that hasExtension returns true when the extension with the given name is present and false when it isn't
      • Test that getExtension returns the extension object when the extension with the given name is present and null when it isn't
    • Tested in Add Extensible.hasExtension and .getExtension microprofile/microprofile-open-api#678

• Existing functionality
  • Repeat existing OpenAPI FAT tests
    • Add MP OpenAPI 4.1 bundles and features #31296
  • Run new TCK test
    • MicroProfile OpenAPI 4.1 Implementation #31368
  • Repeat existing FAT tests for config validation integration
    • MP OpenAPI 4.1 integration with config validation #31608
  • Repeat existing UI tests with an OpenAPI 3.1 document
    • Fix OpenAPI UI for 3.1 documents + update Swagger UI to 5.22.0 #31712
    • Should really have done this last release
    • Note that UI is included in this feature, but is a separate component that's shared with all other OpenAPI features
    • Note that UI has a suite of manual tests run whenever we update it, as well as automated tests

Confidence Level

Collectively as a team you need to assess your confidence in the testing delivered based on the values below. This should be done as a team and not an individual to ensure more eyes are on it and that pressures to deliver quickly are absorbed by the team as a whole.

Please indicate your confidence in the testing (up to and including FAT) delivered with this feature by selecting one of these values:

4 - We have delivered all automated testing we believe is needed for the golden paths of this feature and have good coverage of the error/outlying scenarios. While more testing of the error/outlying scenarios could be added we believe there is minimal risk here and the cost of providing these is considered higher than the benefit they would provide.

[dave-waddling]

@Azquelt: Thanks for completing this FTS. As you point out, it's a fairly simple feature and the test coverage looks good. A few asks:

• In your demo during the EOI, when a jsonSchemaDialect is set a warning is displayed in the UI - does that have an automated or manual test?
• Please can you agree a confidence in the test coverage with your team.
• Please can you let me or one of the other reviewers know when the final API is pulled in.

I'll take a look at any outstanding defect work now, and reach out on Slack if there's anything needing attention.

[Azquelt]

In your demo during the EOI, when a jsonSchemaDialect is set a warning is displayed in the UI - does that have an automated or manual test?

We don't have a test for this. It's not designed behaviour, it's just a consequence of us using Swagger UI. We can add a test so that we notice if this behaviour changes, but I'd prefer to not gate approval of the feature behind that test being added. Task #31826 opened for this.

I will work on the other points.

[Azquelt]

@dave-waddling Final PR is in integration (#31814). Rest of the team is busy this morning so we don't have a confidence yet.

[Azquelt]

@dave-waddling We added a confidence level last week, sorry I forgot to update you.

[abutch3r]

SOE has run cleanly since the Final APIs were merged. all defects are triaged with no actual mpOpenAPI product bugs. As noted the dialogure for the jsonSchemaDialect is part of swagger-ui and is covered by their testing. the automated and manual UI tests check the modifications and core functionality of the page continue to work. Closing as Complete.