First set of indicators

[vuillaut]

We need to publish a first set of indicators

The tools and indicators that have been identified:

Howfairis

• Description: Checks compliance against the five recommendations from https://fair-software.eu/
• Dimension: FAIRness
• Indicator: hasLicense
• GitHub action: yes

Gitleaks

• Description: Detects leaks in repo
• Dimension: Security
• Indicator: hasSecurityLeak
• GitHub action: yes

CFFconvert

• Description: Converts CITATION.cff file to different formats and do validation of CITATION.cff file
• Dimension: FAIRness
• Indicator: hasCffFile
• GitHub action: yes

Super-linter

• Description: Programming language agnostic linter
• Dimension: Quality
• Indicator: hasLintingIssues
• GitHub action: yes

[dgarijo]

Maybe we can do also https://scorecard.dev/

[vuillaut]

Maybe we can do also https://scorecard.dev/

It's an interesting one, but let's focus on a few to test schemas and build something tangible first.

[kaygraf]

While implementing the pipeline, we came across the general issue that those indicators outcome needs interpretation.
E.g.
hasLicense = True is ok
hasLintingIssues = True is not ok
Maybe we should define that binary outcome True is ok, and if a scoring outcome 100 is good, 0 is bad (or similar)?

[dgarijo]

@kaygraf good point, but I think that is not interpretation, that is an explanation of the indicator. In the description instructions we set for indicators we said the description should contain:

### What is being measured?
    Explain what you are measuring
### Why should we measure it?
    Explain why
### What must be provided for the measurement? 
    For example, a zenodo record, or a GitHub id.
### How is the measurement executed?
    Explain the exact process for assessing the indicator
### What is/are considered valid result(s)?
    What outcomes are pass/fail

I believe the "what is/are considered valid results?" goes in the direction of your observation. I would leave it as part of the description, but we may attempt some representation.

IMO, the interpretation of the indicator depends of the community, or the use case. For example, some may consider that failing to have an open license is bad, while other communities may not be bothered.

[kaygraf]

Agreed, that this is necessarily interpretation based on community standards and use cases - but of course for the implementation, the somehow has to be machine actionable in the end, so we can provide a "pass/fail" at least for a standard set of indicators and tools.

[fdiblen]

We could also consider just renaming the indicator. For example, instead of hasLintingIssues we can call it

• hasNoLintingIssues
• noLintingIssues
• islintingValid
• lintingPasses
  I also agree with @kaygraf, hasLintingIssues = True is not ok can be confusing.