Tools and utilities for supporting AIDA evaluations.
Tools and utilities for Phase-II evaluation
Name
Description
Location
Latest release branch
AIDA-Evaluation Docker
Tool for applying the entire evaluation pipeline to task1 submissions
tools/aida-evaluation
AIDAED-v2020.0.1
Tools and utilities for Phase-I evaluation conducted at the end of 2019
Name
Description
Location
Latest release branch
SPAQRL Query Application Docker
The tool for applying SPARQL queries to KBs
tools/sparql-query-application
AIDAQA-v2019.2.5
AIDA-ValidateResponses
Tool for validating responses obtained after applying SPARQL queries to KBs
tools/validate-responses
AIDAVR-v2019.0.3
AIDA-ConfidenceAggregation
Tool (and docker) for aggregating confidences
tools/confidence-aggregation
AIDACA-v2019.0.2
AIDA-ScoreResponses
Tool for scoring responses
tools/scorer
AIDASR-v2019.2.1
AIF Generator
Tool for converting annotations into AIF
python/generate_aif.py
AIDAAG-v2019.0.1
AIDA-Evaluation Docker
Tool for applying the entire evaluation pipeline
tools/aida-evaluation
AIDAED-v2019.0.1
Tools and utilities for 2018 (M9) evaluation
Name
Description
utils/translate-annotations/seedling
Utility to translate LDC's seedling annotation to turtle-RDF format
tools/resolve-queries
Tool for applying SPARQL queries to a knowledge base in AIF format in order to produce output in the simplified xml response format
tools/validate-responses
Tool for validating responses for AIDA M9 evaluation
Please refer to tool/utility specific README file(s) for more details.