-
Notifications
You must be signed in to change notification settings - Fork 42
Evaluation results (Tests, metrics)
Patrick Hammer edited this page Sep 27, 2020
·
15 revisions
Current state, master:
tc@box:~/master/OpenNARS-for-Applications$ python3.6 evaluation.py
<<NAR Follow test successful goods=503 bads=3 ratio=0.994071
System tests successful!
Now running Q&A experiments:
Q&A metrics for test ./examples/nal/school.nal
Average answer time = 140.5
Average answer confidence = 0.22967900000000002
Combined loss = 108.2301005
Q&A metrics for test ./examples/nal/asthma.nal
Average answer time = 184.0
Average answer confidence = 0.7416662500000001
Combined loss = 47.53340999999998
Q&A stress test results for test ./examples/nal/example1.nal
Total questions = 20.0
Correctly answered ones = 16.0
Answer ratio = 0.8
Q&A metrics for test ./examples/nal/symmetry.nal
Average answer time = 265.0
Average answer confidence = 0.726151
Combined loss = 72.569985
Narsese integration tests successful!
Q&A metrics for test ./examples/english/story3.english
Average answer time = 38.0
Average answer confidence = 0.5894055
Combined loss = 15.602590999999999
Q&A metrics for test ./examples/english/story2.english
Average answer time = 565.0
Average answer confidence = 0.540652
Combined loss = 259.53162
Q&A metrics for test ./examples/english/story1.english
Average answer time = 531.0
Average answer confidence = 0.543337
Combined loss = 242.488053
English integration tests successful!
Q&A metrics global
Average answer time = 273.0769230769231
Average answer confidence = 0.5768433076923076
Combined loss = 115.55432751479293
Q&A answer rate global
Total questions = 51.0
Correctly answered ones = 47.0
Answer ratio = 0.9215686274509803
Now running procedure learning examples for 10K iterations each:
Pong metrics: Hits=463 misses=107 ratio=0.812281 time=29346
Pong2 metrics: Hits=338 misses=4 ratio=0.988304 time=15263
Alien metrics: shots=4113 hits=3074 ratio=0.747386 time=32852
Cartpole metrics: successes=9887.000000, failures=115.000000, ratio=0.988502, time=10001
Robot metrics: time=1000 moves=300 move_success_ratio=0.300000 eaten=35 reasonerStep=3459
Procedure learning metrics done
Note: successful tests without metrics are not printed, but if they fail they would appear.