HanLP Smatch提供错误和随机分数,

vfwfrxfs 于 5个月前发布在其他

关注(0)|答案(2)|浏览(102)

描述bug

正如原始Smatch仓库问题中指出的那样，Smatch分数给出错误且不可验证的结果。这对于HanLP也是如此。

重现问题的代码

s = """(r / result-01
:ARG1 (c / compete-01
:ARG0 (w / woman)
:mod (p / preliminary)
:time (t / today)
:mod (p2 / polo
:mod (w2 / water)))
:ARG2 (a / and
:op1 (d / defeat-01
:ARG0 (t2 / team
:mod (c2 / country
:wiki +
:name (n / name
:op1 "Hungary")))
:ARG1 (t3 / team
:mod (c3 / country
:wiki +
:name (n2 / name
:op1 "Canada")))
:quant (s / score-entity
:op1 13
:op2 7))
:op2 (d2 / defeat-01
:ARG0 (t4 / team
:mod (c4 / country
:wiki +
:name (n3 / name
:op1 "France")))
:ARG1 (t5 / team
:mod (c5 / country
:wiki +
:name (n4 / name
:op1 "Brazil")))
:quant (s2 / score-entity
:op1 10
:op2 9))
:op3 (d3 / defeat-01
:ARG0 (t6 / team
:mod (c6 / country
:wiki +
:name (n5 / name
:op1 "Australia")))
:ARG1 (t7 / team
:mod (c7 / country
:wiki +
:name (n6 / name
:op1 "Germany")))
:quant (s3 / score-entity
:op1 10
:op2 8))
:op4 (d4 / defeat-01
:ARG0 (t8 / team
:mod (c8 / country
:wiki +
:name (n7 / name
:op1 "Russia")))
:ARG1 (t9 / team
:mod (c9 / country
:wiki +
:name (n8 / name
:op1 "Netherlands")))
:quant (s4 / score-entity
:op1 7
:op2 6))
:op5 (d5 / defeat-01
:ARG0 (t10 / team
:mod (c10 / country
:wiki +
:name (n9 / name
:op1 "United"
:op2 "States")))
:ARG1 (t11 / team
:mod (c11 / country
:wiki +
:name (n10 / name
:op1 "Kazakhstan")))
:quant (s5 / score-entity
:op1 10
:op2 5))
:op6 (d6 / defeat-01
:ARG0 (t12 / team
:mod (c12 / country
:wiki +
:name (n11 / name
:op1 "Italy")))
:ARG1 (t13 / team
:mod (c13 / country
:wiki +
:name (n12 / name
:op1 "New"
:op2 "Zealand")))
:quant (s6 / score-entity
:op1 12
:op2 2))))
"""

if __name__ == "__main__":
     from hanlp.metrics.amr import smatch_eval
     path = "amr.tmp"
     with open(path, "w") as f:
         f.write(s)
     for _ in range(5):
        smatch_score = smatch_eval("amr.tmp", "amr.tmp")
        print(smatch_score)

当前行为描述

完全错误的随机Smatch分数。

预期行为

确定性的Smatch分数为100

系统信息