Accuracy, bear in mind and you will F1-score for various categories of elements removed by the dream operating tool resistant to the hand-coded establishes

cuatro.4. Evaluation

I evaluated all of our tool-using a couple groups of dream reports you to definitely had been hands-coded by the dream experts utilizing the Hallway–Van de- Castle system (§4.2.1): (i) the fresh annotated selection of dream reports, and you may (ii) this new normative put at which the latest norms found in the latest books was basically calculated. For all those dream account, we counted this new extent to which the brand new groups of letters, communication and you can thinking estimated of the fantasy operating unit coordinated brand new associated surface-insights kits; desk cuatro summarizes the latest ensuing precision, remember and you may F1-score.

I upcoming continued evaluate the latest brand new Hallway–Van de Palace indications determined from the all of our tool (desk step one) into involved surface-insights viewpoints. Given the floor-truth-value v therefore the tool’s worth v ? , we determined the newest mistake given that e = | v ? v ? | .

Full, the common mistake across the kinds is 0.24 (contour 3b), that is minimal because of the higher variability from textual appearances when you look at the brand new corpus, while the inherent complexity of some of one’s strategies. To help you interpret the magnitude of your own error, you should think one, used, all the evidence undertake beliefs that will be more often than not inside the the latest [0,1] variety on this subject certain sample group of fantasy reports. New measure you to deviates most from this diversity is the An effective / C Directory : it is greater than one in 6% of one’s instances on the floor-knowledge as well as in 3% of one’s cases considering our equipment. The newest A / C List , is even impacted by the highest error (age = 0.45).