`comp_accu_freq`

computes a list of current accuracy metrics
from the 4 essential frequencies (`hi`

,
`mi`

, `fa`

, `cr`

)
that constitute the current confusion matrix and
are contained in `freq`

.

comp_accu_freq(hi = freq$hi, mi = freq$mi, fa = freq$fa, cr = freq$cr, w = 0.5)

hi | The number of hits |
---|---|

mi | The number of misses |

fa | The number of false alarms |

cr | The number of correct rejections |

w | The weighting parameter |

A list `accu`

containing current accuracy metrics.

Currently computed accuracy metrics include:

`acc`

: Overall accuracy as the proportion (or probability) of correctly classifying cases or of`dec_cor`

cases:`acc = dec_cor/N = (hi + cr)/(hi + mi + fa + cr)`

Values range from 0 (no correct prediction) to 1 (perfect prediction).

`wacc`

: Weighted accuracy, as a weighted average of the sensitivity`sens`

(aka. hit rate`HR`

,`TPR`

,`power`

or`recall`

) and the the specificity`spec`

(aka.`TNR`

) in which`sens`

is multiplied by a weighting parameter`w`

(ranging from 0 to 1) and`spec`

is multiplied by`w`

's complement`(1 - w)`

:`wacc = (w * sens) + ((1 - w) * spec)`

If

`w = .50`

,`wacc`

becomes*balanced*accuracy`bacc`

.`mcc`

: The Matthews correlation coefficient (with values ranging from -1 to +1):`mcc = ((hi * cr) - (fa * mi)) / sqrt((hi + fa) * (hi + mi) * (cr + fa) * (cr + mi))`

A value of

`mcc = 0`

implies random performance;`mcc = 1`

implies perfect performance.See Wikipedia: Matthews correlation coefficient for additional information.

`f1s`

: The harmonic mean of the positive predictive value`PPV`

(aka.`precision`

) and the sensitivity`sens`

(aka. hit rate`HR`

,`TPR`

,`power`

or`recall`

):`f1s = 2 * (PPV * sens) / (PPV + sens)`

See Wikipedia: F1 score for additional information.

Notes:

Accuracy metrics describe the

*correspondence*of decisions (or predictions) to actual conditions (or truth).There are several possible interpretations of accuracy:

Computing exact accuracy values based on probabilities (by

`comp_accu_prob`

) may differ from accuracy values computed from (possibly rounded) frequencies (by`comp_accu_freq`

).When frequencies are rounded to integers (see the default of

`round = TRUE`

in`comp_freq`

and`comp_freq_prob`

) the accuracy metrics computed by`comp_accu_freq`

correspond to these rounded values. Use`comp_accu_prob`

to obtain exact accuracy metrics from probabilities.

Consult Wikipedia: Confusion matrix for additional information.

`accu`

for all accuracy metrics;
`comp_accu_prob`

computes exact accuracy metrics from probabilities;
`num`

for basic numeric parameters;
`freq`

for current frequency information;
`txt`

for current text settings;
`pal`

for current color settings;
`popu`

for a table of the current population.

Other metrics: `accu`

, `acc`

,
`comp_accu_prob`

, `comp_acc`

,
`comp_err`

, `err`

Other functions computing probabilities: `comp_FDR`

,
`comp_FOR`

, `comp_NPV`

,
`comp_PPV`

, `comp_accu_prob`

,
`comp_acc`

, `comp_comp_pair`

,
`comp_complement`

,
`comp_complete_prob_set`

,
`comp_err`

, `comp_fart`

,
`comp_mirt`

, `comp_ppod`

,
`comp_prob_freq`

, `comp_prob`

,
`comp_sens`

, `comp_spec`

comp_accu_freq() # => accuracy metrics for freq of current scenario#> $acc #> [1] 0.774 #> #> $w #> [1] 0.5 #> #> $wacc #> [1] 0.7986667 #> #> $mcc #> [1] 0.5279731 #> #> $f1s #> [1] 0.6523077 #>comp_accu_freq(hi = 1, mi = 2, fa = 3, cr = 4) # medium accuracy, but cr > hi#> $acc #> [1] 0.5 #> #> $w #> [1] 0.5 #> #> $wacc #> [1] 0.452381 #> #> $mcc #> [1] -0.08908708 #> #> $f1s #> [1] 0.2857143 #># Extreme cases: comp_accu_freq(hi = 1, mi = 1, fa = 1, cr = 1) # random performance#> $acc #> [1] 0.5 #> #> $w #> [1] 0.5 #> #> $wacc #> [1] 0.5 #> #> $mcc #> [1] 0 #> #> $f1s #> [1] 0.5 #>comp_accu_freq(hi = 0, mi = 0, fa = 1, cr = 1) # random performance: wacc and f1s are NaN#> Warning: accu$mcc: A denominator of 0 was corrected to 1, resulting in mcc = 0.#> $acc #> [1] 0.5 #> #> $w #> [1] 0.5 #> #> $wacc #> [1] NaN #> #> $mcc #> [1] 0 #> #> $f1s #> [1] NaN #>comp_accu_freq(hi = 1, mi = 0, fa = 0, cr = 1) # perfect accuracy/optimal performance#> $acc #> [1] 1 #> #> $w #> [1] 0.5 #> #> $wacc #> [1] 1 #> #> $mcc #> [1] 1 #> #> $f1s #> [1] 1 #>comp_accu_freq(hi = 0, mi = 1, fa = 1, cr = 0) # zero accuracy/worst performance, but see f1s#> $acc #> [1] 0 #> #> $w #> [1] 0.5 #> #> $wacc #> [1] 0 #> #> $mcc #> [1] -1 #> #> $f1s #> [1] NaN #>comp_accu_freq(hi = 1, mi = 0, fa = 0, cr = 0) # perfect accuracy, but see wacc and mcc#> Warning: accu$mcc: A denominator of 0 was corrected to 1, resulting in mcc = 0.#> $acc #> [1] 1 #> #> $w #> [1] 0.5 #> #> $wacc #> [1] NaN #> #> $mcc #> [1] 0 #> #> $f1s #> [1] 1 #># Effects of w: comp_accu_freq(hi = 3, mi = 2, fa = 1, cr = 4, w = 1/2) # equal weights to sens and spec#> $acc #> [1] 0.7 #> #> $w #> [1] 0.5 #> #> $wacc #> [1] 0.7 #> #> $mcc #> [1] 0.4082483 #> #> $f1s #> [1] 0.6666667 #>comp_accu_freq(hi = 3, mi = 2, fa = 1, cr = 4, w = 2/3) # more weight to sens#> $acc #> [1] 0.7 #> #> $w #> [1] 0.6666667 #> #> $wacc #> [1] 0.6666667 #> #> $mcc #> [1] 0.4082483 #> #> $f1s #> [1] 0.6666667 #>comp_accu_freq(hi = 3, mi = 2, fa = 1, cr = 4, w = 1/3) # more weight to spec#> $acc #> [1] 0.7 #> #> $w #> [1] 0.3333333 #> #> $wacc #> [1] 0.7333333 #> #> $mcc #> [1] 0.4082483 #> #> $f1s #> [1] 0.6666667 #>## Contrasting comp_accu_freq and comp_accu_prob: # (a) comp_accu_freq (based on rounded frequencies): freq1 <- comp_freq(N = 10, prev = 1/3, sens = 2/3, spec = 3/4) # => hi = 2, mi = 1, fa = 2, cr = 5 accu1 <- comp_accu_freq(freq1$hi, freq1$mi, freq1$fa, freq1$cr) # => accu1 (based on rounded freq). # accu1 # # (b) comp_accu_prob (based on probabilities): accu2 <- comp_accu_prob(prev = 1/3, sens = 2/3, spec = 3/4) # => exact accu (based on prob). # accu2 all.equal(accu1, accu2) # => 4 differences!#> [1] "Component “acc”: Mean relative difference: 0.03174603" #> [2] "Component “wacc”: Mean relative difference: 0.02586207" #> [3] "Component “mcc”: Mean relative difference: 0.1306675" #> [4] "Component “f1s”: Mean relative difference: 0.07692308"# # (c) comp_accu_freq (exact values, i.e., without rounding): freq3 <- comp_freq(N = 10, prev = 1/3, sens = 2/3, spec = 3/4, round = FALSE) accu3 <- comp_accu_freq(freq3$hi, freq3$mi, freq3$fa, freq3$cr) # => accu3 (based on EXACT freq). # accu3 all.equal(accu2, accu3) # => TRUE (qed).#> [1] TRUE