A list containing current accuracy information.

accu contains current accuracy information returned by the corresponding generating function comp_accu_prob.

Usage

accu

Format

An object of class list of length 5.

Details

Current metrics include:

1. acc: Overall accuracy as the probability (or proportion) of correctly classifying cases or of dec_cor cases:

   See acc for definition and explanations.

   acc values range from 0 (no correct prediction) to 1 (perfect prediction).

2. wacc: Weighted accuracy, as a weighted average of the sensitivity sens (aka. hit rate HR, TPR, power or recall) and the the specificity spec (aka. TNR) in which sens is multiplied by a weighting parameter w (ranging from 0 to 1) and spec is multiplied by w's complement (1 - w):

   wacc = (w * sens) + ((1 - w) * spec)

   If w = .50, wacc becomes balanced accuracy bacc.

3. mcc: The Matthews correlation coefficient (with values ranging from -1 to +1):

   mcc = ((hi * cr) - (fa * mi)) / sqrt((hi + fa) * (hi + mi) * (cr + fa) * (cr + mi))

   A value of mcc = 0 implies random performance; mcc = 1 implies perfect performance.

   See Wikipedia: Matthews correlation coefficient for additional information.

4. f1s: The harmonic mean of the positive predictive value PPV (aka. precision) and the sensitivity sens (aka. hit rate HR, TPR, power or recall):

   f1s = 2 * (PPV * sens) / (PPV + sens)

   See Wikipedia: F1 score for additional information.

Notes:

• Accuracy metrics describe the correspondence of decisions (or predictions) to actual conditions (or truth).

  There are several possible interpretations of accuracy:

  1. as probabilities (i.e., acc being the probability or proportion of correct classifications, or the ratio dec_cor/N),

  2. as frequencies (e.g., as classifying a population of N individuals into cases of dec_cor vs. dec_err),

  3. as correlations (e.g., see mcc in accu).

• Computing exact accuracy values based on probabilities (by comp_accu_prob) may differ from accuracy values computed from (possibly rounded) frequencies (by comp_accu_freq).

  When frequencies are rounded to integers (see the default of round = TRUE in comp_freq and comp_freq_prob) the accuracy metrics computed by comp_accu_freq correspond to these rounded values. Use comp_accu_prob to obtain exact accuracy metrics from probabilities.

See also

The corresponding generating function comp_accu_prob computes exact accuracy metrics from probabilities; acc defines accuracy as a probability; comp_accu_freq computes accuracy metrics from frequencies; num for basic numeric parameters; freq for current frequency information; prob for current probability information; txt for current text settings.

Other lists containing current scenario information: freq, num, pal, pal_bw, pal_bwp, pal_kn, pal_mbw, pal_mod, pal_org, pal_rgb, pal_unikn, pal_vir, prob, txt, txt_TF, txt_org

Other metrics: acc, comp_acc(), comp_accu_freq(), comp_accu_prob(), comp_err(), err

Examples

accu <- comp_accu_prob()  # => compute exact accuracy metrics (from probabilities)
accu                      # => current accuracy information
#> $acc
#> [1] 0.775
#> 
#> $w
#> [1] 0.5
#> 
#> $wacc
#> [1] 0.8
#> 
#> $mcc
#> [1] 0.5303301
#> 
#> $f1s
#> [1] 0.6538462
#> 

## Contrasting comp_accu_freq and comp_accu_prob:
# (a) comp_accu_freq (based on rounded frequencies):
freq1 <- comp_freq(N = 10, prev = 1/3, sens = 2/3, spec = 3/4)   # => rounded frequencies!
accu1 <- comp_accu_freq(freq1$hi, freq1$mi, freq1$fa, freq1$cr)  # => accu1 (based on rounded freq).
# accu1
#
# (b) comp_accu_prob (based on probabilities):
accu2 <- comp_accu_prob(prev = 1/3, sens = 2/3, spec = 3/4)      # => exact accu (based on prob).
# accu2
all.equal(accu1, accu2)  # => 4 differences!
#> [1] "Component “acc”: Mean relative difference: 0.03174603" 
#> [2] "Component “wacc”: Mean relative difference: 0.02586207"
#> [3] "Component “mcc”: Mean relative difference: 0.1306675"  
#> [4] "Component “f1s”: Mean relative difference: 0.07692308" 
#
# (c) comp_accu_freq (exact values, i.e., without rounding):
freq3 <- comp_freq(N = 10, prev = 1/3, sens = 2/3, spec = 3/4, round = FALSE)
accu3 <- comp_accu_freq(freq3$hi, freq3$mi, freq3$fa, freq3$cr)  # => accu3 (based on EXACT freq).
# accu3
all.equal(accu2, accu3)  # => TRUE (qed).
#> [1] TRUE