F-beta Score

fbeta.factor

R Documentation

Description

A generic function for the \(F_{\beta}\)-score. Use weighted.fbeta() for the weighted \(F_{\beta}\)-score.

Usage

## S3 method for class 'factor'
fbeta(actual, predicted, beta = 1, micro = NULL, na.rm = TRUE, ...)

## S3 method for class 'factor'
weighted.fbeta(actual, predicted, w, beta = 1, micro = NULL, na.rm = TRUE, ...)

## S3 method for class 'cmatrix'
fbeta(x, beta = 1, micro = NULL, na.rm = TRUE, ...)

## Generic S3 method
fbeta(
 ...,
 beta  = 1,
 micro = NULL,
 na.rm = TRUE
)

## Generic S3 method
weighted.fbeta(
 ...,
 w,
 beta = 1,
 micro = NULL,
 na.rm = TRUE
)

Arguments

`actual`	A vector of - of length \(n\), and \(k\) levels.
`predicted`	A vector of -vector of length \(n\), and \(k\) levels.
`beta`	A `<numeric>` vector of length \(1\) (default: \(1\)).
`micro`	A -value of length \(1\) (default: NULL). If TRUE it returns the micro average across all \(k\) classes, if FALSE it returns the macro average.
`na.rm`	A value of length \(1\) (default: TRUE). If TRUE, NA values are removed from the computation. This argument is only relevant when `micro != NULL`. When `na.rm = TRUE`, the computation corresponds to `sum(c(1, 2, NA), na.rm = TRUE) / length(na.omit(c(1, 2, NA)))`. When `na.rm = FALSE`, the computation corresponds to `sum(c(1, 2, NA), na.rm = TRUE) / length(c(1, 2, NA))`.
`…`	micro = NULL, na.rm = TRUE Arguments passed into other methods
`w`	A `<numeric>`-vector of length \(n\). NULL by default.
`x`	A confusion matrix created `cmatrix()`.

Value

If micro is NULL (the default), a named <numeric>-vector of length k

If micro is TRUE or FALSE, a <numeric>-vector of length 1

Definition

Let \(\hat{F}_{\beta} \in [0, 1]\) be the \(F_{\beta}\) score, which is a weighted harmonic mean of precision and recall. \(F_{\beta}\) score of the classifier is calculated as,

\[ \hat{F}_{\beta} = \left(1 + \beta^2\right) \frac{\text{Precision} \times \text{Recall}} {\beta^2 \times \text{Precision} + \text{Recall}} \]

Substituting \(\text{Precision} = \frac{\#TP_k}{\#TP_k + \#FP_k}\) and \(\text{Recall} = \frac{\#TP_k}{\#TP_k + \#FN_k}\) yields:

\[ \hat{F}_{\beta} = \left(1 + \beta^2\right) \frac{\frac{\#TP_k}{\#TP_k + \#FP_k} \times \frac{\#TP_k}{\#TP_k + \#FN_k}} {\beta^2 \times \frac{\#TP_k}{\#TP_k + \#FP_k} + \frac{\#TP_k}{\#TP_k + \#FN_k}} \]

Where:

\(\#TP_k\) is the number of true positives,
\(\#FP_k\) is the number of false positives,
\(\#FN_k\) is the number of false negatives, and
\(\beta\) is a non-negative real number that determines the relative importance of precision vs. recall in the score.

Examples

# 1) recode Iris
# to binary classification
# problem
iris$species_num <- as.numeric(
  iris$Species == "virginica"
)

# 2) fit the logistic
# regression
model <- glm(
  formula = species_num ~ Sepal.Length + Sepal.Width,
  data    = iris,
  family  = binomial(
    link = "logit"
  )
)

# 3) generate predicted
# classes
predicted <- factor(
  as.numeric(
    predict(model, type = "response") > 0.5
  ),
  levels = c(1,0),
  labels = c("Virginica", "Others")
)

# 3.1) generate actual
# classes
actual <- factor(
  x = iris$species_num,
  levels = c(1,0),
  labels = c("Virginica", "Others")
)

# 4) evaluate class-wise performance
# using F1-score

# 4.1) unweighted F1-score
fbeta(
  actual    = actual,
  predicted = predicted,
  beta      = 1
)

# 4.2) weighted F1-score
weighted.fbeta(
  actual    = actual,
  predicted = predicted,
  w         = iris$Petal.Length/mean(iris$Petal.Length),
  beta      = 1
)

# 5) evaluate overall performance
# using micro-averaged F1-score
cat(
  "Micro-averaged F1-score", fbeta(
    actual    = actual,
    predicted = predicted,
    beta      = 1,
    micro     = TRUE
  ),
  "Micro-averaged F1-score (weighted)", weighted.fbeta(
    actual    = actual,
    predicted = predicted,
    w         = iris$Petal.Length/mean(iris$Petal.Length),
    beta      = 1,
    micro     = TRUE
  ),
  sep = "\n"
)