Skip to content

feat: add GeneralizedEsd, Rosner's test for up to r outliers - #7612

Merged
DenizAltunkapan merged 1 commit into
TheAlgorithms:masterfrom
alxkm:feat/generalized-esd
Sep 22, 2026
Merged

DenizAltunkapan merged 1 commit into
TheAlgorithms:masterfrom
alxkm:feat/generalized-esd

Conversation

@alxkm

@alxkm alxkm commented Sep 21, 2026

Copy link
Copy Markdown
Member

Adds the generalized extreme Studentized deviate test of Rosner, which finds up to r outliers in a sample without being told how many are there.

Grubbs' test asks whether the single most extreme point is an outlier, and it breaks as soon as there are two of them: each one inflates the standard deviation the other is measured against, so a pair of outliers can hide one another completely. This is the masking problem, and the generalized test is the standard answer to it. It removes the most extreme point, recomputes the statistics on what is left, and repeats r times:

R_i      = max |x - mean| / sd            over the points that are still in
lambda_i = (n - i) * t / sqrt( (n - i - 1 + t^2)(n - i + 1) )
where t  = the 1 - alpha / (2(n - i + 1)) quantile of a t distribution with n - i - 1 degrees of freedom

The number of outliers is the largest i whose statistic exceeds its critical value, not the first one. That is what defeats masking: in a sample with three outliers the first two tests can easily fall short while the third one succeeds, and the test then reports all three.

The critical values need a Student t quantile, which the repository does not have anywhere. Rather than ship a table, the class computes it: the t distribution function comes from the regularized incomplete beta function, evaluated with the continued fraction of Lentz over a Lanczos log gamma, and the quantile is recovered by bisection. That keeps the class self contained and lets it answer for any alpha and any sample size instead of the handful a table would cover.

The class is a batch test and needs the whole sample, which is the trade it makes against the streaming detectors already in the package: it has a stated significance level and it decides how many points are outliers, where HampelFilter answers one sample at a time against a threshold the caller has to choose. Both Javadocs point at each other.

GeneralizedEsdTest covers 23 cases. The two that matter most reproduce the worked example Rosner published, which is also the one in the NIST engineering statistics handbook: for its 54 points at alpha = 0.05 the critical values come out as 3.159, 3.151, 3.144 and 3.085, and the test reports exactly three outliers. Among the others: three outliers planted close together are all found, where a single-outlier test would stop at the first; a stricter significance level never reports more outliers than a looser one; a constant sample has none; the bound on r is respected; outliers on both sides are found; and the input array is left untouched. One test states the honest part out loud - at alpha = 0.05 a clean sample does report the odd outlier, so it checks both that this happens and that it stays rare, and that a level of 0.001 leaves the same samples alone.

Checklist

  • I have read CONTRIBUTING.md.
  • This pull request is all my own work -- I have not plagiarized it.
  • All filenames are in PascalCase.
  • All functions and variable names follow Java naming conventions.
  • All new algorithms have a URL in their comments that points to Wikipedia or other similar explanations.
  • All new algorithms include a corresponding test class that validates their functionality.
  • All new code is formatted with clang-format -i --style=file path/to/your/file.java

Grubbs' test asks whether the single most extreme point is an outlier and breaks as soon as there are two of them, because each one inflates the standard deviation the other is measured against. The generalized extreme Studentized deviate test removes the most extreme point, recomputes the statistics on what is left and repeats r times, and the answer is the largest step whose statistic exceeds its critical value rather than the first, which is what defeats that masking.

The critical values need a Student t quantile, which the repository did not have, so the class computes it from the regularized incomplete beta function through a Lentz continued fraction and a Lanczos log gamma, inverted by bisection. No table is needed. The tests check the critical values and the number of outliers against Rosner's published 54 point example.

Signed-off-by: alxkm <19151554+alxkm@users.noreply.github.com>
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 81.88976% with 23 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.17%. Comparing base (6bdb1ac) to head (9c1f0fe).

Files with missing lines Patch % Lines
...va/com/thealgorithms/streaming/GeneralizedEsd.java 81.88% 10 Missing and 13 partials ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master    #7612      +/-   ##
============================================
- Coverage     81.18%   81.17%   -0.02%     
- Complexity     7863     7897      +34     
============================================
  Files           829      830       +1     
  Lines         24885    25012     +127     
  Branches       4855     4882      +27     
============================================
+ Hits          20203    20303     +100     
- Misses         3910     3922      +12     
- Partials        772      787      +15     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@DenizAltunkapan
DenizAltunkapan merged commit 87edc62 into TheAlgorithms:master Sep 22, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants