Spectral Clustering #1425

aamijar · 2025-10-14T15:50:23Z

Resolves #1201

This PR introduces the Spectral Clustering algorithm which builds on top of Spectral Embedding.

This PR will enable us to remove the legacy and inaccurate spectral clustering algorithm called partition

This PR also adds double precision support for spectral embedding and spectral clustering. The new templates require a refactor of the source files to .cuh and the template instantiations in .cu

Gtests have been added to this PR with synthetic data in a similar fashion as kmeans gtests.
I have tested this PR through python bindings and by comparing the plots with sklearn's SpectralClustering.

Here is a plot of sklearn vs ours.

copy-pr-bot · 2025-10-14T15:50:27Z

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

aamijar · 2025-10-15T20:17:05Z

Update:
I have added the current version of the gtests, which has a similar structure to how kmeans.cu does it.
I had to reduce the ARI threshold score.
I was able to setup this same test in python as well to check what happens with sklearn SpectralClustering when you pass in this synthetic data. However, it seems that sklearn gives a perfect 1.0 ARI. So there must be something going wrong in our implementation. I am digging through to see what the possible issue could be.

Update:
So it turns out that the solver is working fine, but we just need to increase the precision using smaller tolerance. I also confirmed that the laplacian step output matrix between sklearn and ours is the same.

aamijar · 2025-10-16T01:06:42Z

Update:
I'm running into some problems with gpu KMeans potentially not being consistent compared to cpu KMeans. I wonder if that is expected? For example if I replace the KMeans in sklearn spectral clustering with cuml.cluster KMeans then it gives a much lower ARI.
This was solved by oversampling_factor=0

Update:
Key findings
oversampling_factor=0 k-means++ init method
increasing solver precision, tol=1e-5->0
fix to remove duplicate entries in laplacian
max_iterations = 10 * n_samples instead of fixed 1000
are able to improve the quality.
we are able to achieve near perfect scores compared to ground truth in the gtests, mean of 0.97 ARI across 20 runs.

cjnolet · 2025-10-21T14:45:53Z

cpp/include/cuvs/cluster/spectral.hpp

@@ -0,0 +1,38 @@
+/*
+ * Copyright (c) 2025, NVIDIA CORPORATION.


I propose we name this file spectral.hpp instead of spectral_clustering. The clustering is already in the namespace, so it's redundant.

Addressed in f1ab56a

cjnolet · 2025-10-21T14:47:10Z

cpp/include/cuvs/preprocessing/spectral_embedding.hpp

 * @}
 */

+void create_connectivity_graph(raft::resources const& handle,


This is not part of the standard "fit()" "predict()" APIs that one would normally use day to day, so please add it to the cuvs::preprocessing::spectral_embedding::helpers namespace.

Addressed in f1ab56a

copy-pr-bot · 2025-10-21T21:49:27Z

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

…into spectral-clustering

coderabbitai · 2025-10-23T20:59:46Z

Important

Review skipped

Auto reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

✨ Finishing touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Post copyable unit tests in a comment

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

viclafargue

Great work, LGTM! It looks like the connectivity graph uses a int as its nnz type. I guess this is intentional and temporary?

viclafargue · 2025-10-24T09:26:06Z

cpp/src/cluster/spectral.cu

+  raft::linalg::transpose(handle,
+                          embedding_col_major.data_handle(),
+                          embedding_row_major.data_handle(),
+                          n_samples,
+                          config.n_components,
+                          raft::resource::get_cuda_stream(handle));


I guess that it is necessary to double VRAM usage and perform a transposition here, but would be great if we could find a workaround in the long term.

Thanks for the review! Yes, we have to switch from col to row major since spectral embedding needs col major and kmeans needs row major. Yes, we are using int as nnz type currently until we resolve the scale up work in spectral embedding.

jnke2016 · 2025-10-26T01:14:38Z

cpp/include/cuvs/cluster/spectral.hpp

+  int n_components;
+  int n_init;
+  int n_neighbors;
+  uint64_t seed;


Can we expose the rng_state instead of the seed?

Addressed in d9d4bc8

jnke2016 · 2025-10-26T01:30:28Z

cpp/include/cuvs/cluster/spectral.hpp

+
+void fit_predict(raft::resources const& handle,
+                 params config,
+                 raft::device_coo_matrix_view<float, int, int, int> connectivity_graph,


raft::device_coo_matrix_view<double, int, int, int> is not supported while cuGraph has templates for both float and double. Can we add support for the latter as well. Otherwise, i get the below error

error: no suitable user-defined conversion from "raft::device_coo_matrix_view<double, int, int, int>" to "raft::device_coo_matrix_view<float, int, int, int>" exists

I have added double support in 5c608f3

jnke2016 · 2025-10-26T02:40:55Z

I tested this PR in cuGraph and provided few suggestions. A cuGraph PR incorporating the cuVS Spectral Clustering API is ready for review and will be merged as soon as this PR is. @aamijar

spectral clustering

1b763de

github-project-automation bot added this to Vector Search, ML, & Data Mining Release Board Oct 14, 2025

github-project-automation bot moved this to Todo in Vector Search, ML, & Data Mining Release Board Oct 14, 2025

Merge branch 'branch-25.12' into spectral-clustering

bf6fe74

aamijar self-assigned this Oct 14, 2025

aamijar added the non-breaking Introduces a non-breaking change label Oct 14, 2025

aamijar moved this from Todo to In Progress in Vector Search, ML, & Data Mining Release Board Oct 14, 2025

aamijar added the feature request New feature or request label Oct 14, 2025

aamijar requested a review from jnke2016 October 14, 2025 15:54

add gtests

58a2d04

aamijar closed this Oct 16, 2025

github-project-automation bot moved this from In Progress to Done in Vector Search, ML, & Data Mining Release Board Oct 16, 2025

aamijar reopened this Oct 16, 2025

aamijar and others added 2 commits October 18, 2025 00:56

update

34e81dd

Merge branch 'branch-25.12' into spectral-clustering

38f12ee

aamijar marked this pull request as ready for review October 18, 2025 01:09

aamijar requested review from a team as code owners October 18, 2025 01:09

aamijar and others added 2 commits October 20, 2025 23:05

update hyperparams

c3069f0

Merge branch 'main' into spectral-clustering

67a21c5

cjnolet reviewed Oct 21, 2025

View reviewed changes

cjnolet mentioned this pull request Oct 21, 2025

eigen_solvers header #1421

Closed

aamijar moved this from Done to In Progress in Vector Search, ML, & Data Mining Release Board Oct 21, 2025

aamijar and others added 3 commits October 21, 2025 19:46

rename to spectral.hpp

f1ab56a

Merge branch 'main' into spectral-clustering

376da2e

Merge branch 'main' into spectral-clustering

5007df9

increase precision

472eb55

Merge branch 'spectral-clustering' of https://github.com/aamijar/cuvs …

2d90bb5

…into spectral-clustering

aamijar force-pushed the spectral-clustering branch from 2bbd1e5 to 2d90bb5 Compare October 21, 2025 21:50

aamijar and others added 7 commits October 21, 2025 22:27

update gtest

c42e74f

rerun CI

913d3aa

rerun CI

e4db202

rerun CI

bb3846b

Merge branch 'main' into spectral-clustering

d90614c

rerun CI

3ecd180

Merge branch 'spectral-clustering' of https://github.com/aamijar/cuvs …

f377524

…into spectral-clustering

aamijar force-pushed the spectral-clustering branch from f364560 to f377524 Compare October 22, 2025 23:01

aamijar and others added 2 commits October 23, 2025 09:07

Merge branch 'main' into spectral-clustering

7d3fe90

rerun CI

4708e69

viclafargue approved these changes Oct 24, 2025

View reviewed changes

Merge branch 'main' into spectral-clustering

6cc47d6

jnke2016 reviewed Oct 26, 2025

View reviewed changes

aamijar and others added 8 commits October 27, 2025 21:05

rng_state

d9d4bc8

Merge branch 'main' into spectral-clustering

eae5ef9

support double types

5c608f3

Merge branch 'main' into spectral-clustering

148bfd5

update gtests

0477a64

Merge branch 'main' into spectral-clustering

0544f46

move templates to src files

b9c5876

remove whitespace

c8ce74f

Spectral Clustering #1425

Are you sure you want to change the base?

Spectral Clustering #1425

Uh oh!

Conversation

aamijar commented Oct 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

copy-pr-bot bot commented Oct 14, 2025

Uh oh!

aamijar commented Oct 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aamijar commented Oct 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cjnolet Oct 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

copy-pr-bot bot commented Oct 21, 2025

Uh oh!

coderabbitai bot commented Oct 23, 2025

Review skipped

Uh oh!

viclafargue left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jnke2016 Oct 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jnke2016 commented Oct 26, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

aamijar commented Oct 14, 2025 •

edited

Loading

aamijar commented Oct 15, 2025 •

edited

Loading

aamijar commented Oct 16, 2025 •

edited

Loading

cjnolet Oct 21, 2025 •

edited

Loading

jnke2016 Oct 26, 2025 •

edited

Loading