My attempts to utilize a custom model function from Sklearn appear to be yielding incorrect distances. I possess two vectors and typically compute their cosine similarity, following it up with 1 - cosine_similarity for usage as a distance metric.
Here's the snippet of code in question:
from sklearn.metrics.pairwise import cosine_similarity
def dist_fun(x, y):
return 1 - cosine_similarity(x.reshape(-1, 1), y.reshape(-1, 1))
nbrs = NearestNeighbors(n_neighbors=499, algorithm='brute',metric = dist_fun)
nbrs.fit(x)
distances, indices = nbrs.kneighbors(x[10])
print (distances)
The output that I am observing is rather perplexing. The distance values show a consistent increase in their magnitude. Even when attempting to predict nearest neighbors with varying sample sizes, this monotonically increasing behavior persists.
array([[ -4.44089210e-16, 3.56164824e-03, 8.85347066e-02,
9.26700271e-02, 9.58609825e-02, 9.64477012e-02,
9.71035356e-02, 9.73237473e-02, 9.80660138e-02,
9.80660138e-02, 9.80660138e-02, 9.83731564e-02,
1.00054271e-01, 1.01234246e-01, 1.01811852e-01,
1.01849141e-01, 1.02621459e-01, 1.03175060e-01,
... (remaining content omitted for brevity) ...
... (remaining content omitted for brevity) ...
1.72180127e-01]])