To start, one approach is to first identify the common elements between variables a and b by utilizing sets. This method works best when the possible values in a and b are limited. Next, iterate through the unique values (contained in variable values) and use np.argwhere to pinpoint the indices in a and b where these values appear. The 2D indices for the sparse matrix can then be constructed using np.repeat and np.tile:
import numpy as np
from scipy import sparse
a = np.random.randint(0, 10, size=(400,))
b = np.random.randint(0, 10, size=(300,))
## Creating matrix post initial method
I1 = sparse.csr_matrix((a[:,None]==b), shape=(len(a), len(b)))
## Identifying common values in a and b:
values = set(np.unique(a)) & set(np.unique(b))
## Collecting indices in a and b where values match:
rows, cols = [], []
## Looping over shared values, finding their indices in a and b,
## and generating 2D indices of the sparse matrix with np.repeat and np.tile
for value in values:
x = np.argwhere(a==value).ravel()
y = np.argwhere(b==value).ravel()
rows.append(np.repeat(x, len(x)))
cols.append(np.tile(y, len(y)))
## Concatenating indices for different values and creating a 1D vector
## of True values for final matrix generation
rows = np.hstack(rows)
cols = np.hstack(cols)
data = np.ones(len(rows), dtype=bool)
## Generating sparse matrix
I3 = sparse.csr_matrix( (data, (rows, cols)), shape=(len(a), len(b)) )
## Verifying correct generation of the matrix:
print((I1 != I3).nnz==0)
The syntax for creating the csr matrix is sourced from the documentation. The test for equality in sparse matrices is inspired by this post.
Previous Response:
While performance may vary, you can avoid constructing a full dense matrix by employing a simple generator expression. Below is some code that initially generates the sparse matrix in the manner demonstrated by the OP, followed by utilizing a generator expression to assess all elements for equality:
import numpy as np
from scipy import sparse
a = np.random.randint(0, 10, size=(400,))
b = np.random.randint(0, 10, size=(300,))
## Matrix generation following OP's method
I1 = sparse.csr_matrix((a[:, None] == b), shape=(len(a), len(b)))
## Matrix creation using generator
data, rows, cols = zip(
*((True, i, j) for i, A in enumerate(a) for j, B in enumerate(b) if A == B)
)
I2 = sparse.csr_matrix((data, (rows, cols)), shape=(len(a), len(b)))
## Checking for matrix equality
print((I1 != I2).nnz == 0) ## --> True
It seems like the double loop is unavoidable, and it would be ideal to optimize this within numpy. However, using a generator expression does somewhat streamline the loops...