How to match real label with predicted y after using model.predict() with disjointloader?

[moonseter]

When using disjointloader, there is a default shuffle=true. I want to use model.predict() to get ACC and other evaluate matrics, but the shuffle makes it hard to match predicted_y and true_label. Could you please help me resolve this issue? Thanks very much.

[danielegrattarola]

Hey,
Is there a reason why setting shuffle=False would not work?

[moonseter]

Hey, Is there a reason why setting shuffle=False would not work?

Actually i have no idea about this fault, it seems that model.predict() just can't work efficiently.

I use the test_data to carry out model.evaluate() and also model.predict(), which give consistent results that meet expectations. But when using independent data which is preprocessed in the same way as training data, the result seems to be random. I get different and lower result every time run prediction. Like code below, predicted 'y' is different every time run y_pred = model.predict(loader_pr.load(), steps=loader_pr.steps_per_epoch).

If you have relevant solutions or can provide some sample code, I would be very grateful.

My dataset is defined: loader_pr = DisjointLoader(data, batch_size=BATCH_SIZE, shuffle=False) ;
Model:
class GATModel(Model):
def init(self, num_heads, hidden_units, **kwargs):
super().init()
self.conv1 = GATConv(channels = hidden_units, attn_heads = num_heads)
self.norm1 = LayerNormalization()
self.conv2 = GATConv(channels = hidden_units, attn_heads = num_heads)
self.norm2 = LayerNormalization()
self.pooling = GlobalAvgPool()
self.dropout = Dropout(0.5)
self.dense1 = Dense(64, activation = 'relu')
self.dense2 = Dense(1, activation = 'sigmoid')
def call(self, inputs):
x, a, i = inputs
x = self.conv1([x, a])
x = self.norm1(x)
x = self.conv2([x, a])
x = self.norm2(x)
x = self.pooling([x, i])
x = self.dropout(x)
x = self.dense1(x)
output = self.dense2(x)
return output

save weights: model.save_weights('./weights/') and loaded: model.load_weights('./weights/')
Predict:
y_pred = model.predict(
loader_pr.load(),
steps=loader_pr.steps_per_epoch
)
y = (y_pred > 0.5).astype(int)
y = np.ravel(y)
labels = []
for i in range(0, len(data)):
labels.append(data[i].y)
labels = np.vstack(labels).squeeze()
acc = accuracy_score(labels, y)