In OCGNN model, why updates radius only in warm-up training epochs

[ahmed3amerai]

I understand that OCGNN model aims to jointly learn the net parameters "w" and minimize the volume of the data description hypersphere which is characterized by a radius "r" and a center "c" as explained in "One-Class Graph Neural Networks for Anomaly Detection in Attributed Networks" paper. That means OCGNN learn "w", "r", and "c" not only "w"

I wonder why PyGod scripts updates radius only in warm-up training epochs, In that setting the radius would be learned from only first few epochs (default is 2).
I checked the OCGNN author implementation in his GitHub repository and found they updates the hypersphere radius in every epochs.

I also found that they calculate the hypersphere center before the first epoch, as the average of all data points.In contrast PyGod start the hypersphere center from (zeros) and calculate it after getting the (distance, loss, score) of the first epoch.
Couldn't that affect the starting loss hence affect the model, especially if the embedding center is far from the zeros (the initial center) ?

Thanks in advance for your clarification.

[kayzliu]

In the official implementation, r and c are not learnable and are determined by this line and this line. According to the author, it has better stability. In our implementation, we alternatively use small warmup epochs to update (not learn by gradient descent) r and c. For the initialization of c, we use the mean of emb in the warmup epochs (this line). Feel free to modify the model (e.g., increase the number of the warmup epochs) if it helps.

[ahmed3amerai]

Based on my experiments of applying OCGNN on security applications, updating r and c in small warmup epochs significantly affects the model stability.
As you mentioned this line in the official implementation updates r on every epochs, which help in minimizing the hypersphere radius while training. One of the training objectives is to descrive the smallest hypersphere that encloses the majority of the training data as mentioned in OCGNN paper.
Also I found that updating c every epoch is essential to achieve stability, since in some cases the location of emb in the embedding space is far from the c initialized in the warmup epochs.

I modified the model accordingly, and could issue a pull request for this modification if you are interested.
I am doing my PhD research in anomaly detection for security applications, we could discuss further collaboration if you wish.

[kayzliu]

Thanks for the insightful findings. The modification can be done by OCGNN(warmup=math.inf). Are there any other changes you made? Also, yes, I am open to collaboration. Feel free to DM me at [email protected].