Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

A question about using your torch model, the results are always divided into one category #4

Open
Bornesf opened this issue Nov 19, 2022 · 0 comments

Comments

@Bornesf
Copy link

Bornesf commented Nov 19, 2022

Hello, @
can you provide your pre trained model?
I ran through your code here, but I encountered a problem. The DBN model always classifies all classes into one class. Specifically, only the class with the largest number of samples can be predicted in all classes. The accuracy and recall of other classes are all zero.
I tried the following:

  1. First of all, I ran the MLP model under the same processed data set and reached the performance indicators mentioned in your paper. It can be determined that it is not a data set preprocessing problem
  2. I reduced the number of Benign class samples in the dataset to 1w, and the output will become that all classes are divided into DOS classes with the second largest number of samples, so it should not be an imbalance problem
  3. The third attempt I made was to use your code to start the pre training, which took about 80 minutes in total, but the final weight file size obtained from the training was only 3kb, so I wondered whether it was a problem of pre training. Then I removed the pre training and randomly initialized the weights, but the effect was not improved. The parameters were strictly in accordance with the requirements in your paper
    The following result reports:
Training Set -- Classification Report 
		precision    recall  f1-score   support

      Benign       0.66      1.00      0.80    800000
 Botnet ARES       0.00      0.00      0.00     50149
 Brute Force       0.00      0.00      0.00     50115
    DoS/DDoS       0.00      0.00      0.00    212102
    PortScan       0.00      0.00      0.00     44460
  Web Attack       0.00      0.00      0.00     50284

    accuracy                           0.66   1207110
   macro avg       0.11      0.17      0.13   1207110
weighted avg       0.44      0.66      0.53   1207110
Validation Set -- Classification Report
		precision    recall  f1-score   support

      Benign       0.84      1.00      0.91    407087
 Botnet ARES       0.00      0.00      0.00       379
 Brute Force       0.00      0.00      0.00      1749
    DoS/DDoS       0.00      0.00      0.00     63966
    PortScan       0.00      0.00      0.00     11576
  Web Attack       0.00      0.00      0.00       383

    accuracy                           0.84    485140
   macro avg       0.14      0.17      0.15    485140
weighted avg       0.70      0.84      0.77    485140
Testing Set -- Classification Report deep_belief_network
 		precision    recall  f1-score   support

      Benign       0.84      1.00      0.91    407115
 Botnet ARES       0.00      0.00      0.00       398
 Brute Force       0.00      0.00      0.00      1672
    DoS/DDoS       0.00      0.00      0.00     64142
    PortScan       0.00      0.00      0.00     11346
  Web Attack       0.00      0.00      0.00       464

    accuracy                           0.84    485137
   macro avg       0.14      0.17      0.15    485137
weighted avg       0.70      0.84      0.77    485137

Look forward to your reply. Please let me know if there is anything unclear

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant