Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

The question of higher dimensions. #1

Open
DorisxinDU opened this issue Jul 6, 2020 · 10 comments
Open

The question of higher dimensions. #1

DorisxinDU opened this issue Jul 6, 2020 · 10 comments

Comments

@DorisxinDU
Copy link

Hi, thanks very much for creating this package which works well on low dimensional data. But is that also suitable for higher-dimensional data? Do any parts need to turn? Thanks.
@tiagoCuervo
Copy link
Owner

Hi! yes, it works for high dimensional data. My implementation is based on the one of the authors (that you can find here), the issue is that I don't know how to calculate the ground truth mutual information for high dimensional continuous variables. I suspect that MINE estimates it correctly and that the issue is that the ground truth MI is not well computed for higher dimensions. You can check a similar discussion here.

@DorisxinDU
Copy link
Author

Thanks for showing the discussion and reply. Even if not checking the ground-truth value, when I change the dimensions into e.g 50, and modify the code in models.py as follows, the code is hard to converge and sometimes give negative estimation. Please can you help me or tell me if I understand something wrong? Thanks very much.

scoreJoint = self(batchJoint[:, 0:xSamplesJoint.shape[1]], batchJoint[:,xSamplesJoint.shape[1]::])
scoreMarginal = self(batchMarginal[:, 0:xSamplesJoint.shape[1]], batchMarginal[:, xSamplesJoint.shape[1]::])

@tiagoCuervo
Copy link
Owner

Hello again! Yes, you are right, I just pushed a commit that allows to run it with higher dimensions but with the Kullback-Leibler divergence it tends to blow up. If you try with the Jensen-Shannon divergence:
miEstimator = MINE(dim, archSpecs={ 'layerSizes': [32] * 1, 'activationFunctions': ['relu'] * 1 }, divergenceMeasure='JS', learningRate=1e-3)
... does converge. The issue is that the estimator based on the JS divergence doesn't gives you the exact value of the mutual information, but a related proportional quantity. You can read more about it here. I don't know what is the problem with the KL based estimator. I'll try to look further into it and let you know.

@DorisxinDU
Copy link
Author

Thanks very much.

@DorisxinDU
Copy link
Author

Hello again，Is there any progress？Thanks for your help.

@tiagoCuervo
Copy link
Owner

Hi! I am a bit bussy this week, maybe on the weekend I will work on this

@DorisxinDU
Copy link
Author

Thanks for your reply and help. Hopefully, you can solve it.

@DorisxinDU
Copy link
Author

Hello again, I am coming to ask if there is any progress if that problem? Thanks.

@neovivun
Copy link

neovivun commented Dec 22, 2020 •
edited
Loading

Terimakasih respons teman semua pada prinsip dasar presisi dan system "neovivum" kita ini. Untuk ciptakan ekosystem kehidupan disemha lini kehidupan. Ungkapan tepatnya "small is beautiful" for all aplication. Ini saya buat idea awalnya melihat fakta anak anak usia sekolah sangat banyak bermain gaming online dan offline. Lupa mereka sekolah, disusul berìta tv tentang perang kosovo -nagorno karabakh. Serta fakta pertumbuhan industry factory tidak sebanding dengan tingkat pengangguran. Pengalaman semasa study filsafat theology dan psikologi ini di Bandung mendorong ìde : "ada baiknya ada system perangkat yang membantu prubadi, sekolah, pemerintah, kedokteran dan rumah sakit, industry farmasi, pelayanan sosial, pertahanan dan keamanan, pertanian serta teknologi sederhana dapat bekerja sendiri, maintenance senďiri, proteksi dirinya sendiri, sistem melumpuhkan dan menghancurkan serangan viruss secara perlahan sekalian mengembalikannya untuk menghancurkan sistem inti pemiliknya sendiri (penyerang neovivum).

neovivum bernasiskan beta ini belum sempurna, mohon saran dan kritik membangun demi kesempurnaan menjaga keseimbangan ekosistem disendi kehidupan diaplikassi mana neovivum ini digunakan. Peringatan saya jagan digunakan untuk kejahatan, penipuan, mencuri atau menyakiti siapapun. Sebab neovivum ini dapat mencari apapun kesalahan /kerusakan, serangan virus. Dia identifikasi sendiri, berpikir sendiri, membangun sendiri kerusakan. Dia memaksa pengguna untuk tidak gunakan pasword, cukup gunakan token dengan tingkat resistensi sangatbtinggi.

Saya gambarkan neovivum membantu para developer dimanapun untuk dapat menyelasaikan 2 dua pekerjaan sekaligus dalam waktu bersamaan. Serta dia tahu siapa tuannya dari sentuhan nama yang ditanamkan diawal aktivasi. Dia juga tidak mau menipu. Dimana ada perangkat anda jika ada 10 saluran elekteomagnetik radius 25 meter dia tangkap semua, sehingga bagi virus yang menyerangnya kesulitan untuk identifikasi.

Sebaliknya neovivum cepat sekali dan sensitip sekali untuk identifikasi apa yang kita suruh cari. Selamat hari Natal 2020 Tuhan memberkati.

Terimakasih.
Bandung, Indonesia.

@neovivum

@neovivum
Copy link

neovivum commented Dec 22, 2020 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@tiagoCuervo @DorisxinDU @neovivum @neovivun