Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GDS installation #8

Open
QYQ0909 opened this issue Jun 25, 2022 · 1 comment
Open

GDS installation #8

QYQ0909 opened this issue Jun 25, 2022 · 1 comment

Comments

@QYQ0909
Copy link

QYQ0909 commented Jun 25, 2022

Hi,I meet some questions when I verify that GDS installation was successfulI. I've already installed MLNX_OFED driver and had one NVMe SSD. But when I run ./gdscheck.py -p, it's still showing that "NVMe : Unsupported" and "--Mellanox PeerDirect : Disabled". I want to know why and how to solve them. Thanks. The results are as follows.
(cuda:11.7 , nvidia driver:515.43.04 , MLNX_OFED:5.6,uname -r (kernel): 5.4.0 , SSD: optane NVMe SSD , GPU:P100 )

============
ENVIRONMENT:

DRIVER CONFIGURATION:
NVMe : Unsupported
NVMeOF : Unsupported
SCSI : Unsupported
ScaleFlux CSD : Unsupported
NVMesh : Unsupported
DDN EXAScaler : Unsupported
IBM Spectrum Scale : Unsupported
NFS : Unsupported
BeeGFS : Unsupported
WekaFS : Unsupported
Userspace RDMA : Unsupported
--Mellanox PeerDirect : Disabled
--rdma library : Not Loaded (libcufile_rdma.so)
--rdma devices : Not configured
--rdma_device_status : Up: 0 Down: 0
CUFILE CONFIGURATION:
properties.use_compat_mode : true
properties.force_compat_mode : false
properties.gds_rdma_write_support : true
properties.use_poll_mode : false
properties.poll_mode_max_size_kb : 4
properties.max_batch_io_size : 128
properties.max_batch_io_timeout_msecs : 5
properties.max_direct_io_size_kb : 16384
properties.max_device_cache_size_kb : 131072
properties.max_device_pinned_mem_size_kb : 33554432
properties.posix_pool_slab_size_kb : 4 1024 16384
properties.posix_pool_slab_count : 128 64 32
properties.rdma_peer_affinity_policy : RoundRobin
properties.rdma_dynamic_routing : 0
fs.generic.posix_unaligned_writes : false
fs.lustre.posix_gds_min_kb: 0
fs.beegfs.posix_gds_min_kb: 0
fs.weka.rdma_write_support: false
profile.nvtx : false
profile.cufile_stats : 0
miscellaneous.api_check_aggressive : false
GPU INFO:
GPU index 0 Tesla P100-PCIE-16GB bar:1 bar size (MiB):16384 supports GDS
PLATFORM INFO:
IOMMU: disabled
Platform verification succeeded

@Pedrexus
Copy link

Pedrexus commented Apr 4, 2023

Any update on this issue?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants