Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Question]: How to utilize charybdefs in the specified scenario here? #16

Open
db-vmw opened this issue Dec 21, 2018 · 4 comments
Open

Comments

@db-vmw
Copy link

db-vmw commented Dec 21, 2018

Dear Team,
My intention here is to validate application behavior for the specified disk latency. The Ubuntu system on which the application will be running has /var mounted as "/dev/mapper/vg-var". The application writes on to /var (underlying storage array). The application launches multiple processes which runs with different users.
I'm planning to use Charybdefs to test the disk latency and plan is to mount charybdefs on to mount point /var/lib/scylla with data directory being /var/lib. With this, I assume , disk latency fault (disk latency) injected using Charybdefs whill have any impact for all the write operation happening on to /var/lib directory. Hence the application will even have an impact.
Question-1:
Is my understanding is correct? If not, can you please suggest a way to test scenario specified above using Charybdefs?

NOTE:
Question-2
I have used the following command to mount Charybdefs on my Ubuntu machine. But, the mounted fs is visible using the command "df" only for the "root" user. While the "df" command using the user other than root (example: "ubuntu") isn't visible.
sudo ./charybdefs /var/lib/kscylla/ -omodules=subdir,subdir=/home/ubuntu/skp/kscylla

Waiting for a quick response.

Thanks,
Dinesh

@bentsi
Copy link
Contributor

bentsi commented Dec 24, 2018

@db-vmw
Question-1: latency fault are not supported
Question-2: try to run the command without sudo

@db-vmw
Copy link
Author

db-vmw commented Jan 2, 2019

@bentsi ,
Thanks for the response.
Question-1: I came across the following lines in Charybdefs blog. Are these applicable only for the write operation (https://www.scylladb.com/2016/05/02/fault-injection-filesystem-cookbook/)?

Now, let’s delay each filesystem operation by 50 ms.

client.set_all_fault(False, 0, 0, "", False, 50000, False)

I thought the above fault will introduce a delay of 50ms for every write operation.

@bentsi
Copy link
Contributor

bentsi commented Jan 2, 2019

@db-vmw
I read the code of the server.cc and indeed there is support for delays. This is applied to all syscalls, not only for write.

@db-vmw
Copy link
Author

db-vmw commented Jan 2, 2019

Thanks @bentsi for the confirmation on support for delay.
But, can you please help me on how to achieve the disk latency fault injection using charybdefs for my use case?
Highlighting my use case again (explained earlier in this ticket as well).
Our application writes the data to /var location which is mounted as "/dev/mapper/vg-var" and it's mapped to a storage array. I want to validate latency for /var/lib location without restarting my application and without modifying my application properties. Do you have any suggestion around here on how to achieve disk latency using charybdefs?
I'm thinking to achieve by following steps and please let me know if this works:

./charybdefs /var/scyla-mnt-pnt -omodules=subdir,subdir=/var/kscylla
With this, what I expect is, charybdefs mounted having data directory being /var/kscylla.

mount --bind /var/scyla-mnt-pnt /var/lib
I assume the above command will bind the existing /var/lib to charybdefs. Please confirm my understanding.

Thanks,
db

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants