Skip to content

Commit

Permalink
feat: Add edge test cases for OpenDAL Core (#3274)
Browse files Browse the repository at this point in the history
* feat: Add edge test cases for OpenDAL Core

Signed-off-by: Xuanwo <[email protected]>

* Fix typo

Signed-off-by: Xuanwo <[email protected]>

* Fix license

Signed-off-by: Xuanwo <[email protected]>

* Use checkout v4

Signed-off-by: Xuanwo <[email protected]>

---------

Signed-off-by: Xuanwo <[email protected]>
  • Loading branch information
Xuanwo authored Oct 13, 2023
1 parent 2ce0e93 commit 652f223
Show file tree
Hide file tree
Showing 7 changed files with 118 additions and 12 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -15,20 +15,29 @@
# specific language governing permissions and limitations
# under the License.

name: Fs write full disk
name: Edge Test

on:
push:
branches:
- main
pull_request:
branches:
- main
paths:
- "core/src/**"
- "!core/src/docs/**"
- "!core/src/services/**"
- "core/src/services/fs/**"
- ".github/workflows/edge_test.yml"

jobs:
test:
test_file_write_on_full_disk:
runs-on: ubuntu-latest

steps:
- name: Checkout code
uses: actions/checkout@v2
uses: actions/checkout@v4

- name: Create disk image
run: |
Expand All @@ -37,16 +46,15 @@ jobs:
- name: Mount disk image
run: |
mkdir ./td
sudo mount -o loop disk.img ./td
mkdir /tmp/test_dir
sudo mount -o loop disk.img /tmp/test_dir
- name: Set permissions
run: chmod a+wr ./td
run: sudo chmod a+wr /tmp/test_dir

# Add more steps for testing as needed
- name: Test
working-directory: core/edge/file_write_on_full_disk
run: cargo run
env:
OPENDAL_FS_ROOT: /tmp/test_dir

- name: Clean up
run: |
sudo umount ./td
rm -rf ./td
rm disk.img
10 changes: 10 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ exclude = ["examples"]
members = [
"core",
"core/fuzz",
"core/edge/*",

"bindings/c",
"bindings/nodejs",
Expand Down
3 changes: 3 additions & 0 deletions core/edge/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# OpenDAL Edge Tests

OpenDAL edge tests served as edge tests for the OpenDAL project. They will have pre-set data and will test the functionality of the OpenDAL project.
28 changes: 28 additions & 0 deletions core/edge/file_write_on_full_disk/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

[package]
name = "edge_test_file_write_on_full_disk"
edition = "2021"
version = "0.0.0"
publish = false

[dependencies]
futures = "0.3"
opendal = { workspace = true }
tokio = { version = "1", features = ["full"] }
rand = "0.8"
14 changes: 14 additions & 0 deletions core/edge/file_write_on_full_disk/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# File Write on Fill Disk

Reported by [The `FsBackend` only writes partial bytes and doesn't raise any errors](https://github.com/apache/incubator-opendal/issues/3052).

Setup:

```shell
fallocate -l 512K disk.img
mkfs disk.img
mkdir ./td
sudo mount -o loop td.img ./td
chmod a+wr ./td
```

42 changes: 42 additions & 0 deletions core/edge/file_write_on_full_disk/src/main.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
// Licensed to the Apache Software Foundation (ASF) under one
// or more contributor license agreements. See the NOTICE file
// distributed with this work for additional information
// regarding copyright ownership. The ASF licenses this file
// to you under the Apache License, Version 2.0 (the
// "License"); you may not use this file except in compliance
// with the License. You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.

use opendal::services::Fs;
use opendal::Operator;
use opendal::Result;
use rand::prelude::*;
use std::env;

#[tokio::main]
async fn main() -> Result<()> {
let mut builder = Fs::default();
builder.root(&env::var("OPENDAL_FS_ROOT").expect("root must be set for this test"));
let op = Operator::new(builder)?.finish();

let size = thread_rng().gen_range(512 * 1024 + 1..4 * 1024 * 1024);
let mut bs = vec![0; size];
thread_rng().fill_bytes(&mut bs);

let result = op.write("/test", bs).await;
// Write file with size > 512KB should fail
assert!(
result.is_err(),
"write file on full disk should return error"
);

Ok(())
}

0 comments on commit 652f223

Please sign in to comment.