Skip to content

cs-course/obs-tutorial

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Introduction

Object Storage Tutorial.

Basic Concept

SNIA Tutorials on Object Storage:

The Storage Networking Industry Association (SNIA) is a not–for–profit global organization, made up of member companies spanning the global storage market.

Lab1: Preparation

Environment

Git and Repository

Git tutorial https://github.com/cs-course/git-tutorial.

Alternatives: bitbucket, gitlab, gitee.

How to establish Python Environment

  • Python Distributions:
  • Fast deployment by docker:
    • Option 3: Python Docker https://github.com/cs-course/python-lab, E.g.:
      • docker pull zhan2016/python-lab:3.6.0
      • docker login daocloud.io && docker pull daocloud.io/zhan2016/python-lab:master-31a932d

How to establish Java Environment

Ongoing course: Java Programming, 2019-2020 2nd semester, just follow your teacher's guide.

Installation helper scripts https://github.com/ShiZhan/java-bundle (For adventurers).

How to establish Rust Environment

How to use Linux in Windows or MacOS (Optional)

Goal: try mock-s3 and s3proxy with less trouble

Method: Virtual Machine: Virtualbox, VMWare ...

Go directly to GUI, or using vagrant,refer to https://github.com/cs-course/vagrant-tutorial.

How to run servers within docker container (Optional)

Goal: try Openstack Swift or Ceph with less trouble

Better run within docker, refer to Docker tutorial https://github.com/cs-course/docker-tutorial.

Object Storage Server

Besides Option 1, Option 2, 3 offer compile-free executable.

Object Storage Client

Binary available for Option 1, Option 2 & 3 require Python, Option 4 require Go, Option 6 require Rust.

Basic Functionality

In computer programming, create, read, update, and delete (as an acronym CRUD) are the four basic functions of persistent storage.

Operation SQL HTTP
Create INSERT PUT / POST
Read (Retrieve) SELECT GET
Update (Modify) UPDATE PUT / POST / PATCH
Delete (Destroy) DELETE DELETE

Try object storage in some applications: zfile.

Lab2: Performance Evaluation

Object Storage Benchmark

Observation

Metrics: Throughput, Latency under different object size, concurrency, server total.

Suggested topics:

  • How object size affects performance?
    • for a particular application, is there a best way to fit into OBS?
  • The main factors behind I/O latency?
    • Get latency distribution first.
    • For evaluating percentile latency, s3bench is recommended.
  • What will happen when clients are crowded?
    • How concurrency affects latency distribution and throughput? How to enforce SLA by controlling?
  • The outcome of scaling out (putting more servers into system)?

More insights are encouraged.

Further thoughts

Lab3: Tail Latency Challenge

How to handle?

  • Hedged request
  • Tied request

How to predict?

  • queueing Theory
  • Sequence prediction

Experiences and Problems

Future Readings

Zhan.Shi @ 2017, 2018, 2019, 2020, 2021

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Jupyter Notebook 88.0%
  • Python 5.5%
  • Shell 4.2%
  • Batchfile 2.3%