better document

CHANGELOG.md

@@ -10,6 +10,10 @@ Note: In this file, do not use the hard wrap in the middle of a sentence for com
 
 ## [Unreleased]
 
+### [0.2.19] - 2024-06-04
+
+- better documente for `fav_core`
+
 ## [0.2.18] - 2024-06-04
 
 - optimize: lto set to `fat` to reduce binary size.

Cargo.toml

@@ -14,7 +14,7 @@ repository = "https://github.com/kingwingfly/fav"
 documentation = ""
 
 [workspace.dependencies]
-fav_core = { path = "fav_core", version = ">=0.0.1" }
+fav_core = { path = "fav_core", version = ">=0.1.0" }
 fav_derive = { path = "fav_derive", version = ">=0.0.1" }
 fav_utils = { path = "fav_utils", version = ">=0.0.1" }
 fav_cli = { path = "fav_cli" }

fav_cli/Cargo.toml

@@ -16,7 +16,7 @@ rustdoc-args = ["--cfg", "docsrs"]
 
 [dependencies]
 fav_utils = { workspace = true }
-fav_core.workspace = true
+fav_core = { workspace = true }
 # CLI
 clap = { version = "4.5", features = ["derive"] }
 clap_complete = { version = "4" }

fav_core/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "fav_core"
-version = "0.0.9"
+version = "0.1.0"
 authors.workspace = true
 description = "Fav's core crate; A collection of traits."
 license.workspace = true
@@ -44,7 +44,6 @@ tracing = { version = "0.1" }
 tokio = { version = "1", features = ["full"] }
 
 [build-dependencies]
-rustc_version = "0.4.0"
 protobuf-codegen = { version = "3" }
 
 [features]

fav_core/README.md

@@ -1,13 +1,59 @@
-[`fav_core`](https://crates.io/crates/fav_core) is the core library of [`fav`](https://github.com/kingwingfly/fav)(A cli tool to download remote resources and keep a local state in protobuf).
+[fav_core](https://crates.io/crates/fav_core) is the core library of [fav_cli](https://github.com/kingwingfly/fav) (A cli tool to download remote resources and keep a local state in protobuf). In simple words, `fav_core` is a helper to build a stateful crawler. 
 
-[`fav_utils`](https://crates.io/crates/fav_utils) provides the utils for [`fav_cli`](https://crates.io/crates/fav_cli), which now only support [bilibili](https://www.bilibili.com)(Like Chinese Youtube). You can see it as a example for using this crate.
+[fav_utils](https://crates.io/crates/fav_utils) provides the utils for [fav_cli](https://crates.io/crates/fav_cli), which now only support [BiliBili](https://www.bilibili.com)(Like Chinese YouTube). You can see it as an example for using this crate.
 
-All you need is to define data structures with [`protobuf`](https://protobuf.dev) like [this example](https://github.com/kingwingfly/fav/blob/dev/fav_utils/proto/bili.proto).
+To save status, instead of using json, this crate uses `protobuf` since it is faster. You need to define data structures with [protobuf](https://protobuf.dev) like [this example](https://github.com/kingwingfly/fav/blob/dev/fav_utils/proto/bili.proto) (To derive trait for code generated by protobuf, see [example](https://github.com/kingwingfly/fav/blob/dev/fav_utils/build.rs)).
 
-And impl traits in `fav_core` like `Res`, `Set`, `Sets`, `Status`, `Attr` to defined resources, and `AuthOps`, `SetOps`, `SetsOps` to operate resources.
+`Sets` contains `Set`s, `Set` contains `Res`s(resource). The workflow is:
 
-After that, many `Ext` method will be available for you to use, helping you **batchly** operating resource, getting subset of resource, and so on.
+1. fetch `Sets` to refresh `Set`s
+2. fetch `Set` to refresh `Res`s
+3. fetch and pull `Res` to download
 
-Examples can be find [here](https://github.com/kingwingfly/fav).
+To implement this workflow and maintain a local state, `fav_core` has many useful traits:
 
-To derive trait for code generated by protobuf, see [example](https://github.com/kingwingfly/fav/blob/dev/fav_utils/build.rs).
+1. network helper
+
+- `Api`: help defining the APIs
+- `ApiProvider`: make app able to provide API based on `ApiKind` enum
+- `Net`: make app able to use the Internet
+
+2. Config
+
+- `Config: HttpConfig + ProtoLocal`: mark the app able to be config and persisted
+
+- `HttpConfig`: define the default headers, cookies
+
+3. Status and attributes
+
+- `Sets`: iterate and get subset of sets
+- `Set`:  iterate and get subset of resources
+- `Res`: `Res: Meta`
+- `Meta`: the metadata of resource, `Meta: Attr + Status`
+- `Attr`: provide resource's id and title
+- `Status`: the status of resource, like saved, fetched, tracked and expired
+
+4. Operations
+
+- `Ops`: `Ops: AuthOps + SetsOps + SetOps + ResOps`, means the **app** can perform all needing operations
+- `AuthOps`: used to login and logout
+- `SetsOps`: used to `fetch_sets` info, for example, add `English` `Chinese` `Japanese` as new movie collections to `Sets` defined in protobuf.
+- `SetOps`: used to `fetch_set` info, for example, add 《Oliver Twist》《Roman Holiday》《Twelve Angry Men》to `English` collection. 
+- `ResOps`: used to `fetch` and `pull` ,  for example,  `fetch`  id of 《Oliver Twist》 in target website, `pull` the resources to local disk based on the fetched id. 
+
+5. Persistence 
+
+- `PathInfo`: defined where to store status and config
+- `ProtoLocal`: `ProtoLocal: PathInfo + MessageFull` used to read and write status and config
+- `SaveLocal`: make app able to download `Res`, and modify local status. 
+
+6. visualize (optional): show status as table
+7. Ext methods:
+
+- `SetOpsExt: SetOps`: batch fetch set in sets
+- `ResOpsExt: ResOps`: batch fetch resources in set
+- `XXStatusExt`: batch modify children's StatusFlags
+
+To draw a conclusion, this crate contains all traits you need to build a stateful crawler. You can define data structures with `protobuf` for fast read and write. Make them stateful, configurable, and able to be persisted. Many network helper is provided, you can `request_json` and `resquest_protobuf` directly. And `Ext` traits provided so that you can batch fetch and pull data or modify the resources' StatusFlags. 
+
+An example can be found in [fav](https://github.com/kingwingfly/fav) repo. 

fav_core/src/ops.rs

@@ -86,10 +86,10 @@ pub trait LocalSetOps: Net + HttpConfig {
 }
 
 /// Making a client able to operate on resource
-#[trait_variant::make(ResOps: Send)]
 /// - [`LocalResOps`]'s async methods cannot be Send.
 /// - [`ResOps`] is generated by [`trait_variant::make`], which implements `Send`.
 /// For more information, see [Rust Blog](https://blog.rust-lang.org/2023/12/21/async-fn-rpit-in-traits.html#async-fn-in-public-traits).
+#[trait_variant::make(ResOps: Send)]
 pub trait LocalResOps: Net + HttpConfig {
     /// The resource type the operations on
     type Res: Res;