Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: priority queues #43

Merged
merged 2 commits into from
Nov 22, 2023
Merged

feat: priority queues #43

merged 2 commits into from
Nov 22, 2023

Conversation

amsmith-pro
Copy link
Contributor

Copied from abandoned PR #42

See the updated SDD for information.

Overview:

Redis queues used to store "tasks" to do
    Different queues for emergency, high, medium, low priority tasks
    Emergency tasks handled first and so on
    Current tasks are "create_itinerary" and "cancel_itinerary"
    Single threaded execution, currently no concurrent tasks. Option to expand here
create_itinerary and cancel_itinerary now immediately return with a task_id, effectively a bakery ticket number
    task status can be checked with get_task_status
    queued tasks can be preemptively deleted with cancel_task
    this means that clients don't block until the itinerary creation/deleted is completed (as in R3), asynchronous
Replace confirm_itinerary with create_itinerary
    No longer saving "draft" itineraries in local memory after a query
    The client will submit the full itinerary to the scheduler (obtained from query_flight) instead of a draft ID number
    "Reserving" multiple itineraries on a query was problematic, reduced options for other queries
    Submitted itineraries undergo similar checks to query_flight but limited to the vertipads and aircraft listed in the itinerary, much faster
Remove calls to svc-compliance
    svc-compliance will periodically check storage for unapproved plans now

@amsmith-pro amsmith-pro requested a review from a team as a code owner November 22, 2023 06:58
@cla-bot cla-bot bot added the cla-signed label Nov 22, 2023
@github-actions GitHub Actions
Copy link

github-actions bot commented Nov 22, 2023 •
edited
Loading

This PR will generate the following release notes when merged:

Release

✨ Features

🛠 Maintenance

@github-actions GitHub Actions
Copy link

github-actions bot commented Nov 22, 2023 •
edited
Loading

📋 Release Checklist ✅

🏡 Check locally with make release-checklist

Release Checklist

📣 Checking log statements...

Test /// Location Type Statement
X X client-grpc/src/client.rs (105) wrong (fn prepend) grpc_warn!("(is_ready MOCK) {} client.", self.get_
X X client-grpc/src/client.rs (106) wrong (fn prepend) grpc_debug!("(is_ready MOCK) request: {:?}", reque
X X client-grpc/src/client.rs (114) wrong (fn prepend) grpc_warn!("(query_flight MOCK) {} client.", self.
X X client-grpc/src/client.rs (115) wrong (fn prepend) grpc_debug!("(query_flight MOCK) request: {:?}", r
X X client-grpc/src/client.rs (128) wrong (fn prepend) grpc_warn!("(create_itinerary MOCK) {} client.", s
X X client-grpc/src/client.rs (129) wrong (fn prepend) grpc_debug!("(create_itinerary MOCK) request: {:?}
X X client-grpc/src/client.rs (145) wrong (fn prepend) grpc_info!("(cancel_itinerary MOCK) {} client.", s
X X client-grpc/src/client.rs (146) wrong (fn prepend) grpc_debug!("(cancel_itinerary MOCK) request: {:?}
X X client-grpc/src/client.rs (162) wrong (fn prepend) grpc_info!("(cancel_task MOCK) {} client.", self.g
X X client-grpc/src/client.rs (163) wrong (fn prepend) grpc_debug!("(cancel_task MOCK) request: {:?}", re
X X client-grpc/src/client.rs (178) wrong (fn prepend) grpc_info!("(get_task_status MOCK) {} client.", se
X X client-grpc/src/client.rs (179) wrong (fn prepend) grpc_debug!("(get_task_status MOCK) request: {:?}"
X X client-grpc/examples/grpc.rs (35) wrong (fn prepend) println!("(main) RESPONSE={:?}", &itineraries);
X X client-grpc/examples/grpc.rs (39) wrong (fn prepend) println!("(main) ERROR={:?}", e);
X X client-grpc/examples/grpc.rs (58) wrong (fn prepend) println!("(main) RESPONSE={:?}", &task_id);
X X client-grpc/examples/grpc.rs (62) wrong (fn prepend) println!("(main) ERROR={:?}", e);
X X client-grpc/examples/grpc.rs (78) wrong (fn prepend) println!("(main) RESPONSE={:?}", &metadata);
X X client-grpc/examples/grpc.rs (82) wrong (fn prepend) println!("(main) ERROR={:?}", e);
X X client-grpc/examples/grpc.rs (101) wrong (fn prepend) println!("(main) RESPONSE={:?}", &task_id);
X X client-grpc/examples/grpc.rs (105) wrong (fn prepend) println!("(main) ERROR={:?}", e);
X X client-grpc/examples/grpc.rs (121) wrong (fn prepend) println!("(main) RESPONSE={:?}", &task_id);
X X client-grpc/examples/grpc.rs (125) wrong (fn prepend) println!("(main) ERROR={:?}", e);
X X client-grpc/examples/grpc.rs (136) (no_fn_prepend) msg println!("Client created");
X X client-grpc/examples/grpc.rs (136) punctuation println!("Client created");
X X client-grpc/examples/grpc.rs (140) (no_fn_prepend) msg println!( "NOTE: Ensure the server is ru...
X X server/build.rs (48) (no_fn_prepend) msg println!("cargo:rerun-if-changed={}", proto_file);
X X server/src/tasks/mod.rs (74) wrong (fn prepend) tasks_warn!("(ToRedisArgs) error serializing task"
X X server/src/tasks/mod.rs (74) punctuation tasks_warn!("(ToRedisArgs) error serializing task"
X X server/src/tasks/pool.rs (96) wrong (fn prepend) tasks_error!("(TaskPool new) no connection address
X X server/src/tasks/pool.rs (103) wrong (fn prepend) tasks_info!( "(TaskPool new) creating p
X X server/src/tasks/pool.rs (107) wrong (fn prepend) tasks_info!("(TaskPool new) pool created.");
X X server/src/tasks/pool.rs (111) wrong (fn prepend) tasks_error!("(TaskPool new) could not create pool
X X server/src/tasks/pool.rs (207) wrong (fn prepend) tasks_error!("(RedisPool new_task) expiry must be
X X server/src/tasks/pool.rs (212) wrong (fn prepend) tasks_error!("(RedisPool new_task) new task status
X X server/src/tasks/pool.rs (217) wrong (fn prepend) tasks_error!("(RedisPool new_task) new task status
X X server/src/tasks/pool.rs (229) wrong (fn prepend) tasks_error!("(RedisPool new_task) Could not conve
X X server/src/tasks/pool.rs (239) wrong (fn prepend) tasks_error!( "(RedisPool updat
X X server/src/tasks/pool.rs (286) wrong (fn prepend) tasks_error!( "(RedisPool new_t
X X server/src/tasks/pool.rs (290) wrong (fn prepend) tasks_error!("(RedisPool new_task) unexpected redi
X X server/src/tasks/pool.rs (303) wrong (fn prepend) tasks_error!( "(RedisPool new_t
X X server/src/tasks/pool.rs (307) wrong (fn prepend) tasks_error!("(RedisPool new_task) could not set t
X X server/src/tasks/pool.rs (320) wrong (fn prepend) tasks_error!( "(RedisPool new_t
X X server/src/tasks/pool.rs (323) wrong (fn prepend) tasks_error!("(RedisPool new_task) could not set t
X X server/src/tasks/pool.rs (337) wrong (fn prepend) tasks_error!( "(RedisPool new_t
X X server/src/tasks/pool.rs (341) wrong (fn prepend) tasks_error!("(RedisPool new_task) could not set t
X X server/src/tasks/pool.rs (346) wrong (fn prepend) tasks_info!("(RedisPool new_task) created new task
X X server/src/tasks/pool.rs (347) wrong (fn prepend) tasks_debug!("(RedisPool new_task) new task #{task
X X server/src/tasks/pool.rs (369) wrong (fn prepend) tasks_error!( "(RedisPool updat
X X server/src/tasks/pool.rs (375) wrong (fn prepend) tasks_error!("(RedisPool update_task) Could not co
X X server/src/tasks/pool.rs (427) wrong (fn prepend) tasks_error!( "(RedisPool updat
X X server/src/tasks/pool.rs (431) wrong (fn prepend) tasks_error!("(RedisPool update_task) could not se
X X server/src/tasks/pool.rs (442) wrong (fn prepend) tasks_error!( "(RedisPool new_t
X X server/src/tasks/pool.rs (445) wrong (fn prepend) tasks_error!("(RedisPool new_task) could not set t
X X server/src/tasks/pool.rs (464) wrong (fn prepend) tasks_error!( "(RedisPool updat
X X server/src/tasks/pool.rs (475) wrong (fn prepend) tasks_error!( "(RedisPool get_t
X X server/src/tasks/pool.rs (503) wrong (fn prepend) tasks_error!( "(RedisPool updat
X X server/src/tasks/pool.rs (513) wrong (fn prepend) tasks_debug!("(RedisPool next_task) no tasks in qu
X X server/src/tasks/pool.rs (518) wrong (fn prepend) tasks_debug!("(RedisPool next_task) unexpected red
X X server/src/tasks/pool.rs (528) wrong (fn prepend) tasks_debug!( "(RedisPool next_
X X server/src/tasks/pool.rs (532) wrong (fn prepend) tasks_debug!("(RedisPool next_task) could not pop
X X server/src/tasks/pool.rs (541) wrong (fn prepend) tasks_error!("(RedisPool next_task) could not get
X X server/src/tasks/pool.rs (553) wrong (fn prepend) tasks_info!( "(RedisPool next_task) pop
X X server/src/tasks/pool.rs (554) wrong (fn prepend) tasks_debug!("(RedisPool next_task) task #{task_id
X X server/src/grpc/server.rs (221) wrong (fn prepend) grpc_warn!("(query_flight MOCK) scheduler server."
X X server/src/grpc/server.rs (222) wrong (fn prepend) grpc_debug!("(query_flight MOCK) request: {:?}", r
X X server/src/grpc/server.rs (238) wrong (fn prepend) grpc_warn!("(create_itinerary MOCK) scheduler serv
X X server/src/grpc/server.rs (239) wrong (fn prepend) grpc_debug!("(create_itinerary MOCK) request: {:?}
X X server/src/grpc/server.rs (256) wrong (fn prepend) grpc_warn!("(cancel_itinerary MOCK) scheduler serv
X X server/src/grpc/server.rs (257) wrong (fn prepend) grpc_debug!("(cancel_itinerary MOCK) request: {:?}
X X server/src/grpc/server.rs (274) wrong (fn prepend) grpc_warn!("(cancel_task MOCK) scheduler server.")
X X server/src/grpc/server.rs (275) wrong (fn prepend) grpc_debug!("(cancel_task MOCK) request: {:?}", re
X X server/src/grpc/server.rs (288) wrong (fn prepend) grpc_warn!("(get_task_status MOCK) scheduler serve
X X server/src/grpc/server.rs (289) wrong (fn prepend) grpc_debug!("(get_task_status MOCK) request: {:?}"
X X server/src/grpc/server.rs (307) wrong (fn prepend) grpc_warn!("(is_ready MOCK) scheduler server.");
X X server/src/grpc/server.rs (308) wrong (fn prepend) grpc_debug!("(is_ready MOCK) request: {:?}", reque
X X server/src/grpc/api/query_flight.rs (66) (no_fn_prepend) msg grpc_error!( "{} Invalid dep...
X X server/src/grpc/api/query_flight.rs (78) (no_fn_prepend) msg grpc_error!( "{} Invalid dep...
X X server/src/grpc/api/query_flight.rs (84) (no_fn_prepend) msg grpc_warn!("{} latest arrival time not provided...
X X server/src/grpc/api/query_flight.rs (89) (no_fn_prepend) msg grpc_warn!("{} earliest departure time not prov...
X X server/src/grpc/api/query_flight.rs (100) (no_fn_prepend) msg grpc_warn!( "{} earliest departu...
X X server/src/grpc/api/query_flight.rs (109) (no_fn_prepend) msg grpc_warn!("{} time range too large.", ERROR_PR...
X X server/src/grpc/api/query_flight.rs (114) (no_fn_prepend) msg grpc_warn!("{} latest arrival time is in the pa...
X X server/src/router/flight_plan.rs (114) punctuation router_error!( "(try_from) Flight p
X X server/src/router/flight_plan.rs (136) punctuation router_error!( "(try_from) Flight p
X X server/src/router/vehicle.rs (71) wrong (fn prepend) router_debug!( "(Availability::subtract
X X server/src/router/vehicle.rs (72) wrong (fn prepend) router_debug!("(Availability::subtract) result: {:
X X server/src/router/schedule.rs (289) wrong (fn prepend) router_error!("(Calendar fmt) {}", e);
X X server/src/router/schedule.rs (295) wrong (fn prepend) router_error!("(Calendar fmt) {}", e);
X X server/src/router/schedule.rs (302) wrong (fn prepend) router_error!("(Calendar fmt) {}", e);
X X server/src/router/schedule.rs (335) wrong (fn prepend) router_debug!( "(Calendar to_ti
X X server/src/router/schedule.rs (348) wrong (fn prepend) router_debug!("(Calendar to_timeslots) timeslot: {

(Some fields hidden. Use -t and -c to show hidden fields.)

💀 Checking #[allow(dead_code)]...

Location Statement
client-grpc/src/grpc.rs (258) struct RpcServiceClient

🎀 Checking unwraps and .expect() calls...

Test /// Location
X X client-grpc/examples/grpc.rs (16)
X X client-grpc/examples/grpc.rs (17)

(Some fields hidden. Use -t and -c to show hidden fields.)

🚧 Checking TODO items and FIXMEs...

Test Location Statement
X server/src/tasks/cancel_itinerary.rs (61) // TODO(R4) Don't allow cancellations within X ...
X server/src/tasks/cancel_itinerary.rs (65) // TODO(R4): Heal the gap created by the remove...
X server/src/tasks/cancel_itinerary.rs (114) // TODO(R4): svc-storage currently doesn't chec...
X server/src/tasks/cancel_itinerary.rs (147) // TODO(R4): Don't cancel flight plan if it exi...
X server/src/tasks/cancel_itinerary.rs (172) // TODO(R4): Internal cancellations should chan...
X server/src/tasks/pool.rs (248) // TODO(R4): Update this to a transaction (atom...
X server/src/tasks/pool.rs (382) // TODO(R4): Use a transaction
X server/src/tasks/pool.rs (508) // TODO(R4): Make this section a transaction if...
X server/src/tasks/create_itinerary.rs (19) // TODO(R5): Do this in a transaction if possib...
X server/src/tasks/create_itinerary.rs (50) user_id: uuid::Uuid::new_v4().to_string(), // T...
X server/src/tasks/create_itinerary.rs (161) // TODO(R5): For R4 we'll manually filter out t...
X server/src/grpc/api/confirm_itinerary.rs (7) // TODO(R4): Compliance service will handle thi...
X server/src/grpc/api/confirm_itinerary.rs (101) // TODO(R4): Compliance service will handle thi...
X server/src/grpc/api/confirm_itinerary.rs (141) // TODO(R4) - Check if flight plan is already i...
X server/src/grpc/api/confirm_itinerary.rs (143) // // TODO(R4) - update record with new par...
X server/src/grpc/api/confirm_itinerary.rs (149) // TODO(R4) - insert all flight plans at same t...
X server/src/grpc/api/cancel_itinerary.rs (51) // TODO(R4) Don't allow cancellations within X ...
X server/src/grpc/api/cancel_itinerary.rs (85) //TODO(R4): svc-storage currently doesn't check...
X server/src/grpc/api/cancel_itinerary.rs (110) // TODO(R4): Don't cancel flight plan if it exi...
X server/src/grpc/api/cancel.rs (26) // TODO(R4): Get the itinerary start time from ...
X server/src/grpc/api/query_flight.rs (123) // TODO(R4): Get needed loading/unloading times...
X server/src/grpc/api/query_flight.rs (131) /// TODO(R5): Return a stream of messages for l...
X server/src/grpc/api/query_flight.rs (169) // TODO(R4): Determine if there's an open space...
X server/src/router/vertiport.rs (137) /// TODO(R4): This will be replaced with a call...
X server/src/router/vertiport.rs (151) // TODO(R4): Use each vertipad's calendar
X server/src/router/vertiport.rs (158) // TODO(R4): This is currently hardcoded, get t...
X server/src/router/vertiport.rs (175) // TODO(R4): This will be replaced with a call ...
X server/src/router/vertiport.rs (261) /// TODO(R4): Remove in favor of read from stor...
X server/src/router/vertiport.rs (267) // TODO(R4): This is currently hardcoded, get t...
X server/src/router/vertiport.rs (280) // TODO(R4): duration should be retrieved from ...
X server/src/router/vertiport.rs (289) // TODO(R4): duration should be retrieved from ...
X server/src/router/flight_plan.rs (216) // TODO(R4): Further filter by vehicle type, etc.
X server/src/router/itinerary.rs (154) // TODO(R4): Include vehicle model to improve e...
X server/src/router/itinerary.rs (212) /// TODO(R4): Return more than one itinerary pe...
X server/src/router/itinerary.rs (463) // TODO(R4) - Get nearest open rest stop/hangar...
X server/src/router/vehicle.rs (18) /// TODO(R4): Hardcoded for the demo. This is s...
X server/src/router/vehicle.rs (189) // TODO(R4): Private aircraft, disabled aircraf...
X server/src/router/vehicle.rs (195) // TODO(R4): Ignore aircraft that haven't been ...
X server/src/router/vehicle.rs (229) let aircraft = AircraftType::Cargo; // TODO(R4)...
X server/src/router/vehicle.rs (234) let liftoff_duration_s: f32 = 10.0; // TODO(R4)...
X server/src/router/vehicle.rs (235) let landing_duration_s: f32 = 10.0; // TODO(R4)...

End - Check Todo Items

🗃 Checking Cargo.toml versions...

All good! 🎉

⚠️ Comments and test sections not checked by default. Check locally with make release-checklist-full

@amsmith-pro amsmith-pro mentioned this pull request Nov 22, 2023
Closed
@github-actions GitHub Actions
Copy link

github-actions bot commented Nov 22, 2023 •
edited by owlot
Loading

🪲 Unit Testing Coverage 🔎

|| Tested/Total Lines:
|| client-grpc/src/client.rs: 20/30
|| client-grpc/src/grpc.rs: 0/38
|| server/src/config.rs: 16/19
|| server/src/grpc/api/cancel.rs: 0/1
|| server/src/grpc/api/create.rs: 0/1
|| server/src/grpc/api/query_flight.rs: 30/40
|| server/src/grpc/client.rs: 9/9
|| server/src/grpc/server.rs: 32/62
|| server/src/lib.rs: 12/28
|| server/src/router/flight_plan.rs: 74/117
|| server/src/router/itinerary.rs: 289/333
|| server/src/router/mod.rs: 21/31
|| server/src/router/schedule.rs: 224/264
|| server/src/router/vehicle.rs: 215/268
|| server/src/router/vertiport.rs: 33/150
|| server/src/tasks/cancel_itinerary.rs: 7/56
|| server/src/tasks/create_itinerary.rs: 11/107
|| server/src/tasks/mod.rs: 0/47
|| server/src/tasks/pool.rs: 0/203
|| server/src/test_util.rs: 206/210
||
59.53% coverage, 1199/2014 lines covered

👉 View in Coveralls.io 👈

@owlot owlot merged commit f9ef466 into develop Nov 22, 2023
18 checks passed
@owlot owlot deleted the am-smith/dw-1311/priority-queue-2 branch November 22, 2023 08:35
@amsmith-pro amsmith-pro mentioned this pull request Nov 30, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@owlot owlot owlot approved these changes

Assignees
No one assigned
Labels
cla-signed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@amsmith-pro @owlot