Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use actioncable to track indexing progress #3182

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

Conversation

dnoneill
Copy link
Contributor

@dnoneill dnoneill commented Oct 9, 2024

closes #3049

@dnoneill dnoneill force-pushed the ws-progress branch 3 times, most recently from 5f4c782 to 3858c27 Compare October 9, 2024 18:58
app/components/spotlight/progress_bar_component.html.erb Outdated Show resolved Hide resolved
<%= t("#{translation_field}.error") %>
</p>

<div class="progress">
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No. This is a bootstrap class: https://getbootstrap.com/docs/4.0/components/progress/. Changing this to it breaks the CSS.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't the semantic element better for accessibility than a bootstrap component?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably. I think if we are trying to solve the accessibility issue then we should open a new ticket. This is the progress bar that was used in app/views/spotlight/catalog/_reindex_progress_panel.html.erb and app/views/spotlight/bulk_updates/_progress_panel.html.erb

File.open(csv_path) do |f|
progress&.total = f.each_line.count(&:present?) - 1 # ignore the header

::CSV.table(f, header_converters: header_converter).each do |row|
process_row(exhibit, row)
progress&.increment
ActionCable.server.broadcast 'progress_channel',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be a different channel from the one the indexing uses?

Copy link
Contributor Author

@dnoneill dnoneill Oct 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They use the same JS file/functions. I would need to update a bunch of JS.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be clear, I'm not saying that it should be. I'm just asking whether you considered it and why you decided that one channel is the best approach.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I considered it. Seemed like I would just be adding another listener for a different channel that would do the exact same thing.

https://github.com/projectblacklight/spotlight/pull/3182/files#diff-a9fcc060ea558da08359b8855bc3feb142bf3f85f9ab5039fa55e82739a310c6R12-R18

@@ -29,5 +30,7 @@
"not IE 11"
],
"dependencies": {
"@rails/actioncable": "^7.2.100",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Won't this depend on the rails version the host app is using?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are suggesting I update the version to be a range?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We want to allow rails 7.1 and rails 8.0 (eventually), right? I think some kind of range would be preferable.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I just wanted to make sure I understood the suggestion and it wasn't something else.

@@ -24,7 +25,7 @@ const rollupConfig = {
name: ESM ? undefined : 'Spotlight'
},
external,
plugins: [includePaths(includePathOptions)]
};
plugins: [includePaths(includePathOptions), nodeResolve()]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is this plugin for?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't work without it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What was the nature of the failure?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(!) Unresolved dependencies
https://rollupjs.org/troubleshooting/#warning-treating-module-as-external-dependency
@rails/actioncable (imported by "app/javascript/spotlight/channels/consumer.js")
created app/assets/javascripts/spotlight/spotlight.esm.js in 184ms

@@ -0,0 +1 @@
// Import all the channels to be used by Action Cable
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this file needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is automatically created by ActionCable. I wasn't sure if it was something we would want to keep.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automatically created into an engine? How does that work?

Copy link
Contributor Author

@dnoneill dnoneill Oct 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In a site. I created the site ran the auto creation task and moved everything.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You don't want to move these files into the engine. They should be generated by the auto creation task in the host app.

@jcoyne
Copy link
Member

jcoyne commented Oct 9, 2024

Does this impose a new Redis dependency on everyone who wants to run Spotlight?

@dnoneill
Copy link
Contributor Author

dnoneill commented Oct 9, 2024

@jcoyne Yeah but the way I understood it, it was already a dependency of spotlight?

@jcoyne
Copy link
Member

jcoyne commented Oct 9, 2024

@dnoneill redis is not a dependency of spotlight as far as I know.

// Action Cable provides the framework to deal with WebSockets in Rails.
// You can generate new channels where WebSocket features live using the `bin/rails generate channel` command.

import { createConsumer } from "@rails/actioncable"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this file belongs in the engine. It should be in the host application. By doing that you can remove the @rollup/plugin-node-resolve

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are suggesting this goes in the install generator?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't it already in the actioncabel generator?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. Okay so you are suggesting I put the

  1. rails generate channel NameOfChannel into the install generator. Remove app/channels/application_cable/channel.rb, app/channels/application_cable/connection.rb, and app/javascript/channels/consumer.js.
  2. Either make the name of channel ProgressChanel and update the app/channels/progress_channel.rb to include the right stream in the generator or make it a random channel that doesn't get used

I am guessing for existing apps we are going to need to include instructions on how to install. I am not quite sure how this works with existing apps.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that is the path I would use. Yes, we would need instructions for how to install this. It should include something like "Install and configure Redis and actioncable" (assuming we're not doing Solid + Rails 8).

@hackartisan
Copy link

@dnoneill could this please link to an issue that says what / why

@dnoneill
Copy link
Contributor Author

moving into draft. Got 90% of the way to moving all the actioncable stuff into the engine. Unfortunately we couldn't figure out how to import channels/consumer into sprockets without it erroring.

It also checks for actioncable, if not falls back to previous behavior.

@dnoneill dnoneill marked this pull request as draft October 10, 2024 19:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Use ActionCable to report indexing progress
3 participants