Models and Resources can now be rendered by Roda automatically (#845)

* This makes working with content within SSR context much more straighforward.
* I also added more documentation around server rendering, with still more to come.
* bridgetown-routes now supports initializer-based config.
* Bridgetown 2.0 will consider both Server Rendering and Island Architecture to be out of "experimental" status.

* BREAKING CHANGES: collection resources are now read on Roda SSR boot by default, with possible performance implications on large site projects. This is configurable. In addition, the handling of file-based routes within Islands has changed, though the previous behavior was undocumented.

bridgetown-core/lib/bridgetown-core/concerns/site/ssr.rb

@@ -33,6 +33,19 @@ def enable_ssr
 
     def ssr_setup(&block)
       config.serving = true
+
+      Bridgetown::Hooks.trigger :site, :pre_read, self
+      ssr_first_read
+      Bridgetown::Hooks.trigger :site, :post_read, self
+
+      block&.call(self) # provide additional setup hook
+
+      return if Bridgetown.env.production?
+
+      Bridgetown::Watcher.watch(self, config, &block)
+    end
+
+    def ssr_first_read
       Bridgetown::Hooks.trigger :site, :pre_read, self
       defaults_reader.tap do |d|
         d.path_defaults.clear
@@ -43,12 +56,13 @@ def ssr_setup(&block)
         coll.read
         self.data = coll.merge_data_resources
       end
-      Bridgetown::Hooks.trigger :site, :post_read, self
-
-      block&.call(self) # provide additional setup hook
-      return if Bridgetown.env.production?
 
-      Bridgetown::Watcher.watch(self, config, &block)
+      # This part is handled via a hook so it is supported by the SSR "soft reset"
+      Bridgetown::Hooks.register_one :site, :post_read, reloadable: false, priority: :high do
+        reader.read_directories
+        reader.sort_files!
+        reader.read_collections
+      end
     end
 
     def disable_ssr

bridgetown-core/lib/bridgetown-core/reader.rb

@@ -38,7 +38,7 @@ def read_collections
       site.collections.each_value do |collection|
         next if collection.data?
 
-        collection.read
+        collection.read unless site.ssr? && collection.metadata.skip_for_ssr
       end
     end
 
@@ -78,7 +78,7 @@ def read_directories(dir = "")
 
       retrieve_dirs(dir, entries_dirs)
       retrieve_pages(dir, entries_pages)
-      retrieve_static_files(dir, entries_static_files)
+      retrieve_static_files(dir, entries_static_files) unless site.ssr?
     end
 
     # Recursively traverse directories with the read_directories function.
@@ -101,6 +101,8 @@ def retrieve_dirs(dir, entries_dirs)
     # @param entries_pages [Array<String>] page paths in the directory
     # @return [void]
     def retrieve_pages(dir, entries_pages)
+      return if site.ssr? && site.collections.pages.metadata.skip_for_ssr
+
       entries_pages.each do |page_path|
         site.collections.pages.read_resource(site.in_source_dir(dir, page_path))
       end

bridgetown-core/lib/bridgetown-core/utils/loaders_manager.rb

@@ -64,7 +64,7 @@ def setup_loaders(autoload_paths = []) # rubocop:todo Metrics/AbcSize, Metrics/C
           loader.enable_reloading if reloading_enabled?(load_path)
           loader.ignore(File.join(load_path, "**", "*.js.rb"))
           loader.ignore(
-            File.join(File.expand_path(config[:islands_dir], config[:source]), "routes")
+            File.join(File.expand_path(config[:islands_dir], config[:source]), "**", "routes")
           )
           config.autoloader_collapsed_paths.each do |collapsed_path|
             next unless collapsed_path.starts_with?(load_path)

bridgetown-core/lib/roda/plugins/bridgetown_server.rb

@@ -41,6 +41,17 @@ def self.load_dependencies(app) # rubocop:disable Metrics
           "500 Internal Server Error"
         end
 
+        # This lets us return models or resources directly in Roda response blocks
+        app.plugin :custom_block_results
+
+        app.handle_block_result Bridgetown::Model::Base do |result|
+          result.render_as_resource.output
+        end
+
+        app.handle_block_result Bridgetown::Resource::Base do |result|
+          result.transform!.output
+        end
+
         ExceptionPage.class_eval do # rubocop:disable Metrics/BlockLength
           def self.css
             <<~CSS

bridgetown-core/test/ssr/config/initializers.rb

@@ -5,4 +5,21 @@
   init :local_ssr_init, require_gem: false
   init :local_ssr_init, require_gem: false
   init :local_ssr_init, require_gem: false
+
+  collections do
+    posts do
+      skip_for_ssr true
+    end
+  end
+
+  # poor man's Inspector plugin
+  hook :resources, :post_render do |resource|
+    next unless resource.site.root_dir.end_with?("test/ssr") # ugly hack or else test suite errors
+
+    document = Nokogiri.HTML5(resource.output)
+    document.css("p.test").each do |paragraph|
+      paragraph.inner_html = paragraph.inner_html.upcase
+    end
+    resource.output = document.to_html
+  end
 end

bridgetown-core/test/ssr/config/local_ssr_init.rb

@@ -2,7 +2,7 @@
 
 Bridgetown.initializer :local_ssr_init do |config|
   config.init :ssr do
-    setup ->(site) do
+    setup -> site do # rubocop:disable Layout/SpaceInLambdaLiteral, Style/StabbyLambdaParentheses
       site.data.iterations ||= 0
       site.data.iterations += 1
     end

bridgetown-core/test/ssr/server/routes/render_resource.rb

@@ -0,0 +1,16 @@
+# frozen_string_literal: true
+
+class Routes::RenderResource < Bridgetown::Rack::Routes
+  route do |r|
+    # route: GET /render_resource
+    r.get "render_resource" do
+      # Roda should know how to autorender the resource
+      bridgetown_site.collections.pages.resources.find { _1.id == "repo://pages.collection/index.md" }
+    end
+
+    r.get "render_model" do
+      # Roda should know how to autorender the model as a resource
+      Bridgetown::Model::Base.find("repo://pages.collection/test_doc.md")
+    end
+  end
+end

bridgetown-core/test/ssr/src/index.md

@@ -1 +1,4 @@
+---
+---
+
 Hello **world**!

bridgetown-core/test/ssr/src/test_doc.md

@@ -0,0 +1,6 @@
+---
+title: Test Document
+---
+
+This is a _test_.
+{: .test}

bridgetown-core/test/test_ssr.rb

@@ -71,5 +71,19 @@ def site
       assert last_response.ok?
       assert_equal("Redirected!", last_response.body)
     end
+
+    should "return rendered resource" do
+      get "/render_resource"
+
+      assert last_response.ok?
+      assert_includes last_response.body, "<p>Hello <strong>world</strong>!</p>"
+    end
+
+    should "return model as rendered resource" do
+      get "/render_model"
+
+      assert last_response.ok?
+      assert_includes last_response.body, "<p class=\"test\">THIS IS A <em>TEST</em>.</p>"
+    end
   end
 end

bridgetown-routes/.rubocop.yml

@@ -4,3 +4,4 @@ inherit_from: ../.rubocop.yml
 AllCops:
   Exclude:
     - "*.gemspec"
+    - "test/ssr/src/_islands/**/*.rb"

bridgetown-routes/lib/bridgetown-routes.rb

@@ -11,13 +11,20 @@ module Routes
 end
 
 # @param config [Bridgetown::Configuration::ConfigurationDSL]
-Bridgetown.initializer :"bridgetown-routes" do |config|
+Bridgetown.initializer :"bridgetown-routes" do |
+  config,
+  additional_source_paths: [],
+  additional_extensions: []
+|
   config.init :ssr # ensure we already have touchdown!
 
   config.routes ||= {}
-  config.routes.source_paths ||= ["_routes", "#{config.islands_dir}/routes"]
+  config.routes.source_paths ||= ["_routes"]
   config.routes.extensions ||= %w(rb md serb erb liquid)
 
+  config.routes.source_paths += Array(additional_source_paths)
+  config.routes.extensions += Array(additional_extensions)
+
   config.only :server do
     require_relative "bridgetown-routes/manifest_router"
   end

bridgetown-routes/lib/bridgetown-routes/manifest.rb

@@ -10,7 +10,7 @@ def generate_manifest(site) # rubocop:todo Metrics/AbcSize, Metrics/CyclomaticCo
           new_manifest = []
           routable_extensions = site.config.routes.extensions.join(",")
 
-          site.config.routes.source_paths.each do |routes_dir|
+          expand_source_paths_with_islands(site.config).each do |routes_dir|
             routes_dir = File.expand_path(routes_dir, site.config.source)
 
             # @type [Array]
@@ -66,13 +66,26 @@ def locale_for(slug, site)
 
         private
 
-        def file_slug_and_segments(site, routes_dir, file)
+        # Add any `routes` folders that may be living within islands
+        def expand_source_paths_with_islands(config)
+          @islands_dir ||= File.expand_path(config.islands_dir, config.source)
+
+          # clear out any past islands folders
+          config.routes.source_paths.reject! { _1.start_with?(@islands_dir) }
+
+          Dir.glob("#{@islands_dir}/**/routes").each do |route_folder|
+            config.routes.source_paths << route_folder
+          end
+
+          config.routes.source_paths
+        end
+
+        def file_slug_and_segments(_site, routes_dir, file)
           # @type [String]
           file_slug = file.delete_prefix("#{routes_dir}/").then do |f|
-            if routes_dir.start_with?(
-              File.expand_path(site.config[:islands_dir], site.config.source)
-            )
-              f = "#{site.config[:islands_dir].delete_prefix("_")}/#{f}"
+            if routes_dir.start_with?(@islands_dir)
+              # convert _islands/foldername/routes/someroute.rb to foldername/someroute.rb
+              f = routes_dir.delete_prefix("#{@islands_dir}/").sub(%r!/routes$!, "/") + f
             end
             [File.dirname(f), File.basename(f, ".*")].join("/").delete_prefix("./")
           end.delete_suffix("/index")

bridgetown-routes/lib/roda/plugins/bridgetown_routes.rb

@@ -57,14 +57,15 @@ def render_with(data: {}) # rubocop:todo Metrics/AbcSize, Metrics/MethodLength
 
           Bridgetown::Model::Base.new(data).to_resource.tap do |resource|
             resource.roda_app = self
-          end.read!.transform!.output
+          end.read!
         end
 
         def render(...)
           view.render(...)
         end
 
         def view(view_class: Bridgetown::ERBView)
+          # TODO: support user choosing templates by extension rather than class
           response._fake_resource_view(
             view_class:, roda_app: self, bridgetown_site:
           )

bridgetown-routes/test/ssr/config/initializers.rb

@@ -2,9 +2,8 @@
 
 Bridgetown.configure do |config|
   require "bridgetown-routes"
-  init :"bridgetown-routes", require_gem: false
-
-  routes.source_paths << File.expand_path("alt_routes", "#{root_dir}/../")
+  init :"bridgetown-routes", require_gem: false, additional_source_paths:
+    File.expand_path("alt_routes", "#{root_dir}/..")
 
   config.available_locales = [:en, :it]
   config.default_locale = :en

bridgetown-routes/test/ssr/src/_islands/paradise/routes/dreamy.rb

@@ -0,0 +1,7 @@
+###ruby
+render_with data: {
+  title: "Ah, island life…"
+}
+###
+
+data.title + " =)"

bridgetown-routes/test/ssr/src/_islands/paradise/routes/index.rb

@@ -0,0 +1,7 @@
+###ruby
+render_with data: {
+  title: "Living in paradise"
+}
+###
+
+data.title + " =)"

bridgetown-routes/test/test_routes.rb

@@ -56,5 +56,15 @@ def site
       get "/nested/123-abc"
       assert_equal "<h1>Nested Page with Slug: 123-abc</h1>\n", last_response.body
     end
+
+    should "return the proper route within an island" do
+      get "/paradise" do
+        assert_equal "Living in paradise =)", last_response.body
+      end
+
+      get "/paradise/dreamy" do
+        assert_equal "Ah, island life… =)", last_response.body
+      end
+    end
   end
 end

bridgetown-website/src/_docs/configuration/initializers.md

@@ -219,6 +219,39 @@ While it's not strictly required that you place a Roda block inside of an `only
   As mentioned above, you can still add and configure plugins directly in your Roda class file (`server/roda_app.rb`) just like any standard Roda application, but using a Roda configuration block alongside your other initialization steps is a handy way to keep everything consolidated. Bear in mind that the Roda blocks are all executed prior to anything defined within the class-level code of `server/roda_app.rb`, so if you write any code in a Roda block that relies on state having already been defined in the app class directly, it will fail. Best to keep Roda block code self-contained, or reliant only on other settings in the Bridgetown initializers file.
 {% end %}
 
+### SSR & Dynamic Routes
+
+The SSR features of Bridgetown, along with its companion file-based routing features, are now configurable via initializers.
+
+```rb
+init :ssr
+
+# optional:
+init :"bridgetown-routes"
+
+# …or you can just init the routes, which will init :ssr automatically:
+
+init :"bridgetown-routes"
+```
+
+If you want to run some specific site setup code on first boot, or any time there's a file refresh in development, provide a `setup` block inside of the SSR initializer.
+
+```rb
+init :ssr do
+  setup -> site do
+    # access the site object, add data with `site.data`, whatever
+  end
+end
+```
+
+For the file-based routing plugin, you can provide additional configuration options to add new source paths (relative to the `src` folder, unless you specify an absolute file path) or add other routable extensions (for example to support a custom template engine):
+
+```rb
+init :"bridgetown-routes", additional_source_paths: ["some_more_routes"], additional_extensions: ["tmpl"]
+```
+
+For more on how SSR works in Bridgetown, check out our [Routes documentation here](/docs/routes).
+
 ## Low-level Boot Customization
 
 If you need to run Ruby code at the earliest possible moment, essentially right when the `bridgetown` executable has finished its startup process, you can add a `config/boot.rb` file to your repo. This is particularly useful if you wish to extend `bridgetown` with new commands.
@@ -241,23 +274,6 @@ require_relative "../ruby_code_file.rb"
 
 Bridgetown ships with several initializers you can add to your configuration. In future versions of Bridgetown, we expect to make our overall architecture a little more modular so you can use the initializer system to specify just those key features you need (and by omission which ones you don't!).
 
-### SSR & Dynamic Routes
-
-The SSR features of Bridgetown, along with its companion file-based routing features, are now configurable via initializers.
-
-```rb
-init :ssr
-
-# optional:
-init :"bridgetown-routes"
-
-# …or you can just init the routes, which will init :ssr automatically:
-
-init :"bridgetown-routes"
-```
-
-Check out our [Routes documentation here](/docs/routes).
-
 ### Dotenv
 
 The Dotenv gem provides a simple way to manage environment variables with your Bridgetown project. Simply add the gem to your Gemfile (`bundle add dotenv`), and then add the initializer to your configuration:

bridgetown-website/src/_docs/content/dsd.md

@@ -136,7 +136,7 @@ What's great about this approach is:
 In addition to the benefits above, you also have the ability to leverage [CSS Shadow Parts](https://developer.mozilla.org/en-US/docs/Web/CSS/::part) (which only work when you have, er, shadow DOM—hence the name!). What's a shadow part? It's when you use the `part=` attribute on an element inside your DSD template, and by doing so it makes it styleable from the "outside". Defining parts and labeling them appropriately is a fantastic way to build up a true "style API" for each layout or component.
 
 {%@ Note do %}
-Declarative Shadow DOM is a fairly new specification. As of the time of this writing, Firefox (and some older versions of Safari) do not offer built-in DSD support. The `<is-land>` web component automatically polyfills DSD, which is an added benefit of using it. Otherwise, the Turbo bundled configuration also includes a site-wide polyfill for DSD.
+Declarative Shadow DOM is a fairly new specification. As of the time of this writing, some older versions of Safari and Firefox do not offer built-in DSD support. The `<is-land>` web component automatically polyfills DSD, which is an added benefit of using it. Otherwise, the Turbo bundled configuration also includes a site-wide polyfill for DSD.
 {% end %}
 
 ## Components with Sidecar CSS