Weekly Recap #1

[kaspth]

Here's the first weekly recap!

• We said Welcome over here Welcome everybody to the first Open Source Retreat! #1

• I shared a bit about how I approach gems, with a sketch of some implementation ideas Using sketches to think out gems #3
  By the way, are y'all receiving notifications about new discussions, because I didn't see much activity on Using sketches to think out gems #3? I've been meaning to write up a guide on some GitHub tips to follow along with the project better, so if GitHub doesn't notify by default I should write how to fix that up.

Code

In #2, I added the first bit of code with some tests. Today, I just fleshed out a whole bunch more structure in #7 and then I added the initial file based loading in #8.

So now we've got this code running:

# test_helper.rb
Oaken::Data.load_from "test/seeds"

# test/seeds/users.rb
User = Struct.new(:name, keyword_init: true)
memory.register :users, User # Uses the Oaken::Stored::Memory provider.

users.update :kasper, name: "Kasper"

# test/seeds/comments.rb
Comment = Struct.new(:name, keyword_init: true)
records.register :comments, Comment # Uses the Oaken::Stored::ActiveRecord provider.

# NOTE: this comments sample is mainly to demo `records` using a different provider.

Contributions

@alexandreruban's kicked us off in #5 and @salochara asked a question in #2.

---

Eventually I'd like the memory.register :users, User lines to go away.

Partly we need to derive User from :users so we don't need that detail (I'm not sure about what to do for namespaces yet, e.g. Post::Comment). If anybody wants to try looking into that, go for it. Or maybe we should flip this so it's memory.register User and then we auto-derive the key? (The latter may also provide a nice register User, some: :default, attributes: "here".)

The other part is having a default provider, so we don't need memory here (and that users could use records by default). Probably, involves def register(...) = default_provider.register(...).

Don't worry about duplicate PRs if you try any of these. We'll figure out how to assign credit or collaborate as needed.

Up next

One of the things I always struggle with is how automatic to make things, and I'm already noticing that pressure. So I'm going to try to think through how we can let things be a little more manual and then we always level-up later once the patterns have been proven out more.

Unlike Fixtures, where a file always maps to a table like test/fixtures/users.yml being User, our files won't map 1-1 to models. We're more like db/seeds.rb in that sense (which is partly why I've started out with test/seeds in our test directory here).

Anyway, I'm hoping that we can have something usable by next week. It might be nice just to map something like:

# test/fixtures/users.yml
one:
  name: Yo

two:
  name: Hm

# test/fixtures/posts.yml
first:
  user: one
  title: First

second:
  user: one
  title: Second

into

# test/seeds/primary.rb # Not entirely sure about the filename.
register User # Assumes we've got the default provider setup as `records`.
users.update :one, name: "Yo"
users.update :two, name: "Hm"

register Post
posts.update :first,  user: users.one, title: "First"
posts.update :second, user: users.one, title: "Second"

I'm hopeful, we'll see! Some of the these early decisions can be really important because they have such a cascading effect down the line.

---

How's the volume so far? Has it been easy to follow along with? Feel free to ask any other questions below here.

cc @alexandreruban @ashwin47 @codergeek121 @diogochaves @felipedmesquita @jean-francois-labbe @lape @larkro @manuelfcreis @manuraj17 @marcelolx @mikepmunroe @mingabire @ricardotk002 @sahglie @salochara @swanson @thomasklemm @willcosgrove

[kaspth]

Oh! I forgot to mention in #4 we set up a section for attributions, if any of y'all would like one open a PR and add your name with - Your Name (in alphabetical order) below the Arrows logo. You can use GitHub's web editor, and/or a local clone — you can reply to this comment if you need help too.

[kaspth]

Reply here I mean.

[manuelfcreis]

By the way, are y'all receiving notifications about new discussions

Yes! I think we are, or at least if we "watch" the Repo. I did receive an e-mail for this one.

[kaspth]

Sweet, thanks!

[manuelfcreis]

One of the things I always struggle with is how automatic to make things, and I'm already noticing that pressure. So I'm going to try to think through how we can let things be a little more manual and then we always level-up later once the patterns have been proven out more.

💯 this. It'll also be easier to automate once it's more functional.

Unlike Fixtures, where a file always maps to a table like test/fixtures/users.yml being User, our files won't map 1-1 to models. We're more like db/seeds.rb in that sense (which is partly why I've started out with test/seeds in our test directory here).

A couple of thoughts that this made me think about:

• We're assuming that users will always load all the seeds, or should we expect some sort of piecemeal seeding, i.e. defining in the beginning of the test which set of seeds to load, or in the test helper what the base seeding script is

• IMO one of the annoying things about fixtures is that the errors are typically very unhelpful:
  RuntimeError: Foreign key violations found in your fixture data. Ensure you aren't referring to labels that don't exist on associations. We definitely need 1. To think about how to handle errors, 2. Make them more helpful.

How's the volume so far? Has it been easy to follow along with? Feel free to ask any other questions below here.

Totally fine 👍🏻 I've been mostly reading over the discussions and the initial commits.

As an aside, I just noticed that there's a nice Tree motif running around "Oaken" as we're mostly using "seeds". Not sure if intentional 😅

[kaspth]

We're assuming that users will always load all the seeds, or should we expect some sort of piecemeal seeding, i.e. defining in the beginning of the test which set of seeds to load, or in the test helper what the base seeding script is

Yeah, I'm not quite sure of this either yet. One of the goals of this project is also to give people more guidance to think of their app or models from the object graph level — and how they connect. That's what the language of primary/secondary/tertiary or stem/branch/leaf in #3 is trying to get at. The idea being that primary defines the base set of models like all the User/Account/Team/Membership in the seed, then secondary is the next level of that, and finally tertiary are all the rest of the models that either hang off of primary or secondary models. It's definitely not fleshed out yet, and I think your instinct with looking at partial seeding is accurate too.

IMO one of the annoying things about fixtures is that the errors are typically very unhelpful:
RuntimeError: Foreign key violations found in your fixture data. Ensure you aren't referring to labels that don't exist on associations. We definitely need 1. To think about how to handle errors, 2. Make them more helpful.

Totally! That's where I think swapping to a Ruby API can make things easier rather than having to learn an extra YML-based syntax and its quirks.

As an aside, I just noticed that there's a nice Tree motif running around "Oaken" as we're mostly using "seeds". Not sure if intentional 😅

Intentional! That's also potentially the idea with stem/branch/leaf, but we'll see 😄

[alexandreruban]

How's the volume so far? Has it been easy to follow along with? Feel free to ask any other questions below here.

Totally fine as well. It's interesting to see your thought process from your sketches #3 and how you seem to think carefully about the API first (rather than how it is going to be implemented under the hood).

[kaspth]

Awesome, that's great to hear!

[manuraj17]

Just finished going through all the changes, was caught up and couldn't go through.

How's the volume so far? Has it been easy to follow along with? Feel free to ask any other questions below here.

Been good, following midweek might be easy with smaller PRs although that's how it's going on now so it's fine 👍🏽

I liked the idea of not being 1:1. If 1:1 is strictly required they can have a file so, but combined is good. I think this can help in making this gem standalone for creating seed data for demo etc like a seed file which has data to create a subscription of 1 year and 3 users, where 1 has cancelled after 2 months.

Eventually I'd like the memory.register :users, User lines to go away.

Partly we need to derive User from :users so we don't need that detail (I'm not sure about what to do for namespaces yet, e.g. Post::Comment). If anybody wants to try looking into that, go for it. Or maybe we should flip this so it's memory.register User and then we auto-derive the key? (The latter may also provide a nice register User, some: :default, attributes: "here".)

Not sure if it sounds dumb but can a block structure help to associate nested structure? I might need to go through the code again but two scenarios is what I think of

1. Namespacing via an existing model - I haven't used this pattern yet so might be wrong here. In this case we can do block scoping like

memory.register :posts, do
  register :comments #using only register here because we are already inside `memory`
end

2. Just module namespaces - In this case we can explicitly pass just the namespace

memory.register :comments, { namespace: :post }

Something like this?

[kaspth]

I liked the idea of not being 1:1. If 1:1 is strictly required they can have a file so, but combined is good. I think this can help in making this gem standalone for creating seed data for demo etc like a seed file which has data to create a subscription of 1 year and 3 users, where 1 has cancelled after 2 months.

Yes exactly! Those kinds of scenarios are what's difficult with FactoryBot because everything's meant to be standalone and recreated on use. Same for regular Fixtures because they're all split by table.

[kaspth]

Good point on namespaces too, I'm really not sure how to handle those yet.

I'm almost wondering if we should go with something like:

table :users do
  update :kasper, name: "Kasper"
end

# and also supporting:
table :users
users.update :kasper, name: "Kasper"

That would let us target a table directly and avoid namespace issues. We'd also be at a lower level so we'd get out of this sorta strange in-between we're in now where the models are sometimes there. I'm gonna have to think on this some more.

[kaspth]

I'd been concerned with the performance if we loaded model classes, but looks like regular Fixtures just do that anyway https://github.com/rails/rails/blob/b6669e5605504362266b7d9c833ed88350a10236/activerecord/lib/active_record/fixtures.rb#L746

So maybe we don't need to go to the table like API.