Move Ibis tutorials away from ibis-project/ibis to ibis-project/ibis-tutorials

[p-a-a-a-trick]

Proposal

Move the ibis tutorial notebooks away from ibis-project/ibis over to a new repository, ibis-project/ibis-tutorials

Reasoning

By moving the notebooks away from the main repository we can:

1. have a designated space for interactive tutorials
2. use a platform like binder to host the notebooks so users can run them before installing
3. clean up the website a bit
4. speed up mkdocs serve

Example

Repository:

• https://github.com/p-a-a-a-trick/ibis-tutorial-notebooks

Binder:

• 

This works out of the box for the most part. We would need to update how users get geography.db, and my personal opinion is that we should prompt users to type the commands (after we tell them the commands) to make sure that they're following along and don't fall into a shift+enter fugue state.

[saulpw]

I like the idea of having separate places for tutorials and docs, as it would make some things cleaner. However, the main issue is keeping them up-to-date with the code, and versioning sync between the code and docs. Recently I made a small API change, and git grep showed that the function was used in a notebook, so I did the search-and-replace inside the notebook too. If it had been in a different repo, I would have missed it. Also, finding version-specific tutorials is easy if they are in the same repo as the main code--just check out the version of the code, and the tutorials at that tag should meant for that right version. Otherwise we have to have a more complicated process for knowing which version of the tutorial repo corresponds to which version of the code repo.

[p-a-a-a-trick]

This makes sense, and great point.

I'm sure there's some CI process out there that keeps repos in sync wrt versioning and reference; I'm sure this isn't the first time someone's wanted to keep code and tutorials in separate repos. I'll do some digging to see if there's an easy/established way of doing this.

Beside that, maybe it would be cleaner to update tutorials as a separate task? So part of a major release would be to comb through notebooks and fix breaking changes, and fixing bugs or making slight modifications in the tutorials would be handled in the tutorials repository rather than as a PR in the main repo.

Part of setting up the binder instance is using a requirements file, which we can lock to specific versions. We can use this as a way to tell people what version the tutorials are on and as a way to release new versions of the tutorial. Seems complicated separately, but then again there might already be an established process for this. I'll look around.

My main motivation is that I think part of why the docs are so tedious to fix is the notebook rendering stage. My prototyping a simple restructure is incredibly tedious; every change requires a 3-5 minute wait before preview, and if that change is broken I need wait another 3-5 minutes. Maybe I'm doing something wrong.

Also, having it hosted on a sandbox somewhere makes it a bit easier for people to play around with it without going through the install process. Supposing someone is on windows and doesn't have conda (e.g. an employee at a company that uses a platform with rigorous protocols in place to install new software), they can just play around with ibis in their browser before going through whatever install process they need to go through to get it working on their system.

We might be able to do that in mkdocs, but just using an existing platform like Binder is a lot easier and more user-friendly (for users familiar with jupyter notebooks/binder).

If this generally isn't worth moving the tutorials over, I can live with that no problem, but it's definitely an obstacle for people wanting to make small administrative changes to the website, and I do think we should pursue some way for users to test out ibis themselves without going through the install process.

[saulpw]

So it sounds like the primary issue is the how tedious it is to change the docs, which is due to how long the notebook rendering takes. Let's try to fix that core issue because that is clearly getting in the way of documentation embetterment.

The proposed solution, of moving the tutorials out, creates other complications; trying to solve those with more CI process makes the whole system more complicated. As they say, the best code is no code. At present, the docs and code can "naturally" stay in sync because of how the system is structured, and it's undesirable to have to solve this already-solved problem with additional process.

Beside that, maybe it would be cleaner to update tutorials as a separate task? So part of a major release would be to comb through notebooks and fix breaking changes, and fixing bugs or making slight modifications in the tutorials would be handled in the tutorials repository rather than as a PR in the main repo.

Thanks to @cpcloud's army of bots, we are heading towards a model in which releases are fully automated and ideally require zero extra tasks besides "pushing the button". Combing through notebooks looking for breaking changes is a personal nightmare and not something I want to build into our process (though if you're motivated to do so, either one time or for every release, please do! I'm sure there are things that are stale and could use some updates).

[cpcloud]

This is done, we now have https://github.com/ibis-project/ibis-examples for this, please take a look!