Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updating readme #26

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Updating readme #26

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 25 additions & 15 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,30 @@
# Stan-dev design documents repo
This is where we will start putting design documents and functional specs that attempt to describe major changes to our code base. We're taking heavy inspiration from the [Rust project's version](https://github.com/rust-lang/rfcs)
# Stan Design Documents Repository

This repository is where the Stan community puts design documents and functional specifications that describe major changes to our code base. These documents cover many aspects of the Stan ecosystem including the Stan language, interface changes, algorithms, and implementation details.

We welcome proposals and discussion from everyone in the community.

Our process for accepting proposals takes inspiration from [IETF Internet-Drafts](https://www.ietf.org/standards/ids/). We want the process to be lightweight and result in accepting documents that have consensus from the community at the time of approval.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems suboptimal to edit history here - this process was inspired by the Rust process. Maybe rephrase give a little more background on which parts we liked from that process and which parts we want to emphasize that we're doing differently? I think Bob gave a good minimal diff in the discourse discussion.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the comment. The point isn't to edit history, but to describe what the current process is. We could just remove this if having a reference doesn't help.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd rather reference rust here as well, imo we want a rust rfc lite

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is that actually what we want? It seemed like on the thread it wasn’t what we were doing or attempting to do. Happy to have that language if that really reflects what we want. (I’m agnostic. I read the Rust RFC and it doesn’t sound like what we’re attempting to do. Even the goals of it. It really sounds more like the IETF Internet-Drafts.)



## Submit a Design Document
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I liked this section better after the context for what design review is and when it's necessary, because then folks have to read or skip (and thus become aware) of those sections on the way to the meat of the process. What are your thoughts on the ordering here?

Updating it to not be in list form seems reasonable.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I moved it all the way to the top intentionally; something should be at the top describing the goals and how and when things get merged. Maybe what would help is to have a second section below that gets into more detail?


Anyone can propose a design document. Please write a design document roughly following [the template](0000-template.md) and submit a pull request. The author of the specification has the flexibility to drive the process to getting the document approved. We suggest discussing the proposal in a pull request or on [Discourse](https://discourse.mc-stan.org).

Design documents rarely go through the process unchanged, especially as alternatives and drawbacks are identified. Proposals can go through edits for months as we discuss major changes to Stan. Everyone in the community is welcome to make comments (relevant to the proposal).

Proposals are accepted and merged into this repository when there is consensus from the community. In practice, this means giving enough time for members of the community to review/comment/request changes on the proposal. Proposals can be accepted if at least one other (non-author) Stan developer endorses the final draft of the design document and there are no denials.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please nix the consensus part here as well - that was intentionally not in the original. I don't think there is a majority of developers who want to move in a direction where major decisions require consensus, and we shouldn't sneak it in here like this.

Copy link
Member Author

@syclik syclik Aug 6, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is actually one of the recurring points that came forward in this discussion: https://discourse.mc-stan.org/t/process-for-design-docs/14956

Specific examples from the thread:

If there is a way to describe what we want better, let's do that! I honestly think the discussion has stated that they're striving for consensus. To be clear, we're talking about consensus having the definition of "general agreement." It's also a mix of the voting rule of "general consent," but since PRs aren't votes and we don't have a list of members that vote, we never have a vote so this doesn't apply. I think "consensus" is the right term (taking it to mean "most people that care would agree"), but happy to have different wording.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just for clarity "majority" doesn't equal "consensus."

Majority requires > 50% of some group.

There's also a second definition of consensus that applies: "the judgment arrived at by most of those concerned." And that's really what we're after.

Copy link
Member Author

@syclik syclik Aug 6, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the last sentence in the document I proposed is incorrectly worded:
"Proposals can be accepted if at least one other (non-author) Stan developer endorses the final draft of the design document and there are no denials."

We can have “denials” and still merge it in.

Copy link
Member

@seantalts seantalts Aug 6, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, I think we agree we don't want to word this as if each design doc required consensus. Nor do we (most of us at least, citing your Bob quote above) want a vetocracy, which I think is how it's worded now. How about,

"Design docs are a chance for the community to discuss and get into alignment on the pros and cons of the proposal. This means giving enough time for members of the community to review/comment/request changes on the proposal. After that time period (TODO: select a time period?) has ended, a proposal can be accepted if at least one other Stan developer endorses the proposal and merges it. Disagreements with merged proposals can be taken up via the SGB technical issue voting proposal as with other technical issues."

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For me here the key is both making sure issues are fully aired and discussed and everyone can be brought as on board as possible while still making forward progress the default.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW I am in favor of this rewording. It's essentially the same thing but does not use the word consensus and the added mention of going to a vote is also a plus.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 from me too!

Yes, the original one wasn't true to the discussion and should definitely be changed.

(If people are allergic to the term "consensus" we don't have to use it.)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And... thank you for the edit!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yay! Haha, I think that's a good way to put it - I do feel allergic to the term "consensus" 😂


Stan Developers: please feel free to merge PRs into this repository when they have reached this point. Acceptance into this repository is non-binding; at the time of the merge, we, the community, think it's a good idea.


# Design Review

## When is design necessary?

Most changes can be implemented and reviewed via the [normal pull request workflow](https://github.com/stan-dev/math/wiki/Developer-Doc#pull-requests). Some changes are substantial enough that we ask that these be put through a design review for approval, similar to how an implementation is approved in a pull request.


## What is a design?

A major change to Stan will include at least two designs. The first is a functional specification that defines user-facing changes; an example here might be new proposed syntax for the Stan language, or an alternative architecture around how the interfaces communicate with a Stan model. The functional specification should be specific enough that a user/client would understand the impact of the change and start out summarizing the proposed change at a very high level in a sentence or two, identifying all of the clients, laying out an ontology of objects the change traffics in from the perspective of these clients, and then highlighting any additional concerns.
Expand All @@ -15,7 +33,6 @@ Once a functional spec has been agreed upon, we next need a design codified as a

[This design](https://github.com/stan-dev/design-docs/blob/master/designs/0001-logger-io.md) has a functional spec in the first 5 or 6 pages and then a fairly detailed technical spec following, and should be used as an example.

## Goals for design review

For user-facing designs, we want to give them a lot of exposure to the world before we implement something, as we rarely take away features and want to maintain backwards compatibility for as long as possible (possibly forever). Once it's in, it's in, and this puts a high bar on new additions and we want to make sure everyone has a chance to look them over.

Expand All @@ -28,21 +45,20 @@ Design review ensures that the high level technical choices are simple, communic
Designs and issues vary in a multitude of details concerning how they are staged. A successful life cycle for a major issue might include:

* a brainstorming session on Discourse where the underlying problem is raised and broad, preliminary ideas are proposed and discussed,
* one or more developers turning these ideas into a design proposal, typically on a shared Wiki page,
* one proposal by the relevant managing stakeholder,
* one or more members of the community turning these ideas into a design proposal, typically on a shared Wiki page,
* the accepted design and accompanying discussion being summarized and specified at greater detail a GitHub issue, which may host its own lower-level discussion of technical issues,
* the design being implemented on a branch from the development branch of the relevant repositories with sufficient unit test coverage and documentation to release the feature,
* a pull request being made for the issue and goes through normal code review on GitHub, returning to the previous step until it is approved through code review
* the issue being merged into the development branches of the relevant repositories, which will include the feature in the next release.

Once a design is accepted, it should not be substantially changed. Minor changes may be submitted as amendments. This is one reason why it may not be productive to go into fine-grained implementation details in the technical specification. An approved design does not assign anyone in particular to work on the design, but one should create an issue linking or incorporating the design document.
An approved design does not assign anyone in particular to work on the design, but one should create an issue linking or incorporating the design document.

## What requires a design review?

What exactly constitutes a truly major change is evolving based on community norms and varies depending on what part of the ecosystem you are proposing to change, but may include:
semantic or syntactic changes to the language that is not a bugfix,
adding or removing language features,
changing the interfaces among modules impacting multiple managing stakeholders, such as math, algorithms, services, and interfaces.
changing the interfaces impacting multiple modules, such as math, algorithms, services, and interfaces.

In general, thinking “oh, this might include a large refactor” or “this is really exciting” or “this might not work everywhere” are good indicators you should seek design approval.

Expand All @@ -55,19 +71,13 @@ bug fixes that touch only a few files.

It is generally a good idea to pursue feedback from other project developers beforehand. The easiest way to do this is in a Discourse post that brings up the problem or issue and perhaps a rough outline of proposed solution at a high level and solicits further ideas and brainstorming. As a rule of thumb, receiving encouraging feedback from longstanding project developers on such a thread is a good indication that the design or issue is worth pursuing.

## The process

Write a design document roughly following [the template](0000-template.md). Designs that do not present convincing motivation, demonstrate understanding of the impact of the design, or are disingenuous about the drawbacks or alternatives tend to be poorly received.
Share it with the Stan core developers, advertising it on discourse and ideally the weekly meeting. Especially share it with one or more developers with design approval privileges.
Designs rarely go through this process unchanged, especially as alternatives and drawbacks are illuminated. Edit the existing Google doc and allow it to preserve the history.
Submit the design to a reviewer and work with them until it is accepted or postponed.

## Conducting design reviews

Anyone involved may schedule meetings to discuss the design with interested parties over a video chat. It’s generally a good idea to get an approving stakeholder involved for at least one of these. A design may be postponed if we don’t want to evaluate the proposal immediately for reasons such as not being urgent or depending on other designs to be completed. Postponing a design indicates that at least someone thinks we might reasonably consider making the change in the future---otherwise it would just be closed. To close a design request for comments, a reviewer should describe the reasoning in the relevant discourse post and lock the thread.
Anyone involved may schedule meetings to discuss the design with interested parties over a video chat. A design may be postponed if we don’t want to evaluate the proposal immediately for reasons such as not being urgent or depending on other designs to be completed. Postponing a design indicates that at least someone thinks we might reasonably consider making the change in the future---otherwise it would just be closed. To close a design request for comments, a reviewer should describe the reasoning in the relevant discourse post.

If the motivation and the design are reasonably sound, a reviewer should interact with the proposer and community to give constructive feedback if the reviewer thinks the design could be accepted in the near term.

## Implementing an accepted design

No one in particular is obligated to implement an accepted design. There will be an associated issue in the relevant repository that keeps track of anyone working on it. If you see an accepted design and want to work on it, post in that issue and tag the relevant parties.