Expose the new primops isByteArrayWeaklyPinned# and isMutableByteArrayWeaklyPinned# from GHC.Exts

[AndreasPK]

We will add these new primops to GHC in 9.12 (https://gitlab.haskell.org/ghc/ghc/-/merge_requests/13144) and likely backport them to 9.10 as well.

I expect them to be stable and used widely going forward and would like to avoid people depending on ghc-internal for this feature. Therefore I propose we expose it from GHC.Exts in line with most other primops.

For background on the issue see https://gitlab.haskell.org/ghc/ghc/-/issues/22255

For details about what these primops do see the changes to docs/users_guide/exts/ffi.rst in https://gitlab.haskell.org/ghc/ghc/-/merge_requests/13144

[tomjaguarpaw]

I'm afraid I've forgotten the latest status, according to the CLC, of the GHC. hierarchy in base, and particularly the status of GHC.Exts. Did we make a decision on this? According to @bgamari's Base stability spreadsheet the "Desired visibility" of GHC.Exts is "see document", but I don't know what document that is.

And for reference, here is a big previous discussion: #146

[AndreasPK]

I assume it's the document in the foot note which merely raises the question:

## 4. The question of `GHC.Exts`

Historically `GHC.Exts` has been the primary entry-point for users wanting access to all of the primitives that GHC exposes (e.g. primitive types, operations, and other magic). This widely-used module poses a conundrum since, while many of these details are quite stable (e.g. `Int#`), a few others truly are exposing implementation details which cannot be safely used in a GHC-version-agnostic way (e.g. `mkApUpd0#`, `unpackClosure#`, `threadStatus#`). There are at least two ways by which this might be addressed:

 * Export only the subset of primops that we can stabilize (e.g. things like `Int#`, `Weak#`, `newArray#`, etc.) in `GHC.Exts`, leaving the rest to only be exposed via `GHC.Prim` (which should not be used by end-users), or
 * Declare the entirety of `GHC.Exts` to be unstable and export the stable subset from another namespace (e.g. `Word#` and its operations could be exposed by `GHC.Unboxed.Word`)

What eventually was voted on was this comment: #146 (comment), according to which the following applies to GHC.Exts:

The API of this module is unstable and not meant to be consumed by general public. If you absolutely must depend on it, make sure to use a tight upper bound, e. g., base < 4.X, not just base < 5, because the interface can change rapidly without much warning.

All that being said none of that really makes a clear proposal about how these or other future primops should be handled in regards to GHC.Exts

[phadej]

@AndreasPK IMO, there is ghc-experimental, so you can leave out GHC.Exts, and expose new primops in ghc-experimental. (I personally am fine depending on ghc-prim if I need primops; so IMHO they don't need to be re-exported anywhere).

[AndreasPK]

@AndreasPK IMO, there is ghc-experimental, so you can leave out GHC.Exts, and expose new primops in ghc-experimental. (I personally am fine depending on ghc-prim if I need primops; so IMHO they don't need to be re-exported anywhere).

Personally I would prefer a wider design/agreement on how to expose primops via ghc-internal/ghc-experiment/base using a sensible module structure over use of GHC.Exts. But without time to design and implement such a proposal exposing them via GHC.Exts seems like the next best option to me.

---

That being said I'm not adverse to adding it to ghc-experimental instead. But as I understand it the purpose of ghc-experimental is:

ghc-experimental, initially empty, depends on base. Functions and data types here are intended to have their ultimate home in base, but while they are settling down they are subject to much weaker stability guarantees. Example: new type families and type constructors for tuples, ghc-proposals/ghc-proposals#475.

So if the CLC wants to stop exposing any primops from base (maybe even deprecating existing ones in the process) that wouldn't be unreasonable but then it seems unclear if these primops should still be exposed via ghc-experimental or simply remain relegated to ghc-internal.

But then we also don't want to encourage dependencies on ghc-internal. Making for no obvious solution from my perspective.

In light of all these complexities and unanswered questions to me the simplest and most consistent solution would be to expose those via GHC.Exts in line with existing primops and hope for someone to draw up a design for primops in the future.

Ultimately the decision lies with the CLC. If the proposal is rejected however I would still welcome concrete suggestions on how to best expose these primops in particular or primops in general.

[mixphix]

I agree that the most sensible solution is to export it from GHC.Exts as the module hierarchy currently stands, if they are to be so stable and widely-used.

Would that those interested in adding primops be the ones drawing up a design for primops...

[Bodigrim]

I'm afraid I've forgotten the latest status, according to the CLC, of the GHC. hierarchy in base, and particularly the status of GHC.Exts. Did we make a decision on this?

No, there was no particular CLC decision on GHC.Exts. I recall that @adamgundry was recently arguing against re-exporting all new primops from GHC.Exts by default?..

In this particular case the primops are of primary interest for packages like bytestring, so I'd strongly support re-exporting them from base instead of ghc-internal / ghc-experimental. Whether it's GHC.Exts or something more specific like a hypothetical GHC.Pinnedness could be up for discussion, if anyone feels strongly enough about it. Otherwise, if there is no huge interest to bikeshed, I'd suggest defaulting to GHC.Exts.

[adamgundry]

I recall that @adamgundry was recently arguing against re-exporting all new primops from GHC.Exts by default?..

In the long term we should move away from the monolithic GHC.Exts, because it is an unstable and unnecessary point of coupling between base and GHC internals, as well as being too large to be easily understood by users. And in the meantime, yes, we certainly shouldn't expose all new primops by default.

That said, there's clearly work to be done to come up with a new design that answers the questions raised in this thread, and that won't happen overnight. Meanwhile, if specific new primops are sufficiently widely useful and stable enough to be desirable in base, following the status quo by adding them to GHC.Exts seems reasonable.

[Bodigrim]

If anyone is in favor of re-exporting these primops from base but has strong objections against doing it via GHC.Exts, please voice your concerns and suggestions until early next week. If there are none, I'll ask @AndreasPK to prepare an MR which we can vote on.

[phadej]

I think new primops pop up regularly enough (this, #203, #188) that it would be great to figure out for the new primop export design sooner before the next primop is being added.

I opened a GHC issue, https://gitlab.haskell.org/ghc/ghc/-/issues/25242 as I think it's more of GHC than CLC issue.