Flag algebra implementation

[bodnalev]

Can handle any theory, any signature. A few interesting theories are included:
-Graphs
-Digraphs
-Hypergrpahs
-Graphs for Ramsey theory
-Tournaments
-Graphs with ordered edges
-Graphs with ordered vertices

It is designed to fit in the sage structure, FlagAlgebras can be constructed over any base ring containing QQ, and over any such combinatorial theory (graphs, hypergraphs etc). FlagAlgebraElements can be added and multiplied together interactively, with good performance and the powerful coercion model. Flag algebraic optimization problems can be solved with a small csdp wrapper csdpy (only working with linux at the moment).

Includes docstring with examples, all running successfully.

There is a small change to coerce.pyx to ask if the elements prefer a different form before applying an operation (so Flag is treated as a FlagAlgebraElement).

📝 Checklist

• The title is concise, informative, and self-explanatory.
• The description explains in detail what this PR is about.
• I have created tests covering the changes.
• I have updated the documentation accordingly.

⌛ Dependencies

[bodnalev]

The docstring tests work well, but I couldn't build sage -b && sage --docbuild reference html, it gave an error saying (this is all the error message, nothing follows):

Error building the documentation.
sage_docbuild.utils.RemoteException:
"""
Traceback (most recent call last):
File ".../sage/pkgs/sage-docbuild/sage_docbuild/utils.py", line 215, in run_worker
result = target(task)
File ".../sage/pkgs/sage-docbuild/sage_docbuild/builders.py", line 97, in build_ref_doc
getattr(ReferenceSubBuilder(doc, lang), format)(*args, **kwds)
File ".../sage/pkgs/sage-docbuild/sage_docbuild/builders.py", line 828, in _wrapper
getattr(DocBuilder, build_type)(self, *args, **kwds)
File ".../sage/pkgs/sage-docbuild/sage_docbuild/builders.py", line 162, in f
runsphinx()
File ".../sage/pkgs/sage-docbuild/sage_docbuild/sphinxbuild.py", line 327, in runsphinx
sys.stderr.raise_errors()
File ".../sage/pkgs/sage-docbuild/sage_docbuild/sphinxbuild.py", line 263, in raise_errors
raise OSError(self._error)
OSError: WARNING: failed to reach any of the inventories with the following issues:
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File ".../sage/pkgs/sage-docbuild/sage_docbuild/main.py", line 508, in
sys.exit(main())
File ".../sage/pkgs/sage-docbuild/sage_docbuild/main.py", line 504, in main
builder()
File ".../sage/pkgs/sage-docbuild/sage_docbuild/builders.py", line 586, in _wrapper
self._build_everything_except_bibliography(format, *args, **kwds)
File ".../sage/pkgs/sage-docbuild/sage_docbuild/builders.py", line 570, in _build_everything_except_bibliography
build_many(build_ref_doc, non_references)
File ".../sage/pkgs/sage-docbuild/sage_docbuild/builders.py", line 328, in build_many
_build_many(target, args, processes=processes)
File ".../sage/pkgs/sage-docbuild/sage_docbuild/utils.py", line 369, in build_many
raise worker_exc.original_exception
OSError: WARNING: failed to reach any of the inventories with the following issues:

[tscrim]

This seems very interesting, but there are a number of things that need to be fixed first.

• You need to explain a bit more about what these objects are (or at the very least include references). For me, a flag is a very different object (a sequence of subspaces).
• I am extremely hesitant to touch anything in coerce.pyx without a very good reason to do so. You will need to explain why you are doing this change, and it might need to get split off to a separate PR.
• You need to add tests for every method and function, not just the public ones.
• I would move all of the elements into the pyx file for speed.
• The different theories should be separated out into their own catalog file.
• Docstrings should be EXAMPLES: with a single colon when it is not followed by code.
• is None instead of == None.

For getting the doc to build, I've found I needed to do make docbuild for at least the first pass. Afterwards, updates can be done as you said.

I will provide more specific comments once the bigger things are more settled.

[bodnalev]

Thank you for taking the time and looking at this PR! To reply to the points:

• I've added an extra link to Razborov's Flag Algebras paper, from the Flag
  file. I can attempt a greater explanation of what is happening there, but the
  paper's terminology is followed. The novelty is CombinatorialTheory, which
  comes with a detailed docstring.

• The reason I changed coerce is the following:
  CombinatorialTheory is a parent, the elements are Flag-s.
  FlagAlgebra is a parent, the elements are FlagAlgebraElement-s.
  But a CombinatorialTheory contains all Flag-s, with every ftype, while a
  FlagAlgebra is only defined over a fixed ftype.

  But Flag-s, with a fixed ftype, form a basis for the FlagAlgebra with the
  same ftype. Therefore it makes sense to add and multiply the Flag-s together
  without the annoying step between, to change them to FlagAlgebraElement-s and
  then do the operation.

  As I was looking at the coercion model, I couldn't find a way to define coercion
  that depends on the elements. It wouldn't really make sense anyway. But when
  evaluating an expression like Flag + Integer, just from the parents,
  (CombinatorialTheory, Integers) it is impossible to figure out which FlagAlgebra
  they land in, since it depends on the Flag's ftype.

  So the extra code I added to the coercion is something really simple.
  It asks the operands (like the Flag in this example)
  if there is a form that actually behaves like an operand. And if they have
  a better form (Flag turns to FlagAlgebraElement) then use that better form
  in the operation.

  The code comes after all the other steps, so after the failure of all the
  other coercion attempts. If the as_operand is not implemented, then it acts
  just like the previous code, and there is only an overhead when an error occurs
  (which I assumed is fine since something went wrong anyway).

  I can add this to a separate PR. But the change only makes sense here, when I
  want to use this. I can add this explanation somewhere, if that helps.

• All private functions should have tests now

• I've experimented with that, but most of the heavy lifting is happening in Flag,
  I didn't see a reason to switch.

• Thank you for catching those, I think I'm using the catalogs properly now,
  let me know if they are still bad. Also replaced == None with is None

• I still can't build the reference manual. make docbuild gives this error:

The following package(s) may have failed to build (not necessarily
during this run of 'make docbuild'):

* package:         sagemath_doc_html-none
  last build time: Dec 20 11:44
  log file:        /home/bodnalev/sage/logs/pkgs/sagemath_doc_html-none.log
  build directory: /home/bodnalev/sage/local/var/tmp/sage/build/sagemath_doc_html-none

• Made small fixes in the documentation

[tscrim]

Sorry for taking so long to respond.

* I've added an extra link to Razborov's Flag Algebras paper, from the Flag
  file. I can attempt a greater explanation of what is happening there, but the
  paper's terminology is followed. The novelty is CombinatorialTheory, which
  comes with a detailed docstring.

The class CombinatorialTheory has no class-level docstring. I did not see a definition anywhere. Since Sage has a very broad user base, there can sometimes be name conflicts. I think a flag in reference to vector spaces is more widely used and general terminology, so it would be good to consider alternative names. (Personally, the name flag seems strange from what I can tell; granted, the vector space name is a mathematician's dad joke...) At the very least, it should be made clear in Flag, FlagAlgebra, etc. that there is no relation; for example, just including a precise definition would be sufficient.

* The reason I changed coerce is the following:
  CombinatorialTheory is a parent, the elements are Flag-s.
  FlagAlgebra is a parent, the elements are FlagAlgebraElement-s.
  But a CombinatorialTheory contains all Flag-s, with every ftype, while a
  FlagAlgebra is only defined over a fixed ftype.
  But Flag-s, with a fixed ftype, form a basis for the FlagAlgebra with the
  same ftype. Therefore it makes sense to add and multiply the Flag-s together
  without the annoying step between, to change them to FlagAlgebraElement-s and
  then do the operation.

You might want to reconsider which object should be the parent. There are also facade parents, which are things that act like a parent and "have" elements, but the resulting objects are associated with something else. For an example where this frequently occurs, see the posets code. However, this is all happening behind the scenes and shouldn't be a bottleneck; if it is, then the user should be more careful about where things live. Although other options for this will follow.

If I was doing this myself, I would just let the coercion framework do its own thing. (As an example, look at the type of 4 / 2.)

  As I was looking at the coercion model, I couldn't find a way to define coercion
  that depends on the elements. It wouldn't really make sense anyway.

Indeed, which is a strong indication that something should be done differently. As you have surmised, the coercion model is only about making sure you find a parent where the operation makes sense.

  But when evaluating an expression like `Flag + Integer`, just from the parents,
  (CombinatorialTheory, Integers) it is impossible to figure out which FlagAlgebra
  they land in, since it depends on the Flag's ftype.
  So the extra code I added to the coercion is something really simple.

Just for clarity (sorry for some repetition), you should not be adding this. Coercion is very important core code that needs to run very fast and for very general objects.

  It asks the operands (like the Flag in this example)
  if there is a form that actually behaves like an operand. And if they have
  a better form (Flag turns to FlagAlgebraElement) then use that better form
  in the operation.
  The code comes after all the other steps, so after the failure of all the
  other coercion attempts. If the `as_operand` is not implemented, then it acts
  just like the previous code, and there is only an overhead when an error occurs
  (which I assumed is fine since something went wrong anyway).
  I can add this to a separate PR. But the change only makes sense here, when I
  want to use this. I can add this explanation somewhere, if that helps.

Adding this would be a change to the coercion model, so I strongly think it should not be included. Again, sorry for some repetition in this.

It seems like you either want to do what I said above or just let the coercion framework put things in a common parent where things make sense. In particular the CombinatorialTheory and the FlagAlgebra can use the same element class. This is done in many places in Sage (e.g., see the combinatorics portion).

* All private functions should have tests now

Not every method and function has a doctest from a quick look. This also includes cpdef methods (strictly speaking, this isn't needed for cdef methods/functions as they cannot be called from Python, but it is still good to have something there showing they are running properly if something does break).

* I've experimented with that, but most of the heavy lifting is happening in Flag,
  I didn't see a reason to switch.

It gets you a little more speed and some better code localization.

Also, from what you've said, why doesn't FlagAlgebraElement inherit from Flag?

* Thank you for catching those, I think I'm using the catalogs properly now,
  let me know if they are still bad. Also replaced `== None` with `is None`

I will check more carefully when I go through things line-by-line. Right now it is more about the bigger picture review.

* I still can't build the reference manual. `make docbuild` gives this error:

There's a number of docstring issues from a quick look.

• Do not indent for line continuations. (Perhaps this is allowed, but it is different than the rest of Sage nevertheless.)
• Anytime you have a ::, it should be followed by a blankline and then an indentation.
• Latex formatting for things not mean to be latex might cause issues (see below).
• Bulletpoints should be preceded by a blankline from the part before.

---

Some other comments:

• Code formatting is double backticks {{{x + y}}} whereas single is $\LaTeX$ formatting \omega + \delta (which will render as $\omega + \delta$ in the doc). We also generally put {{{self}}} in code formatting.
• Use _richcmp_ for the comparisons in element. There are also some constants, e.g., Py_EQ that you can import to make the code easier to read than the integers. It also will have things go through coercion, so you can assume the same parent.
• Use __reduce__ for pickling Cython classes.
• The initial "one-line" docstring description should use the imperative form and end with a period. For example: Return the attribute foo.
• Note :meth: for methods instead of :func:. Also, links only work if it is in the global namespace IIRC; you might need the full path sage.algebras.....
• Instead of having a detailed __init__; it is often better to put this as class-level docstring. This way, you have space to add more developer-focused information that won't scare/confuse a more casual user.

[grhkm21]

Hi, if this PR needs help, either with linting (I guess I'm more familiar than bodnalev), or with reviewing, please ping me! It seems like a big PR but I think it will be useful for combinatorists, so it will be nice to have.

[bodnalev]

Thank you for the careful review of the pull request, and sorry for taking so long to get back. This was initially a quick rewrite of my custom flag algebra code to fit in the Sage framework. It turned out to be a larger project than I expected and I don't have loads of time to work on it next to my other commitments currently.

@grhkm21 I appreciate any help regarding the documentation, especially with the formatting. I was mostly copy-pasting other documentation strings, seems like relatively unsuccessful. I've tried to move things around, and include more detail, but I imagine most of the formatting is still incorrect. Fixing the formatting errors is rather hard blindly, sorry I couldn't make much progress on it. A month ago I did a merge with the up-to-date branch. I suspected an issue with that, so I force-pushed a version where that merge is removed. Now, every change is after the 10.3.beta0 but it seems like that did not resolve the problem. Still no success with building the doc, and it seems the error isn't even related to the flag algebra code I have, as the original branch also fails to build.

CombinatorialTheory is explained at the file level docstring. I've added an extra reference to it from the init.
I have included the definition of a flag in the docstring. Flag is the name of these objects in the literature, I don't think it should be changed. The paper introducing them was called "Flag Algebras" and is now a standard terminology in combinatorics. It is unlucky that the word "flag" is used in multiple uncorrelated settings, although it is not the first time this has happened in mathematics.

I understand the alternatives you've mentioned, but none really reflects the behaviour that I believe is intuitive. If you have a better alternative that satisfies the following conditions, then I can try to implement that:
-Flag and FlagAlgebraElement are distinct: They live in different spaces, and it doesn't make sense to create a hierarchical structure between them. Flag is a model of a theory with some of the members marked. Like a graph or a permutation. A FlagAlgebraElement is an element in an algebra.
-For every Flag, there is a unique FlagAlgebraElement that is equal to it. The Flag-s are mapped to FlagAlgebraElement-s. They span the algebra as a vector space (but don't form a base as there is a dependence between them).
-For convenience, when a Flag is used in an operation, it should automatically change to the equivalent FlagAlgebraElement where the operation is defined. And I find it unelegant to always write some_flag.as_flag_algebra_element() when you want to make any operation with it. It sounds like this is the exact reason why the coercion model was created, to reduce these straightforward coercions.
-The parent for a flag (a CombinatorialTheory) is not enough to determine the FlagAlgebra parent where the flags land, the FlagAlgebra depends on the specific flag.

Just for clarity (sorry for some repetition), you should not be adding this. Coercion is very important core code that needs to run very fast and for very general objects.

I find this a little ridiculous, the code I included keeps the speed and generality. Unless there is an exception, it has the exact same speed. And the code just increases the generality. Now the elements can tell if there is a form that can behave as an operand. Stuff that lives in a space where, for example, multiplication is not defined, can coerce into something that can be multiplied when you try to multiply with it.

For reference, see this accepted commit:
ed72e73?diff=split&w=1
The commit changed the behaviour of a core internal function in structure/category_object.pyx to get a desired behaviour in a different part of the source code. I would like to point out that, while the change increases the generality, the "if" test adds an overhead to all execution of the function, so technically always reduces performance. My change, in comparison, comes with no overhead on all previously running code.

[tscrim]

Flag is the name of these objects in the literature, I don't think it should be changed. The paper introducing them was called "Flag Algebras" and is now a standard terminology in combinatorics. It is unlucky that the word "flag" is used in multiple uncorrelated settings, although it is not the first time this has happened in mathematics.

This is a very weak argument. I am a combinatorialist, and the only place I have ever heard of a flag before this is as a sequence of vector spaces. On the contrary, because it is used in different settings means you should change the name to be able to differentiate it.

-For convenience, when a Flag is used in an operation, it should automatically change to the equivalent FlagAlgebraElement where the operation is defined. And I find it unelegant to always write some_flag.as_flag_algebra_element() when you want to make any operation with it. It sounds like this is the exact reason why the coercion model was created, to reduce these straightforward coercions.

That's not quite correct. It is to convert everything to a common parent where the operation makes sense, and there are rules with what is allowed. If one parent does not coerce into the other, then they try to do a pushout construction. Although by your design, flags don't have operations that can be performed on them.

Just for clarity (sorry for some repetition), you should not be adding this. Coercion is very important core code that needs to run very fast and for very general objects.

I find this a little ridiculous, the code I included keeps the speed and generality. Unless there is an exception, it has the exact same speed. And the code just increases the generality.

Yes, and there can be code that wants to handle those exceptions. I also do not see any real increase in generality either. You've added a check for an extra method that is used in precisely one place as a convenience (with no clear other uses at present).

Now the elements can tell if there is a form that can behave as an operand. Stuff that lives in a space where, for example, multiplication is not defined, can coerce into something that can be multiplied when you try to multiply with it.

I really don't understand this. The fact that you feel you need this is a code smell to me and your implementation or design needs to be changed. You wrote:

-Flag and FlagAlgebraElement are distinct: They live in different spaces, and it doesn't make sense to create a hierarchical structure between them. Flag is a model of a theory with some of the members marked. Like a graph or a permutation. A FlagAlgebraElement is an element in an algebra.

and

For every Flag, there is a unique FlagAlgebraElement that is equal to it.

From this, it seems to me that the distinction you are making between Flag and FlagAlgebraElement is artificial. It sounds like you're saying something analogous to the integers used for counting are distinct from the ring of integers. By not distinguishing between the two, you also alleviate your issues with defining operations. I don't understand why these need to be separated.

For reference, see this accepted commit: ed72e73?diff=split&w=1 The commit changed the behaviour of a core internal function in structure/category_object.pyx to get a desired behaviour in a different part of the source code. I would like to point out that, while the change increases the generality, the "if" test adds an overhead to all execution of the function, so technically always reduces performance.

To begin, that's an apples to oranges comparison. Creating parents is called less frequently than coercion, but that is not used in initialization. Plus, _first_ngens() will almost never be called in tight loops as it is only used for the R.<x,y,z> = QQ[] syntax. Moreover, there is a clear need to handle that case.

My change, in comparison, comes with no overhead on all previously running code.

Again, that is not true, by your own admission. I don't know how much code catches the TypeError coercion exception and then moves on to try something else. Nor do I think it is good policy to allow any arbitrary extensions to such core functionality without any discussion.

If you really want to change how coercion behaves, you should create a separate PR that clearly explains the rationale. However, I think there are much better ways to get the functionality you want (in order of correctness IMO):

1. Have Flag be an element of a FlagAlgebra, which can reference the corresponding CombinatorialTheory.
2. Use a pushout construction.
3. Override the appropriate operations.

[bodnalev]

The third David P. Robbins Prize was awarded to Alexander Razborov "for his paper, "On the minimal density of triangles in graphs" (Combinatorics, Probability and Computing 17 (2008), no. 4, 603-618), and for introducing a new powerful method, flag algebras, to solve problems in extremal combinatorics."
https://www.ams.org/prizes-awards/pabrowse.cgi?parent_id=16
From my perspective, the name is justified. In case a different meaning of "flag" is implemented, I am happy to figure out ways to resolve this collision. For now, I think it is a good name.

It sounds like you're saying something analogous to the integers used for counting are distinct from the ring of integers.

I think your analogy is not quite accurate, and therefore fails to illustrate the reason I changed coerce.pyx. A slightly better analogy would compare some set $S$ with a vector space whose base is indexed by the elements of $S$, like $\mathbb{R}^S$. The object $s \in S$ lives in a completely different space than the base element corresponding with $s$ (0 everywhere, but 1 at index $s$). There is a natural map from $S$ to $\mathbb{R}^S$, there is one unique element in $\mathbb{R}^S$ that corresponds with $s$, but they are different. You wouldn't implement $S$ as something that inherits from $\mathbb{R}^S$, that would be an incredibly inefficient way to store anything. In fact, you wouldn't be able to define $\mathbb{R}^S$ without first defining the index set $S$.

Now imagine you have no + operation on $S$, but there is a vector addition defined in the naturally constructed $\mathbb{R}^S$. Then $s + z$ (where $z \in S$) should automatically coerce to a common parent, to $\mathbb{R}^S$. But from the coercion model's perspective $s, z$ are already in the same parent, so they are good and it will not try to do a coercion. Overwriting the __add__ operation in $S$ is also not enough. For example, in the scenario of flag algebras, $\mathbb{R}$ lives in $\mathbb{R}^S$, where $x \in \mathbb{R}$ is mapped to the all $x^S$ vector. So writing $1 + s$ makes sense and has one clear meaning, to coerce both terms into $\mathbb{R}^S$. But then the coercion model will fail to call __add__ on $s$, and it throws an error.

I know it depends on you trusting me, but I did try out alternatives, including pushout construction, overriding _add_ and __add__ in multiple places etc. I haven't tried making Flag an element of FlagAlgebra because that makes no sense in this setting. This version included the smallest modification to coerce.pyx that gave the behaviour I expected. And even though this code is the only one that uses that functionality, I believe this can be beneficial later. It is not a big stretch to assume a situation similar to $\mathbb{R}^S$ appearing later, where it would be good to treat elements of $S$ in operations as an element of a different space. Just a random example coming to my mind is the calculation of Khovanov homology, where (labelled) smoothings of a knot, are indexing a vector space.

I don't know how much code catches the TypeError coercion exception and then moves on to try something else.

It makes me wonder, why would such a code get accepted. Wouldn't it be easier to use knowledge about the specific situation and speed up the calculation even more, by skipping the checks in coercion (checking python types, checking numpy types, checking if the objects have a sage method ...)? And if we deal with hypothetical code scenarios that can potentially break due to a change in a core functionality, what if someone's code relied on _first_ngens not checking if the generators is a list, and instead wanted to catch the exception? What if someone wanted to use the R.<x,y,z> = QQ[] syntax millions of times?

[tscrim]

From my perspective, the name is justified.

I didn't say it wasn't. Note that the flag manifold/variety (as a set of flags) has been around for longer in multiple fields.

In case a different meaning of "flag" is implemented, I am happy to figure out ways to resolve this collision. For now, I think it is a good name.

This name collision must be solved before this gets merged into Sage. Perhaps CombintaorialTheoryFlag?

It sounds like you're saying something analogous to the integers used for counting are distinct from the ring of integers.

I think your analogy is not quite accurate, and therefore fails to illustrate the reason I changed coerce.pyx. A slightly better analogy would compare some set S with a vector space whose base is indexed by the elements of S, like RS. The object s∈S lives in a completely different space than the base element corresponding with s (0 everywhere, but 1 at index s). There is a natural map from S to RS, there is one unique element in RS that corresponds with s, but they are different. You wouldn't implement S as something that inherits from RS, that would be an incredibly inefficient way to store anything. In fact, you wouldn't be able to define RS without first defining the index set S.

There is a natural map, but it is not canonical. Moreover, you have made one choice that is not natural nor canonical, choosing the field $\mathbb{R}$. On the contrary, I would keep the implementations completely separate and require a user to explicitly construct the vector space they want (in particular, specifying the base ring) and pass objects accordingly. It is very conceivable that a user will want to use different base rings (e.g., ranging over a set of positive characteristics).

Now I think I understand how you are distinguishing between a flag and a flag algebra element. However, what you said about the uniqueness is misleading at best. You have chosen a base ring and a particular embedding to make it unique, but I could do that for any such injective function (say $s \mapsto 2s$ over a ring of characteristic not equal to $2$). How you were saying it implied a canonicalness that is not there.

Now imagine you have no + operation on S, but there is a vector addition defined in the naturally constructed RS. Then s+z (where z∈S) should automatically coerce to a common parent, to RS.

This last statement here is not true as $s, z \in S$, which as you said has no $+$ operation. Furthermore, you cannot guess the ring $R$ and there is no canonical choice (using $\mathbb{Z}$ would be natural since it is the initial object in rings, but coercion should also be canonical).

But from the coercion model's perspective s,z are already in the same parent, so they are good and it will not try to do a coercion.

Yes, but in that parent, there is no defined operation of, e.g., addition. So coercion should not try to do anything more and error out.

Overwriting the __add__ operation in S is also not enough. For example, in the scenario of flag algebras, R lives in RS, where x∈R is mapped to the all xS vector. So writing 1+s makes sense and has one clear meaning, to coerce both terms into RS. But then the coercion model will fail to call __add__ on s, and it throws an error.

You would also need __radd__ in this situation. However, the question is what is special about $\mathbb{R}$? (In fact, in a CAS, real numbers are quite bad because of the finite precision, which needs to be chosen and further exacerbates the problem at hand.)

I know it depends on you trusting me, but I did try out alternatives, including pushout construction, overriding _add_ and __add__ in multiple places etc.

I trust you that you tried. However, without seeing what you tried, I don't know if you made any mistakes in trying. (To this point, you would not want to use _add_ because that assumes the output belongs to the same parent as the inputs, which are assumed to be in the same parent.) It could help both of us to go over this (possible not on this PR but future work).

I haven't tried making Flag an element of FlagAlgebra because that makes no sense in this setting. This version included the smallest modification to coerce.pyx that gave the behaviour I expected.

I would be very surprised by such behavior as I would know that addition is not defined between the flags.

And even though this code is the only one that uses that functionality, I believe this can be beneficial later. It is not a big stretch to assume a situation similar to RS appearing later, where it would be good to treat elements of S in operations as an element of a different space. Just a random example coming to my mind is the calculation of Khovanov homology, where (labelled) smoothings of a knot, are indexing a vector space.

In addition to the ambiguity of the base ring, there are lots of subtleties involved here that you are not considering. What happens if you have two elements that would coerce into two objects that you would then apply the as_operand "coercion". Would you follow with this? What if those objects had, say, addition instead but your initial objects rather had as_operand? Related, which would take precedence? Great care is needed here, even if you can find a clear use case that other methods (e.g., pushout) can be proven not to work.

I don't know how much code catches the TypeError coercion exception and then moves on to try something else.

It makes me wonder, why would such a code get accepted. Wouldn't it be easier to use knowledge about the specific situation and speed up the calculation even more, by skipping the checks in coercion (checking python types, checking numpy types, checking if the objects have a sage method ...)?

In generic code, you might not know whether a particular coercion exist (which can be complicated because of checking sequences of coercions), but you then you might want to try something more extravagant (say, you know what a potential intermediate object is that you could "fill in" the change). Such a thing might be used in the coercion code to construct pushouts; I just don't know without digging back through that. Since this is such core speed-important code, I would much rather be safe.

Independently of the speed considerations, changing the how the model works and its rules needs to be done with great care. There are currently strong rules/restrictions on what is allowed to be a coercion. See, e.g.,

https://doc.sagemath.org/html/en/reference/coercion/index.html

This document (among others) would need to be amended with care with what would be allowed. This really needs to be done on a separate PR with a sage-devel discussion. Before that we can discuss some of the technicalities of the mathematical definition. Assuming we add this, how do you use just the elements (say, two flags) and the information contained within to define the addition?

And if we deal with hypothetical code scenarios that can potentially break due to a change in a core functionality, what if someone's code relied on _first_ngens not checking if the generators is a list, and instead wanted to catch the exception? What if someone wanted to use the R.<x,y,z> = QQ[] syntax millions of times?

I would be telling the author to bring this outside of the loop because there is no point in recreating the same object (well, the poly ring is pulled from the cache). Although if it was somehow necessary, then I would say use the explicit R = PolynomialRing(QQ, 'x,y,z'); x,y,z = R.gens() (which is also true Python code and can avoid the preparser).

---

Let me again say I think this is a great piece of code that I really would like to get merged into Sage. Furthermore, I hope you will continue to contribute code to Sage after this as well. I know I am pushing back (very strongly) on something, but I hope that you don't think I am being hostile or unreasonable.