Program clarifications

[evouga]

Clarify program requirements; addresses #330 and #341

[evouga]

I'm not sure what "Contest systems may provide additional libraries or tools" really means? I would argue that dropping that line does not change the meaning at all (we are definitely not saying that everything not mentioned in the spec is forbidden), and keeping it does not make anythign more clear (at least not to me).

Yes, I agree the line does not actually specify anything. However (see also the discussion at #330) in practice people will want to link against additional libraries in their build and run scripts; my intent here is to (1) explicitly acknowledge that this is allowed and (2) encourage the best practice of documenting the needed libraries in a standard place.

[evouga]

What is meant by "compilation byproducts"?

I've removed this clause. (But is there any case where there are some dynamic libraries, or other files produced during compilation other than the submitted files and the compiled binary, that need to be in the folder?)

[niemela]

Yes, I agree the line does not actually specify anything. However (see also the discussion at #330) in practice people will want to link against additional libraries in their build and run scripts; my intent here is to (1) explicitly acknowledge that this is allowed and (2) encourage the best practice of documenting the needed libraries in a standard place.

I would argue that that discussion actually implies the opposite (and I would definitely argue for the opposite). I.e. if you want an additional library it should be included in the package (which kinda implies that "Contest systems may not provide additional libraries or tools", although it's strange to say that you can't provide additional things, when you have not exactly specified what should be provided).

[evouga]

Yes, I agree the line does not actually specify anything. However (see also the discussion at #330) in practice people will want to link against additional libraries in their build and run scripts; my intent here is to (1) explicitly acknowledge that this is allowed and (2) encourage the best practice of documenting the needed libraries in a standard place.

I would argue that that discussion actually implies the opposite (and I would definitely argue for the opposite). I.e. if you want an additional library it should be included in the package (which kinda implies that "Contest systems may not provide additional libraries or tools", although it's strange to say that you can't provide additional things, when you have not exactly specified what should be provided).

There is some elegance to that approach, but as a practical matter do you really want individual forks of Eigen, matplotlib, libpng, etc. (and all dependencies they recursive pull in) across some fraction of your problem packages?

[niemela]

There is some elegance to that approach, but as a practical matter do you really want individual forks of Eigen, matplotlib, libpng, etc. (and all dependencies they recursive pull in) across some fraction of your problem packages?

Short answer: yes.

Long answer:

I see three possible approaches:

1. systems are allowed to provide anything, and similarly, problem packages are allowed to depend on anything. Pro: this is very simple for contest runners. A problem setter need only to verify with the tech staff for the contest they are running that the packages they need are available. Con: packages (that use this) become very unportable.
2. problem packages must include whatever they depend on (but as long as they do, they can depend on anything). Pro: this is still fairly simple (but less so), and it is reasonably portable. Con: packages become much larger. Popular dependencies in large archive could be stored many times.
3. problem packages may depend on some things they don't include, but must declare this (somehow). Pro: as simple as 1 for setters (and contest runners), and (almost) as portable as 2. Con: we have to set up (and keep up to date if there isn't something we could piggyback on) some kind of registry for libraries (and versions of libraries). Given versions it's not even certain that it's that much smaller than 2 for large archives.

I think 3 is best, but needs to be done well, and that's hard. 1 is worst; it's fine for single contests, but terrible for reuse and archives (i.e. "distributing and sharing problems" which is the stated purpose of the format). 2 is pretty much as good as 1 for single contests, and based on what has been used so far wouldn't be too bad for large archives either. Also, large archvie could use caching to make it better if it turns out to be bad (although, not quite as easily as with 3).

So, 2 is a lot easier than 3, and almost as good.

[evouga]

But Option 2 also places more burden on the package writers. "this is still fairly simple": only for header-only libraries like Eigen, though? libpng is probably already more annoying than package authors are realistically willing to put up with; you need its source, the source of zlib, and some meta-scripts that configure, build, and install both of them reliably.

And from a quick look at matplotlib it seems a real nightmare to build from source without pip, requiring things like FreeType in addition to a dozen Python packages.

We rejected including even just a language version string for the package programs, under the argument that (1) it's too onerous on the package authors and (2) "surely" it won't be too difficult for an archivist in 20 years to figure out how to build the problem package. This dependency-bundling seems a lot more effort (albeit, only for problem packages that need libraries for visualizers etc. which are a minority).

I haven't needed external libraries for problems I've written (except maybe in generators, which are outside the scope of the spec) so I don't have much of a dog in this race---I'm fine with requiring all dependencies to be bundled, if this sounds reasonable to @eldering and others in the linked issue. Though I'm skeptical authors will obey such requirements in practice.