fix Issue 21574 Evaluate Pure Functions With CTFE

[WalterBright]

This implements dlang/DIPs#177

dlang/phobos#7211 takes advantage of this.

For a call to a pure function, if the function arguments are Literals or are Expressions
that are representable by Literals after the normal semantic processing of them, and the
return type of the function can be represented by a Literal, then the function is
evaluated at compile time using CTFE.

If the call cannot be evaluated using CTFE, such as if the pure function contains impure
debug statements in the path of CTFE execution, no error is generated, the function is simply
evaluated at run time instead.

Without this change, common code patterns use templates and enums to force compile time execution, making the code more complex than necessary.

[dlang-bot]

Thanks for your pull request, @WalterBright!

Bugzilla references

Auto-close	Bugzilla	Severity	Description
✓	21574	enhancement	Evaluate pure functions using CTFE

Testing this PR locally

If you don't have a local development environment setup, you can use Digger to test this PR:

dub run digger -- build "master + dmd#10452"

[nordlow]

@UplinkCoder: I wonder how this will affect compilation time if used together with newCTFE, which I presume has a larger latency than the current CTFE-evalutor...

[don-clugston-sociomantic]

I don't think that this quite what we want. There are two problems:

(1) We definitely do not want all CTFEable functions to use CTFE.
The most obvious example of this is compile-time regexp. And suppose you made a function to calculate PI to billions of decimal places, you want to run it at runtime because it's faster.

Because of this, you'd need to remove the 'pure' attribute from many such functions.

(2) Not all pure functions can actually be run at CTFE. For example a pure function may call an extern(C) function.

Essentially, this PR repurposes pure to mean run at compile time if possible. That's a big change, that needs considerable thought.

[don-clugston-sociomantic]

Additionally this doesn't support the case of forcing CTFE even when a parameter is not a literal.

Eg.I'd like to be able to force hexString to be run at compile time. I'd like to even allow it to be used in cases like this:
´hexString("65" ~ foo(7) ~ foo(8));Ie, the calls tofoo` should be CTFEd as well.

[nordlow]

Are we in need of a more direct syntax for calling functions at compile time?

Opposite to

enum x = hexString(...)

and template parameterized overloads.

[dnadlinger]

Without this change, common code patterns use templates and enums to force compile time execution, making the code more complex than necessary.

How much more complex is something like eval!ctfeableFunction(foo, bar) really?

I think Don's point about the meaning of pure is a good one, as is the example of compiling regular expressions.

Sure, we could make evaluation of pure functions the default, and add a special wrapper ensureRuntimeEvaluation wrapper that uses if (__ctfe) to stop evaluation. But both cases should be fairly rare, and the current behaviour is straightforward to teach ("compile-time evaluation is only attempted when required by the context"), which doesn't seem to be the case for the proposed change.

[don-clugston-sociomantic]

My proposal to Walter was this (in the context of HexString):

Templates are the wrong tool for this job. They create an instance of the template for every string you use, And the mangled name is stored in the binary, along the contents of the template.

Converting a UTF8 string to a string of ubytes is a perfect task for CTFE. And in the case where you're creating a named sequence of bytes, this works perfectly, right now:

// a global static, not an enum, so that it is stored in only place in the binary
immutable ubyte [] JPEGheader = hexBytes("FF D8 FF") ;

The problem with this solution, though, is that it only works when the function call gets CTFE'd. So it doesn't work (for example) when used to initialize a stack variable.

So, hexString is implemented as a template for only one reason: to force it to be evaluated at compile time.

In my view a much nicer solution would be to add a function attribute, to force a function to be CTFEd. One possibility would be @ctfe. Another interesting one would be to repurpose the magic __ctfe keyword.

This would have two effects: (1) Any call to such a function would be CTFEd during the const folding pass. (2) The function should not be written to the .obj file. It does not exist at runtime. (Note that __ctfe is not transitive. A __ctfe function can call any runtime function, just like normal CTFE).

I don't know where you would put the __ctfe, there are a few possibilities, including:

string __ctfe foo(string s);

string foo(string s) __ctfe;

__ctfe string foo(string s);

I suspect that a great many extant templates are like this. They're just using template to force the use of CTFE. I think that a very large fraction of templates which accept value parameters, fall into this category.

[UplinkCoder]

I am agreeing with those who commented before me.

Invoking CTFE (on the top-level) has to predictable by a simple rule.
(Of course which function is executed by a CTFE as a results of a top-level ctfe call, is only possible by inspecting the function bodies recursively)

@nordlow As for your question, newCTFE does currently have somewhat higher latency in trivial cases, yes. That's an implementation issue and not necessarily systemic.

[WalterBright]

(1) We definitely do not want all CTFEable functions to use CTFE.
The most obvious example of this is compile-time regexp. And suppose you made a function to calculate PI to billions of decimal places, you want to run it at runtime because it's faster.
Because of this, you'd need to remove the 'pure' attribute from many such functions.

It's easy enough to cause these to be run at runtime:

auto a = pureFoo("hello");  // compile time
auto s = "hello";
auto b = pureFoo(s); // run time

This is because the front end does not attempt data flow analysis optimizations. Another way is to insert into the pure function a call to another pure function, where the source isn't available to the compiler.

(2) Not all pure functions can actually be run at CTFE. For example a pure function may call an extern(C) function.

Exactly. In such cases where CTFE fails, the function is run at runtime.

[WalterBright]

I'd like to be able to force hexString to be run at compile time. I'd like to even allow it to be used in cases like this: ´hexString("65" ~ foo(7) ~ foo(8));Ie, the calls tofoo` should be CTFEd as well.

If foo is a pure function, and it's arguments are literals, it will be run at compile time and the result constant folded into a literal before the hexString call is looked at, which delivers what you ask for.

[WalterBright]

How much more complex is something like eval!ctfeableFunction(foo, bar) really?

It causes a template to be evaluated and the arguments wind up in the mangled template name, which if the arguments are large can be a serious problem. A better approach is to use the enum method to force evaluation.

[WalterBright]

In my view a much nicer solution would be to add a function attribute, to force a function to be CTFEd. One possibility would be @ctfe. Another interesting one would be to repurpose the magic __ctfe keyword.

Don, you've (rightfully) complained to me about the current attribute soup for functions. You're proposing adding more!

This would have two effects: (1) Any call to such a function would be CTFEd during the const folding pass. (2) The function should not be written to the .obj file. It does not exist at runtime.

Removing unreferenced functions is a more general problem, and is dealt with by the linker.

(Note that __ctfe is not transitive. A __ctfe function can call any runtime function, just like normal CTFE).

Then the __ctfe attributed functions will fail to execute at compile time, and the compilation will fail.

[nordlow]

How do other languages handle this?

C++ has constexpr and in C++20 consteval. Rust has const fn. Both of these are explicit qualifications for CTFE.

[PetarKirov]

How about a solution that also addresses the code bloat problem? See dlang/DIPs#177 (comment)

[WalterBright]

https://issues.dlang.org/show_bug.cgi?id=21574

[WalterBright]

Shelving for later. Leave bug report open.