declare superinterfaces of existing types

[ghost]

I think we should be able to declare supertypes of existing types. Declaring superclasses would be fragile, but I think declaring superinterfaces could be pretty cool. Let's suppose I want the Object class to have one more member. html, for example, which returns a DOM representation of the object. I could do the following:

shared interface DOMRepresentable
abstracts Object
{
    shared default DOMNode html
    {
        dynamic
        {
            return(document.createTextNode(this.string));
        }
    }
}

Whenever someone imports our module, and our DOMRepresentable interface, they see that Object contains a html attribute, and them can override it in their subclasses.

This interfaces have to provide an implementation for all their members.

This could be a neat way to add members to types that aren't represented by classes/interfaces.

shared interface Foo
abstracts Iterable&Bar
{
    shared void foo() => baz();
}

void test()
{
    Iterable iterable => "Hello";
    Bar bar => Bar();
    Iterable&Bar ibar => getIterableBar();

    // iterable.baz(); // error
    // bar.baz(); // error
    ibar.baz(); // ok
}

I'm unsure whether there should be a special super annotation for those interfaces, nor if they should be implementable, or usable as types. I certainly don't want supertypes of Anything, specially it they are populated with objects that aren't instances of Anything...

[lucaswerkmeister]

Potential problem:

class C() {}
class D() {}
Set<C> cs = …;
Set<D> ds = …;
Set<Nothing> empty = cs & ds; // well-typed: C and D are disjoint

shared interface Foo
        abstracts C & D {}

Set<Foo> empty2 = cs & ds;
Set<Nothing> empty3 = empty2; // well-typed?

[lucaswerkmeister]

Let's suppose I want the Object class to have one more member.

I think extension methods, as proposed by #1146, are a much better solution to this need.

[ghost]

Well, if C and D are disjoints, C&D is the same as Nothing. This is reasoning done by the compiler, which makes interface Foo abstracts C&D the same as interface Foo abstracts Nothing, which doesn't make much sense...

[ghost]

I think it would be cool to declare new members to existing types that can be overridden, which isn't the case in #1146...

[lucaswerkmeister]

Another potential problem:

// module a
shared interface Described abstracts Object {
    shared default String description => string;
}

// module b
shared class Place(name, coordinates, description, rating) {
    shared String name;
    shared Coordinates coordinates;
    shared String description;
    shared Float rating;
}

// module c
module c "1.0.0" {
    shared import a "1.0.0";
    shared import b "1.0.0";
}
// error: non-actual member refines an inherited member: 'description' in 'Place' refines 'description' in 'Described'

Modules a and b can never be visible together, since then Place().description would be ambiguous.

[ghost]

No, those would be completely different members, that happen to have the same name. It could be disambiguated by explicitly importing one of the types, and renaming one of the problematic members. Sure, adding a abstracting interface to an already existent module could break modules that import it, but currently it isn't allowed for a module to "import the newest available version of another module", and things like that are the reason. Importing the newer module would either not cause problems, or fail at compile time, instead of compiling with a different behavior than expected...

[lucaswerkmeister]

shared interface Foo
abstracts Iterable&Bar

Could you explain in more detail what this means? I’m not sure why, in your example above, some lines in test get errors and other’s don’t…

[ghost]

It means that Foo is a supertype of Iterable&Bar. Since Iterable isn't a subtype of Iterable&Bar, nor is Bar, they don't get to be subtypes of Foo...

[lucaswerkmeister]

But why Iterable&Bar? Can I put any type expression there? I feel like Iterable|Bar would be more useful.

[ghost]

Yeah... I'm just showing that you could use intersections if you wanted as well.. Unions feel obvious enough, so I decided to not bother mentioning it... But indeed, I struggled to think of the usefulness of having intersections there... =P

[ghost]

Also, I can't agree with myself on what should be the behavior of having type parameters on those interfaces... I think it simply shouldn't be allowed...

[lucaswerkmeister]

I have a slightly amended proposal:

shared interface Super
        abstracts Sub1 | Sub2 {
    shared default String member => "";
}

The abstracts clause lists one or more type names, separated by a pipe (in analogy to of X | Y and satisfies X & Y). Super is a new type that covers the listed types, but they don’t become subtypes of Super. Thus:

Sub sub = …;
// Super sup = sub; // error: Sub is not a subtype of Super
value v1 = sub.member; // unambiguously refers to Sub.member
Super sup = sub of Super; // okay: Super covers Sub and Super is a subtype of Super
value v2 = (sub of Super).member; // unambiguously refers to Super.member
// value v3 = (sub of Super&Sub).member; // error: reference ambiguous between Super.member and Sub.member

Note: I don’t propose this should be added (I think I’m still against it), but I think this fixes some holes of the original proposal.

[ghost]

Is there any reason why you think this is better? I don't think it makes much sense for a type to inherit stuff from a type that it isn't subtype of... The of operator always increases the amount of members that can be accessed on an instance, but in your case, since Super isn't a supertype of Sub, Sub might declare members that Super doesn't have... This feels weird... What does Sub&Super mean? Because normally, with coverage, it would always mean Super, but if Sub isn't a subtype of Super, I don't think that that makes much sense...

I guess with my proposal, doing foo.member could mean always the .member of the most specific type. This could only cause problems if someone added member to an already existent class that didn't have member before. But again, this is something that can currently cause problems...

[lucaswerkmeister]

The of operator always increases the amount of members that can be accessed on an instance

Not true, it can always be used to widen to a supertype ("string literal" of Object).

What does Sub&Super mean? Because normally, with coverage, it would always mean Super, but if Sub isn't a subtype of Super, I don't think that that makes much sense...

Sub&Super can go both ways. String&Object is String (left side). Anything&<Null|Object> is Null|Object (right side).

[ghost]

Right. I got my thinking inverted. The of operator never increases the amount of operations that can be done on an instance. I don't think there is any relationship between the intersection of a type that contains another like I claimed there was. I was thinking about subtypes, and I got that inverted as well... =P

I don't like the fact that an object can be an instance of two types that aren't related. What does type(Sub()) evaluate to? How can it be an instance of Sub, and fit in a Super value, even through Sub isn't a subtype of Super?

[lucaswerkmeister]

The of operator never increases the amount of operations that can be done on an instance.

abstract class C() of D {}
class D() extends C() {
    shared String d = "d";
}
shared void run() {
    C c = D();
    // print(c.d); // error
    print((c of D).d); // okay
}

I don't like the fact that an object can be an instance of two types that aren't related. What does type(Sub()) evaluate to? How can it be an instance of Sub, and fit in a Super value, even through Sub isn't a subtype of Super?

Well that’s exactly the part that I also dislike :D how does your proposal deal with this?

[ghost]

Sure. I was going to correct myself, but you beat me on that ;P

Well that’s exactly the part that I also dislike :D how does your proposal deal with this?

The type in the abstracts clause of an interface is a subtype of the interface... =P

[ghost]

I generally get more confused when thinking about coverage than I do when thinking about subtyping, so sorry for the derps ;P

[lucaswerkmeister]

Yeah, coverage is confusing. But I feel like adding the ability to freely add new supertypes to any existing type might break the soundness or decidability of the type system. It just feels way too powerful to me. That’s why I thought restricting it to coverage might be better.

[ghost]

I don't think so... Specially because they are just interfaces. All you are saying is "add this interface to that type's satisfies list".

I also am unsure if it's right to allow you to use those interfaces as types, to explicitly satisfy them, or to allow they to satisfy stuff... That could be possibly be problematic, but I haven't thought about this too much...

[lucaswerkmeister]

But a type doesn’t have a satisfies list. Classes and interfaces have one; other types derive it from their component types.

With this proposal, in the code

X0&X1&X2&X3&X4&X5&X6&X7&X8&X9 x = nothing;
value v = x.member;

member could be defined in a superinterface defined for any of 1023 combinations of X0…X9. And in combinations of supertypes of X0…X9, and so on. That doesn’t yet make the typechecker undecidable, but it’s still a complexity explosion that I don’t think can be produced today.

[gavinking]

This looks a lot like (but perhaps not exactly like?) a notion we discussed extensively several years ago under the title "introductions". You can see some of that discussion here. The idea was dropped because there were quite serious concerns about implementability on the JVM, and because it introduced some ambiguities that would impact modularity. Introductions are quite a lot like (but admittedly not quite as bad as) implicit type conversions.

[gavinking]

I have a slightly amended proposal:

shared interface Super
        abstracts Sub1 | Sub2 {
    shared default String member => "";
}

For this to be actually useful, Sub1 would be defined in a third module, separate from the module Super, and from the module that contains the client of Super.

What happens if the owner of this third module adds a member named member to Sub1? Now, from the point of view of the client, Sub1 has two different conflicting definitions of member.

[ghost]

As I said above, Sub1 has two different members, both with the same name. This can be disambiguated by explicitly importing one the interface, and renaming the member.

// module/package some.module
shared class Sub()
{
    shared String member => "Hello";
}

// module/package another.module
shared interface Super
abstracts Sub
{
    shared String member => "Goodbye";
}

// module/package my.module
import some.module
{
    Sub
    {
        hello = member
    }
}

import another.module
{
    Super
    {
        goodbye = member
    }
}

shared void run()
{
    Sub s = Sub();
    print(s.hello); // prints "Hello"
    print(s.goodbye); // prints "Goodbye"
}

[gavinking]

So according to that, a seemingly innocuous change to Sub1 (adding a member) breaks some of its clients. Perhaps that would be acceptable if introductions added some real additional expressiveness, as is the case with subtyping, which admitted can break in the same way. But introductions don't really let us express anything we can't already easily express without opening ourselves up to this risk.

[gavinking]

I mean, basically all an introduction is is a single multiplexed object reference that could just as easily be represented using two distinct object refs.

[gavinking]

To be clear: the current rule is that addition of a member to a type cannot break clients of that type. It can potentially break a subtype, and thus, potentially, clients of the subtype. That's a risk you run when you use subtyping. Now, sure, we could say that you take on the same risk when you use introductions. And that might be OK if introductions added a whole lot of value. But I've come to the conclusion that they don't. And they would be quite difficult to implement unless you introduced some pretty draconian restrictions:

• no Identifiable introduced types
• introduced types are always stateless interfaces

And probably some others I can't remember right now.

[gavinking]

Oh yeah now I remember another huge source of problems with this: if X is introduced to Y then we can't be sure that Y doesn't have a subclass Z that already implements X. Even worse, if X is generic, then Z might implement a different instantiation of X! Then we are really in the shit.

[quintesse]

I would like to add that with this kind of additions to the language it's not sufficient to show that it can be done but it's necessary to come up with real examples that show how the new feature will make things better, how it will be an obvious improvement over the alternatives available to the language right now. We don't want Ceylon to be a kitchen sink of "neat ideas".

[RossTate]

@gavinking, you can check all those things. The important thing here is that the introductions are listed in the same module as the interface being introduced. That gets around a lot of ambiguities and conflicts (though not the method one you point out). There still are a variety of challenges, but I figured I should at least rule those ones out.

[gavinking]

@RossTate but introductions defined in the same module are not very useful.

[RossTate]

That's not true. Suppose you're writing a pretty-print module. You create a Formattable interface. You want to retrofit the standard library to use your interface. That's a practical use case.

Now, there is another practical use case that this doesn't address. Namely if you're using a database module and the pretty-print module and you want to make the queries in the database module implement Formattable. You can't do this with the proposed approach.

[gavinking]

@RossTate well that sounds like a case that can be adequately handled using a type alias for an intersection type.

[gavinking]

alias Formattable => String | Integer | Float | Date | Boolean | CustomFormattable;

[ghost]

Besides the fact that the "standard library" is another module...

[ghost]

The problem is still there. If you guys decide to add some other member to a type in the language module, some clients could break...

[gavinking]

@Zambonifofex WDYM?

[ghost]

Suppose my module adds an attribute to String named sort. It straightforwardly sorts the characters in the string by codepoint value. My module gets popular, and people start to call this sort attribute all around. Now, suppose you also decide that String deserves an attribute named sort. Suddenly, modules that used my module, and its sort member stop compiling/working as intended...

[gavinking]

@Zambonifofex well, yeah. Now, with something simpler like extension methods that might not be such a big deal, since you could just say that the extension method hides the real method, and that might be good enough. But honestly I just feel like none of these things are solving any real problems.

[ghost]

Well,there isn't any particular problem I'm trying to fix. I gave you an example of where this could be useful (adding a .html member to object), but you could always have a regular HTMLable interface, and an html function, that calls a .html attribute if it's parameter is HTMLable, and create a text node otherwise. But that's not what I really mean. I'm basically writing inheritance by myself.

It isn't because there aren't problems that something solves, that that something is not worthwhile. I mean, object orientation didn't come to fix any particular problem. It was just something that was pretty cool, and people started adopting it. I know this is a very loose comparison, but I think that this is one of the things that you need to try out in order to see more concrete usefulness. I'm not trying to solve problems here, but rather, to open up possibilities...

I also think that there aren't many downsides. You can only ever break stuff if you change the imports of your module. And in that case, the compiler will instantly warn you. It's a 3 minute fix. Go to the file, refactor, add an explicit import, repeat. It isn't something that should happen frequently either. A member needs to be added that have exactly the same name as the member in the abstracting interface in order for something to happen.

I guess that it could harm decidability, but I can't think how. It doesn't feel like a potentially-dangerous feature to me...

[quintesse]

@Zambonifofex I can assure you OO was not created and adopted by people just because "it was cool". At that time you either programmed in C, Pascal, Basic (of not assembly) and for many people not ending up with this big plate of spaghetti code was a huge problem. So language designers were always experimenting with ways to modularize code. And in the end OO won out. It still remember it as this paradigm shift that made you think about objects and relations and whatnot. Nowadays we might discuss the actual merits of OO but back then they seemed pretty obvious.

Now, Scala on the other hand, for many of us is the prime example of a language that is just chock full of things that are "cool", some turn out nice, others not so nice. But the creators admit it's because they see it as an academic language where you can try out new ideas.

But we always talk about our "complexity budget", each thing you add to a language has to be weighed: the advantage it gives the programmer against the cost of learning it (naively one might say: if you don't understand it, don't use it, but you still have to read and understand other people's code, a big problem IMO with languages like C++ and Scala)

So that's why Gavin says "I just feel like none of these things are solving any real problems". Unless you can show some real advantage (or low "cost") you'll meet resistance. And even when we had examples of "real advantage" we've sometimes agonized over adding things to the language, see for example Tuples and Constructors. But that's because we kept running into situations where code was obviously worse without them so in the end we decided it was (probably) worth the trouble.

[ghost]

Well, you can add member to already existing classes, and allow classes that are aware of that override those methods. WIth a modified version of this concept, you could add methods only to classes with certain type parameters, and even make them satisfy new interfaces. This feels like a nice approach to simple conditional inheritance... You could, for instance, have predicates inherit a common supertype of booleans, and allow people to use && and || in them...

I don't see that being too hard to understand. It straightforwardly adds a supertype to an existing type... I think it could be really useful - in ways we can't maybe see yet - for not much learning cost. Object orientation indeed solved a problem. But people didn't knew it would solve it yet... Developers started adopting it, because it looked like a cool thing, and only after it was being used, that people could more clearly see the advantages of it, and that's what made the concept grow popular. People found ways to solve things that weren't considered problems before.

[quintesse]

I don't see that being too hard to understand

It's not about being hard to understand. Having too many features or several ways of doings things incurs a cost in itself. That's why we talk about "complexity budget", and that's why simpler languages are often more popular.

Developers started adopting it, because it looked like a cool thing

I think that for many developers that's never how it goes. They have work to do, want things finished yesterday, hopefully with as little work as possible. For them "cool things" are a luxury and often an obstacle. Scala's coolness is an obstacle, it makes code hard to understand if you're not proficient with the language (see http://www.scala-lang.org/old/node/8610). That's explicitly not where we want to go.

I'm not saying this is not a useful feature, but it being "cool" is not going to help it get adopted, giving examples of the things that you can do with it and how the alternative without it would be much worse will.

[ChristopheLs]

Agree with @RossTate,
I explain
With a standard class from one module and a Formatable interface from another, you have to use the adapter pattern

shared interface Formatable {
    shared formal String format();
}
shared class SdtClassLib() { }

shared class AdapterStdClassLib(SdtClassLib wrapped) 
        satisfies Formatable {
    shared actual String format() {  return "X";  }
}

void fun() {
    SdtClassLib x = SdtClassLib();

    // here, to use Formatable interface, you have to know that
    // the implementation of the classe of x is AdapterStdClassLib
    // (and even more if there are other implementation for SdtClassLib's subclasses).
    AdapterStdClassLib adap = AdapterStdClassLib(x);
    adap.format();
}

The pb here is that you have to know the AdapterStdClassLib class in function "fun" (and more if StdClassLib has subclasses with other adapter).

With the proposition, you could have something like

shared interface Formatable
abstract SdtClassLib {
    shared actual String format() { return "X"; }
}

shared class SdtClassLibSub() extends SdtClassLib() { ... }

// Another implementation for the sub class
shared interface Formatable
abstract SdtClassLibSub {
    shared actual String format() { return "sub X"; }
}

void fun2() {
    SdtClassLib x = myFun();

    // automatically take the good implementation of Formatable
    // of the real class of x
    x.format();
}

here, i don't know if x is of class SdtClassLib or SdtClassLibSub, and then which method format will be actually call (exactly the same way if these two classes were satisfies Formatable interface directly).

[RossTate]

@gavinking, that's a cool solution! The one downside is that it may have poor performance due to having to case-match in the JVM. But that me a reasonable tradeoff.

[gavinking]

@ChristopheLs

With a standard class from one module and a Formattable interface from another, you have to use the adapter pattern

No, you do not. That's my point. You would in Java, but not in Ceylon, since Ceylon's type system is just so much more powerful.

shared alias Formattable 
        => String | Integer | Float | CustomFormattable;

shared interface CustomFormattable {
    shared formal String format;
}

shared String format(Formattable arg) 
        => switch (arg) 
        case (is String) arg 
        case (is Integer) formatInteger(arg)
        case (is Float) arg.string
        case (is CustomFormattable) arg.format;

shared void printf(String text, Formattable* args) 
        => print(args
            .map(format)
            .fold(text)
                ((str, arg) => str.replaceFirst("$", arg)));

There are no adaptors in sight!

[gavinking]

@Zambonifofex @RossTate Note that using the pattern above, one can achieve quite a great deal of what can be achieved using introductions. (Except of course you have a regular function invocations instead of postfix-style method invocations.)

What it doesn't provide is the ability to make an object from a third library masquerade as a Formattable when neither the formatting library nor the third library know about each other. But if I'm not mistaken, this is the very case you would have to disallow anyway even if you were to add introductions to the language.

Of course, that problem can be solved using a wrapper:

class Unformattable2Formattable(unformattable)
        satisfies CustomFormattable {
    shared Unformattable unformattable;
    format => unformattable.string;
}

Now, sure, that requires you to call printf() like this:

printf("$ $ $ $", "hello", 1.0, 
        myCustomFormattable, 
        Unformattable2Formattable(unformattable));

But I don't see that as really that painful, frankly.