Implement support for account abstraction

[EdgeDLT]

This issue is intended to replace and continue discussions that started in #2907.

Summary or problem description

Neo lacks flexibility in regards to how users may interact with the network. We make too many assumptions about usability and it leaves massive barriers to adoption that are not feasible to overcome at the application layer. Examples include the fact that we require regular users to manage private keys or to initially load an account with GAS before they can do anything with the network.

We can build better applications and experiences by abstracting these things away from the end user. Depending on the approach taken, possible benefits include:

• GAS-free transactions (pay blockchain fees with any token)
• Support alternative verification approaches (e.g. quantum resistant signing schemes and keyless validation)
• Secure arbitrary transaction sponsorship Practical transaction sponsoring #2577
• Solve the node incentivization problem through proposer-builder separation [suggestion] P2P module Enhancement #2862 (comment)
• Multi-sig aggregation Network assistance for multisignature transaction forming #1573
• Transaction conflict resolution conflict attribute stops the network #2907

Do you have any solution you want to propose?

Taking inspiration from ERC-4337, Neo should implement support for user operations. A user operation documents a user's intent for a transaction and grants increased flexibility for verification.

User operations are shared with the network through a separate channel than conventional transactions. Special operators, "bundlers", presumably an enhancement to P2P Notary, bundle these operations into transactions targeting a new native contract (Notary extension?), which unbundles and executes them. By sending the bundle transactions, bundlers pay any associated system/network fees, and are expected to be reimbursed in the process by the user operation. This reimbursement can be in any token desired by the bundler. Presumably a non-altruistic bundler will refuse any operations that do not meet its fee or validity checks.

User operations should allow the use of a new interop, along the lines of CheckData #2866. This interop should allow arbitrary verification logic to be validated, beyond what is possible with ECDSA alone. This, as an example use case, allows us to support end users that don't need to store private key material.

This facilitates a new type of Smart Account capable of onboarding any user regardless of their background. It also opens the door to several other benefits, such as Social Recovery services and transfer quotas, offering protection in the event that a user loses access or otherwise compromises it.

Optionally, we can explore the ERC-4337 concept of Paymasters as a method for doing practical arbitrary transaction sponsorship. A dApp should be able to maintain a GAS balance that any user operation can spend providing the operation abides by the requirements defined by the dApp.

Neo Version

• Neo 3

Where in the software does this update applies to?

Probably small changes to most of it, some more significant ones in parts.

• Compiler
• Plugins
• Ledger
• Network Policy
• P2P (TCP)
• RPC (HTTP)
• SDK
• VM

[EdgeDLT]

@roman-khimov Continuing from #2907 (comment):

We can think of extending it. At the moment it can do both multi-signature (multiple keys for a single signer) and multi-signer, given the signer flexibility of Neo that can solve a lot of things.

It's great. There are several use cases that already interest me on MainNet, GrantShares voting for example. But before we can extend this, we have to extend the Neo protocol. This is part of the proposal that is novel to Neo. The rest of the blockchain world is learning to not hate its regular joe users by having unrealistic expectations of them. In user land, private keys are going away. Most KOL's in the industry are convinced this is how the floodgates to blockchain adoption will ultimately be opened, and I find it hard to disagree.

This is the purpose of account abstraction. It's considered one of the "holy grails" in Ethereum and is the overarching goal of ERC-4337. All it really means is arbitrary validation logic. As in, you can swap in any signature scheme you want, say a quantum resistant one, or you could not require any secret material provided by the user at all.

Because Ethereum is too bloated and adopted to make protocol changes, they landed on alt mempool + smart contract wallets with "counterfactual addresses" (just means deterministic address, so you can derive it and transfer to it before the contract is deployed). But that's just the thing. They still have to deploy a new proxy contract for every single user. Okay for L2, too expensive at L1. It's the same smart account problem again.

On the other hand, in Neo-land we have arbitrary verification logic just sat there, not to mention a willingness to make protocol changes. It's ironic that without any intention to be in this position, we are so much closer to a so much more powerful solution than what Ethereum has been chasing for years 😄

To me it looks like most of your problems are #2577, but maybe I'm missing something.

The GAS problem is separate from the keyless verification problem, though they can be solved together.

While we are custodial, we can sponsor trivially as sender, and we do. But we are forced to go non-custodial. And we don't want to ask a user to store a key or download a wallet in order to perform actions. It defeats the entire point of the application, which is Web2 -> Web3 onboarding, i.e. use the blockchain from within the applications you are already using.

So our limitation is ECDSA. CheckSig, CheckMultiSig, CryptoLib.VerifyWithECDsa are useless to me. This is where #2866 comes in. Currently it asks these questions to "signers":

Who is paying for it?
What nonce/fees/other technical fields are?
What is the script this transaction will execute?

Which is a good start. But we have to move beyond signers. We need "users", freed from:

What's a private key?
How do I get GAS?

A CheckData-like interop is exactly the direction I'm looking for. But it needs to be general. For some apps, maybe I get lucky and can get standard signatures out of it. For example, we can get ed25519 signatures out of Discord. That's fine, we can just support the curve (hi #1614).

But maybe Telegram or GitHub doesn't support it. Maybe I want to do a biometric verification, or an email OTP. In that case, I need to be able to point the interop at my custom verification handler. Maybe I have to deploy a contract for that, I'm not sure, it's an implementation detail, but if so that's fine, my user still shouldn't need a proxy contract.

[vang1ong7ang]

I believe these are the features that users on neo want most.Neo's witness mechanism allows us to have full flexibility in transaction authentication. This is something that many other layer-1 networks cannot provide.

[vang1ong7ang]

something like the mentioned CheckData is necessary to enhance the functionality. However, the design in #2866 is not rigorous enough.

I wonder why witness scope was not mentioned there. witness scopes are necessary and important part under current authorization policy of neo.

---

I'm not sure if the smart contract worm virus is known by @roman-khimov that happened on NEP-5 on neo 2.0 and I'm going to explain it a bit. it would be the first smart contract virus in the history.

For a long time, users have some understandings of blockchain usage that not always true (may influenced by ethereum)

• Transferring an asset from my wallet will not affect my remaining assets
• Extra assets in your wallet are harmless
• the allowance - transferFrom pattern

at that time, a fake NEP-5 asset named neo was airdropped to many accounts and when someone tried to transfer the fake neo, during the execution of the transfer (written in the smart contract of fake neo), all other valuable assets are transfered to the hackers' addresses. ( while the fake neo went to the next victims and let it be able to spread )

that's the exact reason why witness scope is introduced in neo3.

No one is going to take responsibility for this incident at that time:

• the protocol devs said: users should know what will happen before sign the tx
• the wallet devs said: we follow the standard NEP-5 operation
• the users said: I'm just trying to transfer the (fake)NEO, why my BTC/USDT/GAS got disappeared?

We often put many unrealistic and even higher requirements for our users and application devs than ourselves.

[roman-khimov]

#2866 is made to be app-specific in that it's a contract that decides how to interpret a particular request. That is its strength (not hard to implement) and that is its weakness (can't do any invocation with it) at the same time, as usual.

What's a private key?

This can be hidden by contracts that manage accounts for their users. In fact, that was one of the things I had in mind with #2866, things like Contract.VerifyUser(name, data) where name is hardcoded in the verification script (giving each user a unique address even though the contract is the same) and data (whatever that is) comes from the invocation one are entirely possible.

How do I get GAS?

Well, that's #2577 as is, what we're trying to do in our sidechain now is to have a scheme where no sidechain GAS is required for any operations. And here I mostly mean storage node operations, users don't need it already. In general it should work this way:

• a new account can request some amount of GAS to be locked for him in the Notary contract (like 5 GAS)
• he can send Notary transactions then that are paid for by the proxy contract (which has a lot of GAS)
• these transactions are confirmed by the Inner Ring if they're fine (and Notary deposit remains untouched)
• or not confirmed if they're not (in which case account can quickly run out of deposited GAS and be effectively blocked)

In some ways we can do that because NeoFS sidechain is just that, a NeoFS sidechain, we don't have any random activity there, only NeoFS management operations. While this scheme is transaction-based now, we do have some notion of user intents at the same time. Container creation (done by end users) is an example of that, currently it works this way:

• container structure is serialized and signed by the user
• it's sent to some storage node
• which creates a transaction paid for by the Proxy contract (with fallback from SN account) and requiring a confirmation from IR
• IR receives a notary request notification, checks the transaction, checks the container structure and adds signatures in a usual Notary manner
• once there are enough signatures, the transaction is good to go into the chain
• on-chain container management contract can (but doesn't currently do that, I have to add, relying on validation by IR) check for a signature, but only if it's a standard ECDSA
• it creates a container (unfortunately, it's not NEP-11 now), but this container is owned by the user and only this user can delete it (in a similar manner)

But again all of this is app-specific and container registration just costs some real (mainnet) GAS that is written off of the user's balance (relying on IR multisig as well). Doing arbitrary executions is more challenging. Oracle invocations can be an example in some ways (signers are substituted there and some other native contract could do that in a different manner based on some token data), but I'm not ready to propose anything here.

[EdgeDLT]

This can be hidden by contracts that manage accounts for their users

One account for one authorization method is fine, but app-specific accounts are simply not good enough. If a user can create a wallet with their Gmail, that Gmail wallet should work with any application. It shouldn't have a different address based on what application someone is using. There's nothing user-friendly about that.

If I can use my Gmail wallet to get bNEO from NeoBurger, I expect to be able to use that Gmail wallet to stake it with Flamingo too.

From what I've studied of Neo's architecture and forgetting the current design of CheckData for now, I believe what we require is something that takes a target scripthash (pointing at some target auth verification contract) and some unique data from the user (related to the auth method they are using). This would generate a unique address from the verification script for each user in the same manner as you described. Invocation script should then contain any additional data needed to pass authorization depending on the verification contract chosen.

Wouldn't this approach give one address per user per verification method in a way that is completely dApp-agnostic?

Again, hard to overstate how superior this solution is compared to Ethereum's account abstraction approach, thanks to verification scripts. An ERC-4337 user pays N fee for N smart wallets. An N3 user has to pay 0 for N smart wallets.

a new account can request some amount of GAS to be locked for him in the Notary contract (like 5 GAS)

It falls short of the requirements. Sponsorship is an option that dApps should have available to them, but it's not a silver bullet for user onboarding. A user that has ANY token in their account should be able to make a transaction, without first acquiring GAS or requiring someone else to pay it for them. The proposed system solves this by letting the user allocate some of a NEP-17 to the bundler in return for the bundler covering tx fees. This is completely generic. If the bundler accepts the NEP-17 held by the user, it just works, and it only ever took one on-chain transaction.

[roman-khimov]

Wouldn't this approach would give one address per user per verification method in a way that is completely dApp-agnostic?

Sure, by "contracts that manage accounts for their users" I just mean that user accounts are tied to this particular contract, but they're proper Neo accounts this way and can enjoy all the properties of proper Neo accounts. Some trivial example is a contract that stores name->public key mapping and expects a signature in invocation script. So verification one will look like contract.AuthUser(name) and won't have any key, the key can be k1 (stored in a contract) and it can be changed if needed without script hash change. The contract itself will be the same for as many users as needed, no need to deploy additional ones per-user.

A user that has ANY token in their account should be able to make a transaction

I see what you mean, but transactions are paid for with GAS. And someone has to pay some GAS to have things happen. This entity can theoretically take other tokens for its service, but they have to convertible into GAS in an easy way, otherwise this sponsor will run out of GAS anyway. But if something can be converted into GAS easily, why a user can't do that? If we imagine transaction sponsor taking arbitrary tokens it'll likely require some premium for doing that, because taking non-GAS is a risk for him. I'd expect this to be more than just a conversion rate, so likely users will be economically motivated to convert by themselves.

[cschuchardt88]

I feel like there is all these options, but no written down in stone defined way to go about executing the situation, what that is; Is variable. To be honest doesn't really matter, What matter most is know how to drive, we know we need keys, gas and money. What is the neXt? That's a plan for a trip. But we are missing the roadmap, goals... a governance if i may. Where do these bylaws come from? Where and how do we seek these entities? Its the 90s and domain names just came out and now i want a GPS to find where to go. But you know how to find a store. You put your order in knowing someone is responsible. When you step into a bank we all know how it works. And at the same time don't even care what the cost is; to do something or the responsibly. All we know is we want someone to do it for us at whatever the cost may-be. We're in the ever growing world of NOW NOW NOW NOW. All things aside these are my cryptic thoughts on my 1st take. But one problem i had; That you shouldn't need; to read code to figure out how stuff works. I think getting more proposals updated and set in stone in a reasonable time frame. That is what holds me back from dump tons of hours into anything that isn't set in stone... get funding and do it all yourself.... Now you need PHD in cryptographic and business... Off-topic..... 😺 ☺️

[Jim8y]

@shargon how do you think?

[EdgeDLT]

And someone has to pay some GAS to have things happen

Yes, someone has to pay the GAS at the end of the day. But we want to abstract that away from the user. It should not be the user's problem. The whole issue with blockchain today is that we make everything the user's problem.

Consider the user experience with the proposed solution:

1. User opens their wallet (signs in with OAuth2)
2. User sees they have $5 in FLM from some event
3. User sends a "transaction" (actually a user operation) to transfer $4.70 FLM somewhere with $0.30 FLM fee
4. Bundlers accepts the operation, packaging it along with other operations into a real transaction, pays the GAS
5. User sees that their transaction completed on-chain

At no point in this flow does the user need to understand how private keys work and how to keep them secure, or that transactions on Neo need GAS, or where to get that GAS. They just sent a "normal" transaction from an easy-to-use wallet and it magically worked.

But if something can be converted into GAS easily, why a user can't do that

Expecting too much from the user again.

It's absolutely fine if a technically capable user wants to manage their own keys and send their own transactions in order to save on costs. But we should absolutely not require that of everybody in order to make use of the chain. The majority of people are not technically capable. No one is adopting the technology like this.

it'll likely require some premium for doing that

There's nothing wrong with it costing more to make use of these abstractions. Maybe to do a manual FLM transfer, it would cost $0.12 GAS instead of $0.30 FLM via the bundler. So what? If you want to learn how everything works to save money, be my guest. Right now we require the user to learn how everything works. It's not good enough.

All we know is we want someone to do it for us at whatever the cost may-be

You're exactly right.

Everything we require users to do in blockchain is front-loaded with an impossible amount of information and knowledge. What's an exchange, how do I safely store keys, if I have $value of tokens in my wallet why can't I afford the fees, and 100 other such questions.

Account abstraction means hiding these unnecessary confusing details around account ownership away so that the user doesn't have to care. I don't care what the final solution looks like really, but I know what barriers need to be removed when it comes to UX.

[cschuchardt88]

To clarify i love all these ideas. I would to love to turn neo into operating system or vm into a kernel based subsystem and we all could but we need more goals set into place with clear roadmaps i know help is low or limited or maybe not. but im ready to run on fire... cryptic math so much fun.