Added initial async substrate changes.

[thewhaleking]

No description provided.

[arjanz]

Ok I had a go with AsyncSubstrateInterface, some feedback:

• I had to rename async.py to async_module.py because of a reserved name conflict
• You renamed kwarg url to chain_endpoint which I agree is a better name, but is currently incompatible with current SubstrateInterface.
• I created the example below, which ran smooth for a storage query and sending sequential extrinsics. But when I tried to submit extrinsics in parallel using asyncio.gather it got stuck after the first extrinsic got submitted. But I'm not sure if I'm doing it right:

import asyncio

from substrateinterface import Keypair, ExtrinsicReceipt
from substrateinterface.async_module import AsyncSubstrateInterface
from substrateinterface.exceptions import SubstrateRequestException

import logging
logging.basicConfig(level=logging.DEBUG)


async def balance_transfer(substrate, keypair, amount) -> ExtrinsicReceipt:
    async with substrate:
        result = await substrate.query(
            module="System",
            storage_function="Account",
            params=[keypair.ss58_address],
        )
        print(f"Current free balance for {keypair.ss58_address}: {result['data']['free']}")

        call = await substrate.compose_call(
            call_module='Balances',
            call_function='transfer_keep_alive',
            call_params={
                'dest': '5FHneW46xGXgs5mUiveU4sbTyGBzmstUspZC92UhjJM694ty',
                'value': amount * 10 ** 15
            }
        )

        extrinsic = await substrate.create_signed_extrinsic(
            call=call,
            keypair=keypair,
            era={'period': 64}
        )

        try:
            # wait_for_inclusion resulted in error `TypeError: a coroutine was expected, got {'jsonrpc': '2.0', 'result': True, 'id': 1}`
            # receipt = await substrate.submit_extrinsic(extrinsic, wait_for_inclusion=True)
            receipt = await substrate.submit_extrinsic(extrinsic)

            print(f'Extrinsic "{receipt.extrinsic_hash}" submitted')

            return receipt

        except SubstrateRequestException as e:
            print("Failed to send: {}".format(e))


async def main():
    substrate = AsyncSubstrateInterface(
        chain_endpoint="ws://127.0.0.1:9944"
    )
    keypairs = [
        Keypair.create_from_uri('//Alice'),
        Keypair.create_from_uri('//Bob'),
        Keypair.create_from_uri('//Charlie')
    ]
    
    # Sending sequential extrinsics works fine
    # for amount, keypair in enumerate(keypairs):
    #     await balance_transfer(substrate, keypair, amount + 1)

    # Run multiple transfers in parallel
    # After first transfer it got stuck and eventually got: DEBUG:websockets.client:= connection is CLOSING
    await asyncio.gather(
        *(balance_transfer(substrate, keypair, amount + 1) for amount, keypair in enumerate(keypairs))
    )

if __name__ == "__main__":
    asyncio.run(main())

Do you have any idea?

Besides this there is of course the need for some unit tests and documentation, but I will take care of that. Probably I'll merge this PR first in a new branch before merging to main.

[thewhaleking]

Ran into similar problems with our own implementation. I've fixed most of them. Will update the PR this week.

[thewhaleking]

I did update this with some improvements, but this stems from this being an incomplete port. Specifically regarding the way that the SCALE decode is handled. Calls are still generated using the original py-substrate-interface code, and this is not thread-safe, so it tends to break.