added a proxy server for create-unsigned-tx to delegate the taker side for easy wallet integration

[Giszmo]

To use joinmarket in all wallets, it has to be as easy as possible for the user and the developers to integrate, while providing the maximum security assuming bad/malicious code in the projects that are considered to be integrated (JM in this case).

As most wallets are not python based, an integration via the network was chosen. This allows integration with any wallet, having the JM part run either on an external server or on the device that also runs the wallet, maybe in a sandboxed environment.

The provided changes are

• create-unsigned-tx-proxy.py : a flask based modified version of create-unsigned-tx.py
• test-create-unsigned-tx-proxy.py : a "user", test-running the new server script on testnet.
• taker.py is extended to work with a signature instead of a private key as private keys should be exposed to the least code possible and with hardware wallets they are not available at all. Hardware wallet providers might be convinced to provide means to delegate communication to certain NaCl keys though, so there is hope this can be integrated with those on the medium to long term.

The test script run looks something like this:

    $ python test-create-unsigned-tx-proxy.py 
    2015-12-28 18:07:34,007 [MainThread  ] [DEBUG]  hello joinmarket
    2015-12-28 18:07:34,040 [MainThread  ] [DEBUG]  {'amount': 600000, 'testnet': True, 'makerCount': 1, 'recipient': 'mzGNyEEJNYtRi7fGnMAKTV5EZDFN8J9jTX', 'utxos': ['cb2caca7500d2d943b4cd03db20ee67fcabc6afa9548e930e81632d164f6bfa1:0'], 'authUtxoPK': '036ff095628f9623dcf5b5496c94ebe0783ce1a3a17bc1c5d339be1a8d6e08885e', 'naclKeySig': '47346d684b6c683131416473366d494b754730646b614474756a324651506262524c6231556d562b437436476a7555566f553953702f4341374856544f4750775150755064316f42694d6f384b474374765355706f6f673d', 'authUtxo': 'cb2caca7500d2d943b4cd03db20ee67fcabc6afa9548e930e81632d164f6bfa1:0', 'change': 'mtrxCNUMYXRy2hQh2bBPeLjtRHWFKZcRkU'}
    2015-12-28 18:07:34,057 [MainThread  ] [DEBUG]  sig is good: True
    2015-12-28 18:07:51,221 [MainThread  ] [DEBUG]  {u'result': None}
    leo@locurro:~/workspace/joinmarket$ python test-create-unsigned-tx-proxy.py 
    2015-12-28 18:15:03,684 [MainThread  ] [DEBUG]  hello joinmarket
    2015-12-28 18:15:03,716 [MainThread  ] [DEBUG]  {'amount': 600000, 'testnet': True, 'makerCount': 1, 'recipient': 'mzGNyEEJNYtRi7fGnMAKTV5EZDFN8J9jTX', 'utxos': ['cb2caca7500d2d943b4cd03db20ee67fcabc6afa9548e930e81632d164f6bfa1:0'], 'authUtxoPK': '036ff095628f9623dcf5b5496c94ebe0783ce1a3a17bc1c5d339be1a8d6e08885e', 'naclKeySig': '47346d684b6c683131416473366d494b754730646b614474756a324651506262524c6231556d562b437436476a7555566f553953702f4341374856544f4750775150755064316f42694d6f384b474374765355706f6f673d', 'authUtxo': 'cb2caca7500d2d943b4cd03db20ee67fcabc6afa9548e930e81632d164f6bfa1:0', 'change': 'mtrxCNUMYXRy2hQh2bBPeLjtRHWFKZcRkU'}
    2015-12-28 18:15:03,733 [MainThread  ] [DEBUG]  sig is good: True
    2015-12-28 18:15:24,307 [MainThread  ] [DEBUG]  {u'result': u'0100000002ba61e952dff3a63ee5c0f2e1a7a6b288c98b3ca8571c3113aa157e3448daf460040000006b483045022100b9118c48f4bfa55eb33916407a7d9129660805149f6c132f9509ae32edfbd0fe02200edb518f747df599e6d6a1e6300bdee61e82952effbb2788c60a15fba391fd110121029f67f5e1657edd41be0e948109af97b35f4c9ca0c24099c449991e0ca410cf3affffffffa1bff664d13216e830e94895fa6abcca7fe60eb23dd04c3b942d0d50a7ac2ccb0000000000ffffffff047bfa0500000000001976a9149261df64df7c2ec2941c62662dd1b25a6e12bd2088acc0270900000000001976a914cda8bf9c1ca736fdaa44861c738dda9266457fe988acc0270900000000001976a91481e4c88af9a84d3db7977aed9b1558cde6fa72f388ac1d340f00000000001976a9141385290972304786a127d85167361e98670c4ef988ac00000000'}

[raedah]

I suggest using SimpleHTTPServer from the python standard lib. We wont want to include the flask web framework as a dependency for JM. We use SimpleHTTPServer as such here https://github.com/JoinMarket-Org/joinmarket/blob/develop/ob-watcher.py . Should be easy to switch over.

edit: ghtdak's new version of ob-watcher uses twisted. here https://github.com/ghtdak/joinmarket/blob/txjoinmarket/joinmarket/ob_watcher.py and his txjoinmarket branch is what we are suppose to be moving to soon.

[Giszmo]

As discussed on IRC (I understood that @ghtdak changed ob_watcher to twisted, which would get merged anyway) SimpleHTTPServer is almost obsolete, while develop is in feature freeze, making a merge not an option until after develop got merged to master?

I like flask and have no idea (yet) about twisted and generally would hope that new features could get merged without being too picky about the implementation details that can easily be straightened out later. It's not like ob_watcher would have much to do with the joinmarket proxy and if joinmarket did more with web servers, using flask would really be more appropriate than SimpleHTTPServer but maybe there's just no hurry.

Also I'm confused, reading examples that use both flask and twisted, so probably I don't understand the problem at hands.

Ok, my code seams to be actually very very bad, as it would only serve to one taker at a time but this blog post for example suggests using twisted with flask. May I do that or is flask a problem? I'll copy the @ghtdak twistification, assuming that's acceptable.

[ghtdak]

That Joinmarket needs an html front end is clear. That we need something even as relatively lightweight as Flask built into the infrastructure is somewhat less clear.

I've been looking at Cyclone (a Twisted integration with Tornado) which includes bottle but I don't have strong opinions on html template type things. Twisted also has a tiny template language which they make a point of not pushing. Really, any or multiple template mechanisms will work.

Cyclone supports websockets which seems like a nice way to support dynamic pages when we get to that point but this is also completely independent of templating. (Cyclone has some good / simple examples)

At this stage, we're talking about a couple of resource paths for a web front end and to support wallet integration. A minimal HTTP front end which is async (Cyclone or straight Twisted) seems more than enough. If a more full blown front end is necessary, we can revisit.

All this being said, if someone has strong opinions, that's cool as long as it supports async. Most lightweight frameworks integrate with twisted. I recommend strongly against threading unless there is a demonstrated need, something I think is very unlikely.

[Giszmo]

@ghtdak sorry I had posted my last comment too quickly and updated it several times. I'm currently digging into twisted.web and stuff that's making ob_watcher work with twisted. So far I'm confused where there is an entry point at all. Will report back or push my results. Thank you.

[ghtdak]

https://github.com/fiorix/cyclone/blob/master/demos/helloworld/helloworld_simple.py

[Giszmo]

twisted.web now and cyclone.web later?

Anyway, I tried to turn it into twisted and that worked so far (many code paths would still crash, due to flask.abort and stuff) and it returns an unsigned tx with the help of some makers but it's still blocking and I haven't figured out yet, how to best solve that. Maybe with callInThread? But so far I failed to not finish the reply. request.write() always results in write() after finish().

[ghtdak]

twisted.web now and cyclone.web later?

No, I'd probably start with Cyclone though I doubt you need even that

Anyway, I tried to turn it into twisted and that worked so far (many code paths would still crash, due to flask.abort and stuff) and it returns an unsigned tx with the help of some makers but it's still blocking and I haven't figured out yet, how to best solve that. Maybe with callInThread?

Edit 1: As is often the case, I missed the context switch. I'll take a look and see if I can't hack something together given what you want on txJoinmarket even though you're asking about how to do Flask on the develop branch.

No. First, you can't really pull another entire framework onto Twisted (if that is indeed what you were trying). Frameworks involve event loops so unless the framework is integrated with Twisted, it ain't gonna work. And, there's virtually zero chance that you would want to for something as simple as this.

I tried looking through your code but there's too much Flask specific stuff to wade through easily. And, I definitely would avoid all the auth / security stuff at first. TLS and other security stuff can be added after you've got the application specific XML / Json / whatever generated.

Just start simple. Fork txjoinmarket and try to build something similar to OrderbookPageRequestHeader. That is a hack of ob_watcher I literally threw together in an hour so no guarantees. It seemed to work but wasn't my focus at the time.

Another approach might be to hack something from the Cyclone demo directories with a fake application object just to get the interaction going. Once you've got communication between your client and the test harness going, it will be pretty straightforward to import it onto the txJoinmarket implementation... and that'll give me a few days to finalize making txJoinmarket ready for use. (My code is still in the high-entropy refactoring stage though I am in the final stages so I have no problem helping an intrepid early adopter make txJoinmarket dance)

Later today I'll put something in experimental (a directory in my tree) to see if I can't point you in a better direction.