Can we use an existing Python figshare package?

[dhimmel]

For example can we use:

• rmcgibbo/figshare (MIT License)
• PyRDM: pyrdm/pyrdm / paper (GPLv3 License)

Figshare also has some python examples although they are not packaged

[dhimmel]

@vasudevanv writes in cognoma/cognoml#15 (comment):

Awesome. I started working on a fish are module with upload new, download and update existing article features. Can push something over the weekend.

@vasudevanv let's make sure rmcgibbo/figshare isn't what we need first. It may be easier to just contribute to that repository. @rmcgibbo do you have the maintenance bandwidth to deal with pull requests?

[vasudevanv]

@dhimmel Agreed. From a cursory glance this seems to fit some of our requirements and functionally similar to what I have so far.

[dhimmel]

Here's what I currently see our needs as:

1. Download: given a figshare article_id and version (could be None for latest), download all files (or a specified subset of the files).

2. Upload: we probably don't need to create new articles, just update existing ones. The update should see if a file hasn't changed (same checksum) and not reupload this file.

So don't feel pressured to use rmcgibbo/figshare, just if you think it will save time. We have the luxury of not caring about python 2 support, so we can make using newer syntaxes and no six.

[rmcgibbo]

I am not currently using rmcgibbo/figshare or actively maintaining it (although I have no reason to believe a-priori that it's broken). If you wanted to modify it, fork it, take over ownership of it, or anything else, I would be really happy. I don't have a ton of bandwidth, so I don't want to promise to help in a way that I can't be confident I'll be able to deliver on. And I don't want to hold you back in any way, but I'd be happy if you were able to get any utility out of code I wrote.

[wisygig]

@rmcgibbo: Oh, hey cuz.

[rmcgibbo]

hahaha, lolz.

[NickleDave]

Hi @rmcgibbo @wisygig @dhimmel @vasudevanv I just want to check where you're at with developing/maintaining Figshare packages.

I would really like to have a command-line interface to the Figshare API that is as transparent to the user as possible. I took a stab at writing my own tool, just because starting from scratch helps me have the whole thing in my head
https://github.com/NickleDave/pyfigshare
and I have drawn a lot of inspiration from what you put together as well as the RFigshare package from @cboettig et al. (https://github.com/ropensci/rfigshare).

It would also be great if we had a similar tool for PyOpenSci (https://github.com/pyOpenSci), but to meet their requirements, the package would need docs / tests / continuous integration. I care so much about being able to get my data on Figshare from the command-line that I am actually willing to do that work. (Maybe I need a hobby.)
I ran into a brick wall with the code though, where I can't get OAuth2.0 web app workflow to work from the command-line, I think because of issues with Figshare requiring an https redirect url.

I asked people about this on Twitter and one of the organizers of PyOpenSci @lwasser suggested I also post on their Discourse about it, and reach out to you all. Maybe we could join forces to get a really easy-to-use tool for the community, and/or you could help me figure out what I'm getting wrong now.

Happy to hear your reply here and if you want to give input to the PyOpenSci community, the discussion is at: https://pyopensci.discourse.group/t/figshare-package/113

Thank you for developing these packages and sharing them

[dhimmel]

@NickleDave you are right that this package is not actively maintained at the moment. It'd be great to get a single Python package for interfacing with figshare that is robust and complete enough to serve as the primary Python API going forward.

I can't help you with the OAuth2.0 stuff, but interested in following development progress and testing it out when I have a use.

I wasn't aware of https://www.pyopensci.org/. I don't see any actual packages on their website currently. Is the idea that eventually they will create a suite of packages? Is the idea that pyfigshare would live under their organization and be maintained that way?

[NickleDave]

Hey Daniel. Understood -- I'm beginning to wonder if anyone can help with OAuth2.0. I think it might be a question of asking Figshare to make their API a little more amenable to CLI tools.

Great, would love to have someone who knows what to look for test it out.

Yes, PyOpenSci is pretty new, they have one package through review so far and a few in the pipeline.

Depends on whether they decide as a community that a figshare package would make sense, but I think it's likely given that there's a similar package for ROpenSci.

[vasudevanv]

Do you think it is worth exploring the option of updating this repository and maybe merging it with some other code to create a complete python API? If yes, we could submit that package to pyopensci.

I can contribute some code, but definitely do not have time at the moment to commit to maintaining the package. Maybe in the future.

[dhimmel]

Do you think it is worth exploring the option of updating this repository and maybe merging it with some other code to create a complete python API?

I'd say that's up to whoever is taking the lead on maintaining the PyOpenSci package, which it sounds like will be @NickleDave? I agree it would be great to get the contributors from cognoma/figshare involved, whether this is cherry-picking certain parts of the code base with appropriate commit authorship or having us submit PRs with certain aspects.

[gwaybio]

Also very interested in this 👍

[lwasser]

hey all! this is an awesome discussion. the way pyopensci works is that awesome people like yourselves submit packages for review. We can however help with processing like this where it might make sense to merge efforts to create a usable end package! however success of a package does also depend on it having atleast one committed maintainer. this would be a really excellent package to add to the pyopensci suite!

note we hope to update our website soon with our packages under review vs reviewed!! we are new :)

[NickleDave]

Thanks for the replies @vasudevanv @dhimmel.
Yes, I'm volunteering myself to be a committed maintainer.
I definitely want to include contributions from everyone where I can. I think this will require looking at the 2-3 existing packages in more depth though, to figure out what would best produce a codebase that will be easy to maintain and easiest for new contributors to understand.

And thank you also for @lwasser for your reply and additional context. I am planning to participate in the community meeting this week. Maybe we can discuss a little then.

The main thing would be for the package to work with Figshare's current API, v2.
On Twitter, Figshare told me to reach out to them by email, so let me do that too.
I'm not sure right now if I can even get a command-line tool that will work as easily as I want.