[REVIEW]: BoARIO: A Python package implementing the ARIO indirect economic cost model #6547
Comments
editorialbot
added
Python
review
TeX
Track: 4 (SBCS)
|
Hello humans, I'm @editorialbot, a robot that can help you with some common editorial tasks. For a list of things I can do to help you, just type: For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type: |
|
|
Software report: Commit count by author: |
|
Paper file info: 📄 Wordcount for ✅ The paper includes a |
|
License info: 🟡 License found: |
Review checklist for @mwtConflict of interest
Code of Conduct
General checks
Functionality
Documentation
Software paper
|
Review checklist for @potterzotConflict of interest
Code of Conduct
General checks
Functionality
Documentation
Software paper
|
|
👋 @spjuhel, @mwt, and @potterzot - This is the review thread for the paper. All of our communications will happen here from now on. Please read the "Reviewer instructions & questions" in the first comment above. Both reviewers have checklists at the top of this thread (in that first comment) with the JOSS requirements. As you go over the submission, please check any items that you feel have been satisfied. There are also links to the JOSS reviewer guidelines. The JOSS review is different from most other journals. Our goal is to work with the authors to help them meet our criteria instead of merely passing judgment on the submission. As such, the reviewers are encouraged to submit issues and pull requests on the software repository. When doing so, please mention #6547 so that a link is created to this thread (and I can keep an eye on what is happening). Please also feel free to comment and ask questions on this thread. In my experience, it is better to post comments/questions/suggestions as you come across them instead of waiting until you've reviewed the entire package. We aim for the review process to be completed within about 4-6 weeks but please make a start well ahead of this as JOSS reviews are by their nature iterative and any early feedback you may be able to provide to the author will be very helpful in meeting this schedule. |
|
👋 @spjuhel, @mwt, and @potterzot - Just checking in to see how the review is going. Let me know if you have any questions! |
|
@spjuhel Hi, I am going through this right now and have identified some issues.
For the second point, I think it would be ideal if the quickstart guide contained all of the information that was needed to understand and execute the quickstart example. This means that we should get a rundown of the events api in the quickstart guide with just enough information to do the example. The rest of the docs can go into more details about the options and alternative ways to specify events. Smaller points:
|
|
@mwt Hi, thank you a lot for your remarks, Indeed, the documentation suffered from several problems. I did some redesigning, and it should be much better now (Both API and User guide), no huge change for the User guide, but some restructuring. The rework on the API focused on the event module to bring more clarity to the API, and I also switched to a less cluttered and more tree oriented documentation. In parallel, I should have addressed all your specific points regarding the documentation. See below for more details if you want. I will push these as a minor patch on the main branch today or tomorrow, alongside an addition in the paper.md regarding the state of the field. Am I correct in thinking this should go into the State of need section?
I think this was due to the function being defined only for the Event main class, the documentation now shows in the subclasses as well, and I have added a lot of information on each subclass specificities notably around the differences in instantiation. Hopefully this is clearer now.
I fixed the documentation. I did not have a KeyError on my side, it is possible this was fixed at some point, but I'm unsure... Could you share a minimal example and the version used?
Yes indeed! I removed this.
Sadly this is a typo from the pymrio package, I will probably suggest a PR on their repo at some point.
Yes, I think this came from automated docstring generation I tested at some point and I did not realize it was not in the same format. This should be fixed in the minor patch. |
|
@editorialbot generate pdf |
|
👋 @spjuhel, @mwt, and @potterzot - Just checking in to see how the review is going. Could you provide a brief update to your status here in this thread? |
|
@spjuhel, this is a big improvement. I really like the new documentation. I was running version 0.5.7 and am now on 0.5.9. I no longer get a key error, but the function still fails to converge when I use numerical shares instead of gdp. Perhaps this was fixed though in a version that isn't yet on pypi? Here is my example. For me, there are NaN for every step after 1. # import pymrio for the test MRIO
import pymrio
# import the different classes
from boario.simulation import Simulation # Simulation wraps the model
from boario.extended_models import ARIOPsiModel # The core of the model
from boario.event import EventKapitalRebuild # A class defining a shock on capital
# Load the IOSystem from pymrio
mrio = pymrio.load_test().calc_all()
# Instantiate the model and the simulation
model = ARIOPsiModel(mrio)
sim = Simulation(model,n_temporal_units_to_sim=730)
# Instantiate an event.
ev = EventKapitalRebuild.from_scalar_regions_sectors(
impact=500000,
regions=["reg1"],
sectors=["manufactoring", "mining"],
impact_sectoral_distrib = [0.5, 0.5],
rebuilding_sectors={"construction": 0.55,"manufactoring": 0.45},
rebuilding_factor=1.0,
rebuild_tau=90,
)
# Add the event to the simulation
sim.add_event(ev)
# Launch the simulation
sim.loop(progress=False)
# You should be able to generate a dataframe of
# the production with the following line
df = sim.production_realised
# This allows to normalize production at its initial level
df = df / df.loc[0]
df.loc[:, ("reg1", slice(None))].plot()I think you addressed everything else. The "See Also" links in the function documentation that lead to tutorials is a nice touch. I ran your tests. When I use I think you should raise an error in this case? That would cause the test to fail. Do you know why the impact has size zero in this test case? |
|
@crvernon said:
I'm debugging. My current thought is that the functions were designed to work with a particular set of inputs and is very well-tested with those inputs. I'm trying to understand if there are other inputs and edge cases where it fails. |
|
@mwt Thanks for noticing these issues, For the example, I realize that the shock is just too great on the mining sector with this setup, and that make the model "crash", but thus it should at least raise a proper warning. And a functioning example would be more appropriate.
I realize the test/code is not very well-designed, I don't have a clear method for instantiating from a numpy array, and in the test this case (empty_np, where input is an empty numpy array) goes to the
That would be so helpful, I did look out for these, but testing is not my forte and there's a lot of room for improvements here. I actually plan (long term) to highly simplify the event module, such that:
But these require to redesign technical parts of all modules, so I'm not sure if they should be considered in this review? |
Hello, I haven't gotten far beyond cloning and installing the package but should have time this Friday (May 3). |
|
@potterzot just following up to see when you will have time to start your review? Thanks! |
|
@crvernon my apologies, I will have it posted by Tuesday evening. |
|
Thanks @potterzot! |
|
I was able to install and reproduce the example models as well as successfully run tests. No issues there. I do have several suggestions on the documentation and for the paper. I've also made a PR with some small grammar fixes. I've checked off all of the items in the review checklist, but I think the package would benefit from addressing some of the comments below before publication. Most of these are very minor. This is beyond the scope of the current paper, but given the goal of the project stated in both the README and the paper, it would be useful to have more in depth case studies that demonstrate both how to conduct various real-world analysis of impacts and how to modify the base assumptions and data used in the model. I think this would substantially help with adoption of this software for use in impact modeling by other researchers and impact modelers. A few well-designed examples would both highlight how users should think about measuring impacts of different kinds of shocks as well as how to modify the model parameters to reflect changes. In addition, an example of how to take an IO table and create a pymrio object from it would allow a user who doesn't have access to a prebuild IO model but does have access to their country's make and use tables to build their own underlying model. I know I've wanted something like this to exist for quite some time. In the US there are some efforts to provide an open source IO model to "democratize" the use of impact modeling in policy advocacy (for example, see Tapestry). There are some integrations and events that are referenced in the documentation and paper but that aren't currently working. For example, one of the three possible impact events (EventArbitraryProd) is currently not available because of a critical bug. The paper implies a fully working model that has extensive documentation and case studies, but there aren't case studies or multiple types of shocks described or linked in the documentation or the paper. The paper also states that integration to various other modeling platforms and workflows (e.g. "shock on demand, shock on production, shock on both, shock involving reconstruction or not, etc" suggests shock possibilities that are not described in the documentation. Mostly this could be addressed by toning down the promise of I hope the following comments are helpful. Some of the comments are very minor. Probably they do not all need to be addressed, but I think they all would improve the overall sense of "completeness" of the software and the documentation. Documentation comments
Paper comments
|
|
Thanks a lot for all these precious comments,
I totally agree. I intend to extend the documentation with more detailed examples based on my papers, but these are still in the reviewing process, or even unfinished. Another thing I would like to do is to present examples reproducing previous studies / well-known disasters in the indirect impact literature (such as Hurricane Katrina or the 2001 floods in Thailand). This however requires a substantial amount of work, which I'm not at liberty to do at the moment.
I agree this would be a nice addition, but I feel like this is something that should be part of the pymrio documentation (and already does to some extent).
Yes, I might have been too ambitious when writing some parts of the paper. I discovered only during this review that I just pushed these changed to the |
|
@editorialbot generate pdf |
|
@editorialbot check references |
|
|
Thanks @spjuhel. Please correct the following errors in the paper:
Please make sure all of the references are formatted correctly and no "nil" values exists. Thank you! |
|
@editorialbot generate pdf |
|
Forgot to say, (And thanks a lot for your detailed reviewing effort) |
|
👋 @spjuhel - we are almost there! Next is just setting up the archive for your new release. We want to make sure the archival has the correct metadata that JOSS requires. This includes a title that matches the paper title and a correct author list. So here is what we have left to do:
I can then move forward with accepting the submission. |
|
I think this should be good: https://doi.org/10.5281/zenodo.11580697 |
|
@editorialbot set v0.5.10 as version |
|
Done! version is now v0.5.10 |
|
@editorialbot set 10.5281/zenodo.11580697 as archive |
|
Done! archive is now 10.5281/zenodo.11580697 |
|
🔍 checking out the following:
|
|
@editorialbot recommend-accept |
|
|
|
👋 @openjournals/sbcs-eics, this paper is ready to be accepted and published. Check final proof 👉📄 Download article If the paper PDF and the deposit XML files look good in openjournals/joss-papers#5488, then you can now move forward with accepting the submission by compiling again with the command |
editorialbot
added
the
recommend-accept
|
@editorialbot accept |
|
|
Ensure proper citation by uploading a plain text CITATION.cff file to the default branch of your repository. If using GitHub, a Cite this repository menu will appear in the About section, containing both APA and BibTeX formats. When exported to Zotero using a browser plugin, Zotero will automatically create an entry using the information contained in the .cff file. You can copy the contents for your CITATION.cff file here: CITATION.cff
If the repository is not hosted on GitHub, a .cff file can still be uploaded to set your preferred citation. Users will be able to manually copy and paste the citation. |
|
🐘🐘🐘 👉 Toot for this paper 👈 🐘🐘🐘 |
|
🚨🚨🚨 THIS IS NOT A DRILL, YOU HAVE JUST ACCEPTED A PAPER INTO JOSS! 🚨🚨🚨 Here's what you must now do:
Any issues? Notify your editorial technical team... |
|
🥳 Congratulations on your new publication @spjuhel! Many thanks to @mwt and @potterzot for your time, hard work, and expertise!! JOSS wouldn't be able to function nor succeed without your efforts. Please consider becoming a reviewer for JOSS if you are not already: https://reviewers.joss.theoj.org/join |
|
🎉🎉🎉 Congratulations on your paper acceptance! 🎉🎉🎉 If you would like to include a link to your paper from your README use the following code snippets: This is how it will look in your documentation: We need your help! The Journal of Open Source Software is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following:
|
Submitting author: @spjuhel (Samuel Juhel)
Repository: https://github.com/spjuhel/BoARIO
Branch with paper.md (empty if default branch): main
Version: v0.5.10
Editor: @crvernon
Reviewers: @mwt, @potterzot
Archive: 10.5281/zenodo.11580697
Status
Status badge code:
Reviewers and authors:
Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)
Reviewer instructions & questions
@mwt & @potterzot, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review.
First of all you need to run this command in a separate comment to create the checklist:
The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @crvernon know.
✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨
Checklists
📝 Checklist for @mwt
📝 Checklist for @potterzot
The text was updated successfully, but these errors were encountered: