v2.0.4

What's Changed

• Census Service Annual Survey in #421
• BEA Personal Consumption Expenditures by state in #420
• Census FBA datasets in #427
• Revised state level GHG data for CBEI in #428
• employment updates in #437

New FBAs

• Personal consumption expenditures by state (BEA_PCE)
• BLS Consumer Expenditures Survey (BLS_CES)
• Census American Community Survey (Census_ACS)
• Census County Business Patterns (Census_CBP), revised
• Census Economic Census, Class of Customer Statistics (Census_EC)
• Census Service Annual Survey (Census_SAS)
• State Inventory Tool (EPA_SIT); requires state data
• GHG Inventory data for select states to support EPAs Consumption Based Emissions Inventories
• Updated USDA ERS Farm Income and Wealth Statistics (USDA_ERS_FIWS)
• Updated USDA ERS Farm Income and Wealth Statistics (NOAA_FisheriesLandings)

New FBSs

• State_employment_m1 (added 2021-2023, updated all other years)
• National_employment_m1 (added 2023, updated all other years)

Full Changelog: v2.0.3...v2.0.4

v2.0.3

Flow By Activity

• Updates EPA_GHGI through 2022 (2012-2022) in #406
• New BEA data for 2012-2022 (Summary & Gross Output), 2012 & 2017 Detail (using 2017 BEA schema)
• New CoA data (2022) and updated USGS_MYB_Lead (2020) in #405

Flow By Sector

• BEA_Detail FBS for 2013-2016, 2018-2022
• Updated GHG national FBS (m1 and m2) 2012-2022

Full Changelog: v2.0.2...v2.0.3

v2.0.2

What's Changed

• add source publication dates to FBAs in #275
• option to specify git version/hash when returning an FBA via git_version in #399
• FBS metadata captures sequential FBAs in #399 (resolves #397)
• option to generate FBS that contains activity cols (retain_activity_columns=True) and sector name cols (append_sector_names=True) in #398
• Update 2017-2022 employment FBS in #410
• Updates Energy based datasets in #411
• add national CRHW methods in #414
• Global Materials Database in #415
• Enables calling multiple years at once for generating FBAs using call_all_years: True (#407)
• Allows skipping of standardize_units (#408)
• Limit numpy < 2.0.0 (see #418)

Flow-by-Activity

• Substantial updates to IEA Monthly Energy Review (EIA_MER)
• Adds UNEP Global Materials Flow Database (UNEP_IRP_GMFD)

Flow-by-Sector

• Updates Employment_national to 2017 NAICS schema, and adds 2021 and 2022 (#410)
• Updates CRHW_national to 2017 NAICS schema, and adds 2021 (#414)
• Adds Energy_fossil_national
• Adds Raw_Material_Extraction_national

Full Changelog: v2.0.1...v2.0.2

v2.0.1

• new USEEIOv2 detail target schema
• fix broken FBAs (changed urls, changed excel tab names) for EIA_AEO, NOAA_FisheriesLandings, EIA_SEDS, EPA_CDDPath, USGS_MYB, EPA_WARMer
• reassign USDA_IWMS '111333' NAICS code to Berry Totals (from Orchards) to align with USDA_CoA_Cropland assignment c7c4c4f
• assign USGS_WU_Coef "Beef and other cattle" to "11213" in addition to "11212" a9c01c4
• update method_status.md to reflect current status of FBA/FBS errors when generating
• updates to stackedBarChart() to work in situations where df is already a collapsed FBS and where there are unique input parameters
• updates to FBSscatterplot() - add boxplot option
• update sector_aggregation() to work for collapsed FBS df
• generalize return_primary_activity_column() to return_primary_flow_column() so function works for both FBA and FBS 46ddf48
• new proxy_sector_data() to enable substituting an FBS sector value for a missing sector
• correct zenodo authorship
• update links to new data commons server
• rename "foss_landings.csv" to "NOAA_FisheriesLandings.csv" in external data folder
• update waste sector names, add 2 additional waste sectors 854eab3
• add warning when an FBS method uses "direct" when it should use "equal"
• update Land and Water FBS to use "equal" over "direct"
• edit log statements to make more concise

Changes to FBS

• Water_national_2015_m1, Water_national_2010_m1, Water_state_2015_m1 have new results to do reassigned activity to sector mapping for USGS_IWMS and USGS_WU_Coef

Full Changelog: v2.0.0...v2.0.1

v2.0.0

Major updates:

• Turn FlowByActivity and FlowBySector into classes
• Create FlowBy class for functions used in both FBA and FBS classes
• FBS yamls revised to work for unlimited recursive and sequential attribution methods
• Update how suppressed data and parent sectors are equally attributed to child sectors, by equally attributing parent values to the next level of child values, rather than equally attributing parent values to target-level child sectors
• Ability to attribute dfs on non-sector columns
• Option to fill in primary source data columns with attribution columns
• Add support for 2017 NAICS codes
• Add mappings for 2017 BEA codes to 2017 NAICS
• All state FBS model results sum to national FBS model results

Minor Updates:

• Updates to Paths, which require most up-to-date versions of fedefelmflowlist, esupy, stewi
• Source_catalog.yaml updated - 'activity_schema' can be year-dependent
• New function compare_national_state_fbs() which compares aggregated results of state vs land FBS
• New github action to generate a single FBA
• Change BEA FBA names to align with useeior naming schema
• Updates to BLM FBAs - adds previously dropped state and national data for select activities
• Rename "CAP_HAP_national_2017" to "CAP_HAP_national_2017_m1"
• Add D.C. data to ERS MLU and NWIS WU
• Drop support for Python 3.8, add support for Python 3.11
• Rename sector_aggregation_levels from "aggregated" and "disaggregated" to "flat" and "Parent-completeChild" and "parent-incompleteChild"

New Flow-By-Sector Models

• CAP_HAP_Nonpoint 2014, 2017, 2020
• CAP_HAP_Nonroad 2014, 2017, 2020
• CAP_HAP_Onroad 2014, 2017, 2020
• CAP_HAP_national m1 and m2 2014, 2017, 2020
• CAP_HAP_state m1 2014, 2017, 2020
• CRHW_state 2013, 2015, 2019
• Detail Make, Supply, and Use tables
• Employment national 2002, 2016, 2019, 2020
• GHG national m1 and m2, 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019, 2020
• GHG state m1 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019, 2020
• GRDREL national 2020
• GRDREL state 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019, 2020
• Land_state_2012
• TRI_DMR_state 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019, 2020

New Flow-By-Activity Models

• Annual BEA summary make and use tables
• Bureau of Transportation Statistics Airline fuel cost and consumption (BTS_Airlines) 2000-2021
• EPA State Inventory Tool (EPA_SIT) 2018-2019
• EPA State GHG Inventories (EPA_StateGHGI) 2020
• State GHGI for Maine (1990-2019), New York (2017-2019), Vermont (1990-2019)

Justifications for changes in FBS model results

• All FBS models have revised results
• Updates to how suppressed data are estimated and to how parent sector values are equally attributed to child sectors impacted all FBS models
• CNHW_national_2018, CNHW_state_2014 - changes due to employment FBS; fix error in missing F01000
• Food_waste_national_2018_m2 - changes due to new CNHW 2018 as primary data source
• Water_national_2015_m1 - Use state employment data for attribution instead of national employment data and reassigned an "Orchard" code to "Berry Totals"

Features removed:

• Generating Sankey diagrams (will be re-added in future version)
• Producing .bib files (will be re-added in future version)
• Appending material codes to sector codes (will be re-added in future version)
• Drop support for Water_national_m2 FBS
• Drop support for Electricity_gen_emissions FBS

What's Changed

• Seea recursive by @matthewlchambers in #243
• Adds display_tables() method to FBS by @matthewlchambers in #246
• allocate regional mecs data to states by @bl-young in #225
• update state ghg with develop by @catherinebirney in #257
• wrapped code in a function call so it won't all execute on import by @matthewlchambers in #289
• Override .astype() from DataFrame to fix issues introduced by pandas 1.5.0 by @matthewlchambers in #288
• Removes unused keys from certain FBS methods by @matthewlchambers in #291
• allow cached datasets to undergo futher selection by @bl-young in #294
• Add argument to FB constructor to prevent (when necessary) adding columns by @matthewlchambers in #295
• Pull master into state_ghg branch by @ericmbell1 in #299
• Several small fixes by @matthewlchambers in #312
• Document flowby functions by @catherinebirney in #331
• Update GHGI method to v2.0 by @bl-young in #332
• Update saving of file and metadata by @bl-young in #337
• update recursive-refac with sector_mapping (includes up-to-date develop branch) by @catherinebirney in #330
• drop attributionsources col in substitute_nonexistent_values() by @catherinebirney in #338
• Updates to support state methods on recursive branch by @bl-young in #339
• Seea time series by @matthewlchambers in #341
• expand exclusion_fields by @catherinebirney in #343
• update industry_spec_key() so it is not based on sector string length by @catherinebirney in #342
• Drop deprecated code and revise log handling by @bl-young in #347
• update NEI nonpoint data with 2020 by @bl-young in #348
• minor updates to support CAP_HAP FBS by @bl-young in #349
• specify NAICS year in activity_schema by @catherinebirney in #345
• Refactor with pathlib by @bl-young in #350
• Update cnhw by @catherinebirney in #352
• update recursive-refac with proportional_attribution by @catherinebirney in #354
• incorporate parent-incompleteChild sector hierarchy into map_to_sectors() by @catherinebirney in #355
• Gha error2 by @catherinebirney in #356
• Add action to create an FBA by @bl-young in #357
• update methods to estimate suppressed data by @catherinebirney in #360
• Set FBA columns and column dtypes by @catherinebirney in #362
• Split flowby.py into 3 python scripts by @catherinebirney in #363
• Updates to GHG state method by @bl-young in #365
• New GHG FBS method files by @ysrivas08 in #366
• update develop with recursive-refac (flowsa v2.0) by @catherinebirney in #364
• Bea refac by @catherinebirney in #372
• State ghgi v2 by @catherinebirney in #374
• update develop with changes to harmonize state and national fbs values by @catherinebirney in #375
• update flowsa2.0-release branch with develop by @catherinebirney in #376
• Flowsa2.0 release by @jbousquin in #381
• Update flowsa2.0 with updates from reviewer comments by @catherinebirney in #382
• 2017 Supply and Use tables from useeior by @WesIngwersen in #383
• update develop with Flowsa2.0 release branch by @catherinebirney in #386
• update state ghgi with develop/flowsa2.0-release branch by @catherinebirney in #387
• skip select_by_fields when using activity_sets by @bl-young in #392
• Revised approach to GHG models by @bl-young in #378
• Updates to generate 2014 CAP_HAP_Nonroad without memory errors by @bl-young in #393
• update 2.0-release branch with changes from develop by @catherinebirney in #394
• Flowsa2.0 release by @catherinebirney in #367

New Contributors

• @ysrivas08 made their first contribution in #366
• @jbousquin made their first contribution in #381

v2.0.0 reviewers

Thanks to David Graham and Justin Bousquin for reviewing FLOWSA for the v2.0.0 release.

Full Changelog: v1.3.2...v2.0.0

v1.3.2

Pins final release supported by esupy<=0.2.2

Full Changelog: v1.3.1...v1.3.2

v1.3.1

FLOWSA v1.3.1 release coincides with supply-chain-factors v1.2 release

Greenhouse Gas (GHG) Flow-By-Sector (FBS) Method changes:

• Updates GHG FBS m1 to equally allocate BEA and EIA MECS to sectors rather than use employment for attribution
• Updates GHG FBS m1 for 2016, 2017, 2018, and 2019 with latest inventory
• Use a common GHG FBS m1 yaml file as basis for all years
• Update some GHG FBS attribution source data years (MECS)
• Updates to GHGI activity names and activity to sector mapping

New Flow-By-Activity and Flow-By-Sector datasets:

• Adds 2020 GHGI FBA and GHG national FBS

Additional, minor changes:

• Update stackedbarchart() to use colors defined in visualizationessentials.csv, to include option to specify target sector level and to generalize attribution methods (direct vs attributed)

v1.3.0

Major Updates:

• Option to append material codes to end of sector code in FBS
• New waste-related FBAs/FBSs, and 7/8 digit sector codes
• New food waste and concrete waste specific FBS
• New data visualization functions: Sankey, stackedBarChart
• Updates to methodology since v1.2.4 release changed the results for: CAP_HAP_national_2017, CNHW_national_2014, Land_national_2012, Water_national_2010_m2, Water_national_2015_m2, Water_national_2015_m3
• option to return an FBS with any mix of sector lengths, SPB and SCB columns no longer need to have matching sector lengths
• option for multiplication as allocation method
• update allocation methodology so if the allocation dataset is at a more aggregated geoscale than the primary FBA, that the primary df is not aggregated to match the allocation geoscale. Instead the more aggregated allocation dataset merges with the primary FBA on all related less aggregated geoscales
• Changes to github actions to address memory issues
• in equal_allocation() aggregate column before allocating to child naics - this impacts some NAICS6 results
• option to retain activity names in a primary data source after calling on those activities in an FBS, so the activity names can be used again
• requires plotly, kaleido

Minor Updates:

• new external_paths.env file to store local file paths
• rename load_api_key to load_env_file_key(), used for api keys and external_paths.env
• getFlowByActivity accepts flowclass of class or list
• new sector codes: S00203 - other state and local government enterprises, F040 - exports of goods and services, F050 - imports of goods and services, new 7/8 digit sectors for waste sectors (5622121, 5622191, 5622192, 5629201, 5629202, 5629203)
• New required columns in FBS: ProducedBySectorType, ConsumedBySectorType, AttributionSources (primary attribution source name)
• new selection_fields FBS parameter to subset FBA using column names/values
• new VisualizationEssentials.csv with standard colors for graphing sectors
• new csv "sector_2012_names" which includes additional names beyond official NAICS

New/Modified FBA:

• expand epa_cddpath sector mapping
• EPA_FactsAndFigures, EPA_REI, EPA_WARMer, EPA_WFR, Census_ASM, EPA_CDDPath (2018), EIA_AEO, EIA_SEDS

New/Modified FBS:

• CNHW_national_2018, Employment_national (2012, 2014, 2015), Food_waste_national_2018 (m1, m2), GHG_national_m1 (2016, 2017, 2018, 2019), REI_waste_national_2012

v1.2.4

FBA Updates

• USDA_ERS_FIWS through 2020
• NOAA_FisheriesLandings through 2021
• BLS_QCEW data through 2021

FBS Updates #255

• Move most employment FBS text to Employment_common.yaml
• New Employment_national_2018
• New Employment_state_2020

BEA updates from useeiorv1.1.0 #230

• Updates 2012 Make and Use tables to use latest BEA release in Fall of 2021 (see USEPA/useeior#221)
• Updates industry gross output data from same release, and expands to include 2019 and 2020 data years
• Removed deprecated BEA_2012_Detail_Use_Industry_Transactions.csv
• Changes to BEA data modify results of CAP_HAP_national_2017, Water_national_2010_m2, and Water_national_2015_m2

Other updates

• Correct metadata URLs for FBA and FBS YAMLs #262
  • Old metadata URLs can be corrected by replacing "data" with "methods"

Full Changelog: v1.2.3...v1.2.4

v1.2.3

Flow-By-Activity Updates

1. Fixed broken link for USGS_MYB_Copper #252
2. Add BLS_QCEW 2019 FBA #248
3. Add method_status.yaml to document list of known inactive method files #251
4. New Census_QWI FBAs for 2002, 2010 - 2020 #242

Flow-By-Sector Updates

1. Add state employment FBS for 2018/2019 #248

Requirement Updates

1. Update to pandas 1.4.0 #238
2. Drop support for Python 3.7 (due to pandas update) #237

GitHub Actions

1. Separate actions for FBA config and FBS method testing #251
2. In FBA testing, skip inactive FBA methods identified in method_status.yaml #251

Full Changelog: v1.2.2...v1.2.3