From 0bcd7a9f9a70f328e51549e5c6d99369fcc857e2 Mon Sep 17 00:00:00 2001 From: Stephen Lang <65951795+sketchkey@users.noreply.github.com> Date: Thu, 21 Jul 2022 11:33:41 +0100 Subject: [PATCH 01/20] Update README.Rmd "key" argument missing for get_vessel_info & get_event (doesn't work for me without it) --- README.Rmd | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/README.Rmd b/README.Rmd index 74fc0b5..846b4ed 100644 --- a/README.Rmd +++ b/README.Rmd @@ -182,20 +182,22 @@ As another example, let's combine the Vessels and Events APIs to get fishing eve usa_trawlers <- get_vessel_info( query = "flag = 'USA' AND geartype = 'trawlers'", search_type = "advanced", - dataset = "fishing_vessel" + dataset = "fishing_vessel", + key = key ) # Collapse vessel ids into a commas separated list to pass to Events API -usa_tralwer_ids <- paste0(usa_trawlers$id[1:10], collapse = ',') +usa_trawler_ids <- paste0(usa_trawlers$id[1:10], collapse = ',') ``` Now get the list of fishing events for these trawlers in January, 2020: ```{r example_event_type_4b, eval=T} get_event(event_type='fishing', - vessel = usa_tralwer_ids, + vessel = usa_trawler_ids, start_date = "2020-01-01", - end_date = "2020-02-01" + end_date = "2020-02-01", + key = key ) ``` From e71ec9badce9f2f8cc8cfa4e450299498d0abe44 Mon Sep 17 00:00:00 2001 From: natemiller Date: Wed, 10 Aug 2022 17:35:15 -0700 Subject: [PATCH 02/20] Update LICENSE.md --- LICENSE.md | 395 +++++++++++++++++++++++++++-------------------------- 1 file changed, 201 insertions(+), 194 deletions(-) diff --git a/LICENSE.md b/LICENSE.md index b62a9b5..c1eba75 100644 --- a/LICENSE.md +++ b/LICENSE.md @@ -1,194 +1,201 @@ -Apache License -============== - -_Version 2.0, January 2004_ -_<>_ - -### Terms and Conditions for use, reproduction, and distribution - -#### 1. Definitions - -“License” shall mean the terms and conditions for use, reproduction, and -distribution as defined by Sections 1 through 9 of this document. - -“Licensor” shall mean the copyright owner or entity authorized by the copyright -owner that is granting the License. - -“Legal Entity” shall mean the union of the acting entity and all other entities -that control, are controlled by, or are under common control with that entity. -For the purposes of this definition, “control” means **(i)** the power, direct or -indirect, to cause the direction or management of such entity, whether by -contract or otherwise, or **(ii)** ownership of fifty percent (50%) or more of the -outstanding shares, or **(iii)** beneficial ownership of such entity. - -“You” (or “Your”) shall mean an individual or Legal Entity exercising -permissions granted by this License. - -“Source” form shall mean the preferred form for making modifications, including -but not limited to software source code, documentation source, and configuration -files. - -“Object” form shall mean any form resulting from mechanical transformation or -translation of a Source form, including but not limited to compiled object code, -generated documentation, and conversions to other media types. - -“Work” shall mean the work of authorship, whether in Source or Object form, made -available under the License, as indicated by a copyright notice that is included -in or attached to the work (an example is provided in the Appendix below). - -“Derivative Works” shall mean any work, whether in Source or Object form, that -is based on (or derived from) the Work and for which the editorial revisions, -annotations, elaborations, or other modifications represent, as a whole, an -original work of authorship. For the purposes of this License, Derivative Works -shall not include works that remain separable from, or merely link (or bind by -name) to the interfaces of, the Work and Derivative Works thereof. - -“Contribution” shall mean any work of authorship, including the original version -of the Work and any modifications or additions to that Work or Derivative Works -thereof, that is intentionally submitted to Licensor for inclusion in the Work -by the copyright owner or by an individual or Legal Entity authorized to submit -on behalf of the copyright owner. For the purposes of this definition, -“submitted” means any form of electronic, verbal, or written communication sent -to the Licensor or its representatives, including but not limited to -communication on electronic mailing lists, source code control systems, and -issue tracking systems that are managed by, or on behalf of, the Licensor for -the purpose of discussing and improving the Work, but excluding communication -that is conspicuously marked or otherwise designated in writing by the copyright -owner as “Not a Contribution.” - -“Contributor” shall mean Licensor and any individual or Legal Entity on behalf -of whom a Contribution has been received by Licensor and subsequently -incorporated within the Work. - -#### 2. Grant of Copyright License - -Subject to the terms and conditions of this License, each Contributor hereby -grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, -irrevocable copyright license to reproduce, prepare Derivative Works of, -publicly display, publicly perform, sublicense, and distribute the Work and such -Derivative Works in Source or Object form. - -#### 3. Grant of Patent License - -Subject to the terms and conditions of this License, each Contributor hereby -grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, -irrevocable (except as stated in this section) patent license to make, have -made, use, offer to sell, sell, import, and otherwise transfer the Work, where -such license applies only to those patent claims licensable by such Contributor -that are necessarily infringed by their Contribution(s) alone or by combination -of their Contribution(s) with the Work to which such Contribution(s) was -submitted. If You institute patent litigation against any entity (including a -cross-claim or counterclaim in a lawsuit) alleging that the Work or a -Contribution incorporated within the Work constitutes direct or contributory -patent infringement, then any patent licenses granted to You under this License -for that Work shall terminate as of the date such litigation is filed. - -#### 4. Redistribution - -You may reproduce and distribute copies of the Work or Derivative Works thereof -in any medium, with or without modifications, and in Source or Object form, -provided that You meet the following conditions: - -* **(a)** You must give any other recipients of the Work or Derivative Works a copy of -this License; and -* **(b)** You must cause any modified files to carry prominent notices stating that You -changed the files; and -* **(c)** You must retain, in the Source form of any Derivative Works that You distribute, -all copyright, patent, trademark, and attribution notices from the Source form -of the Work, excluding those notices that do not pertain to any part of the -Derivative Works; and -* **(d)** If the Work includes a “NOTICE” text file as part of its distribution, then any -Derivative Works that You distribute must include a readable copy of the -attribution notices contained within such NOTICE file, excluding those notices -that do not pertain to any part of the Derivative Works, in at least one of the -following places: within a NOTICE text file distributed as part of the -Derivative Works; within the Source form or documentation, if provided along -with the Derivative Works; or, within a display generated by the Derivative -Works, if and wherever such third-party notices normally appear. The contents of -the NOTICE file are for informational purposes only and do not modify the -License. You may add Your own attribution notices within Derivative Works that -You distribute, alongside or as an addendum to the NOTICE text from the Work, -provided that such additional attribution notices cannot be construed as -modifying the License. - -You may add Your own copyright statement to Your modifications and may provide -additional or different license terms and conditions for use, reproduction, or -distribution of Your modifications, or for any such Derivative Works as a whole, -provided Your use, reproduction, and distribution of the Work otherwise complies -with the conditions stated in this License. - -#### 5. Submission of Contributions - -Unless You explicitly state otherwise, any Contribution intentionally submitted -for inclusion in the Work by You to the Licensor shall be under the terms and -conditions of this License, without any additional terms or conditions. -Notwithstanding the above, nothing herein shall supersede or modify the terms of -any separate license agreement you may have executed with Licensor regarding -such Contributions. - -#### 6. Trademarks - -This License does not grant permission to use the trade names, trademarks, -service marks, or product names of the Licensor, except as required for -reasonable and customary use in describing the origin of the Work and -reproducing the content of the NOTICE file. - -#### 7. Disclaimer of Warranty - -Unless required by applicable law or agreed to in writing, Licensor provides the -Work (and each Contributor provides its Contributions) on an “AS IS” BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, -including, without limitation, any warranties or conditions of TITLE, -NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are -solely responsible for determining the appropriateness of using or -redistributing the Work and assume any risks associated with Your exercise of -permissions under this License. - -#### 8. Limitation of Liability - -In no event and under no legal theory, whether in tort (including negligence), -contract, or otherwise, unless required by applicable law (such as deliberate -and grossly negligent acts) or agreed to in writing, shall any Contributor be -liable to You for damages, including any direct, indirect, special, incidental, -or consequential damages of any character arising as a result of this License or -out of the use or inability to use the Work (including but not limited to -damages for loss of goodwill, work stoppage, computer failure or malfunction, or -any and all other commercial damages or losses), even if such Contributor has -been advised of the possibility of such damages. - -#### 9. Accepting Warranty or Additional Liability - -While redistributing the Work or Derivative Works thereof, You may choose to -offer, and charge a fee for, acceptance of support, warranty, indemnity, or -other liability obligations and/or rights consistent with this License. However, -in accepting such obligations, You may act only on Your own behalf and on Your -sole responsibility, not on behalf of any other Contributor, and only if You -agree to indemnify, defend, and hold each Contributor harmless for any liability -incurred by, or claims asserted against, such Contributor by reason of your -accepting any such warranty or additional liability. - -_END OF TERMS AND CONDITIONS_ - -### APPENDIX: How to apply the Apache License to your work - -To apply the Apache License to your work, attach the following boilerplate -notice, with the fields enclosed by brackets `[]` replaced with your own -identifying information. (Don't include the brackets!) The text should be -enclosed in the appropriate comment syntax for the file format. We also -recommend that a file or class name and description of purpose be included on -the same “printed page” as the copyright notice for easier identification within -third-party archives. - - Copyright [yyyy] [name of copyright owner] - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright 2022 Global Fishing Watch + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. From e0202bba1e4ab303fc7db234d19ceb4dd00522d9 Mon Sep 17 00:00:00 2001 From: natemiller Date: Wed, 10 Aug 2022 17:55:07 -0700 Subject: [PATCH 03/20] Update LICENSE --- LICENSE | 202 ++++++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 198 insertions(+), 4 deletions(-) diff --git a/LICENSE b/LICENSE index 4fcfc81..3f80a84 100644 --- a/LICENSE +++ b/LICENSE @@ -1,7 +1,201 @@ -Copyright 2022 Global Fishing Watch +Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ -Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION -http://www.apache.org/licenses/LICENSE-2.0 + 1. Definitions. -Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright 2022 Global Fishing Watch + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. From bd50a910750b71b2908ce7e6b29af92cae0f5c21 Mon Sep 17 00:00:00 2001 From: natemiller Date: Wed, 10 Aug 2022 17:56:01 -0700 Subject: [PATCH 04/20] Update LICENSE --- LICENSE | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/LICENSE b/LICENSE index 3f80a84..ea91bd8 100644 --- a/LICENSE +++ b/LICENSE @@ -1,4 +1,4 @@ -Apache License + Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ From 0c3b23f5acf12e379920cfe4b04d1b9f527169f4 Mon Sep 17 00:00:00 2001 From: Tyler Date: Thu, 18 Aug 2022 13:37:57 -0400 Subject: [PATCH 05/20] Updated get_event to return an empty tibble when API zero entries. Can be bound to results that do contain entries. --- R/get_event.R | 75 ++++++++++++++++++++++++++++++++------------------- 1 file changed, 47 insertions(+), 28 deletions(-) diff --git a/R/get_event.R b/R/get_event.R index 4b2c772..6118272 100644 --- a/R/get_event.R +++ b/R/get_event.R @@ -49,38 +49,57 @@ get_event <- function(event_type='port_visit', # Extract all entries from list of responses all_entries <- purrr::map(all_results, purrr::pluck, 'entries') %>% purrr::flatten(.) - # Create progress bar - pb <- progress::progress_bar$new( - format = "Processing events: [:bar] :current/:total (:percent)", - total = length(all_entries) + # Process results if they exist + if(length(all_entries) > 0){ + # Create progress bar + pb <- progress::progress_bar$new( + format = "Processing events: [:bar] :current/:total (:percent)", + total = length(all_entries) ) - # Function to extract each entry to tibble - event_entry <- function(x){ - df_out <- tibble::tibble( - id = x$id, - type = x$type, - start = x$start, - end = x$end, - lat = x$position$lat, - lon = x$position$lon, - regions = list(x$regions), - boundingBox = list(x$boundingBox), - distances = list(x$distances), - vessel = list(x$vessel), - event_info = list(x[length(x)]) + # Function to extract each entry to tibble + event_entry <- function(x){ + df_out <- tibble::tibble( + id = x$id, + type = x$type, + start = x$start, + end = x$end, + lat = x$position$lat, + lon = x$position$lon, + regions = list(x$regions), + boundingBox = list(x$boundingBox), + distances = list(x$distances), + vessel = list(x$vessel), + event_info = list(x[length(x)]) + ) + # Iterate progress bar + pb$tick() + Sys.sleep(0.01) + # return data + return(df_out) + } + + # Map function to each event to convert to data frame + event_df <- purrr::map_dfr(all_entries, event_entry) %>% + dplyr::mutate(dplyr::across(c(.data$start, .data$end), make_datetime)) + + } else { + # Create empty dataframe to return when zero entries. + event_df <- tibble::tibble( + id = NA_character_, + type = NA_character_, + start = NA, + end = NA, + lat = NA_real_, + lon = NA_real_, + regions = NA, + boundingBox = NA, + distances = NA, + vessel = NA, + event_info = NA ) - # Iterate progress bar - pb$tick() - Sys.sleep(0.01) - # return data - return(df_out) } - # Map function to each event to convert to data frame - event_df <- purrr::map_dfr(all_entries, event_entry) %>% - dplyr::mutate(dplyr::across(c(.data$start, .data$end), make_datetime)) - - # Return final data frame + # Return results return(event_df) } From a4850a5ce29a52639f89318368c3d110905c9277 Mon Sep 17 00:00:00 2001 From: Tyler Date: Wed, 24 Aug 2022 17:17:31 -0400 Subject: [PATCH 06/20] Made get_event MUCH faster. Removed progress bar and added a bit more documentation. --- R/get_event.R | 47 ++++++++++++++++------------------------ README.Rmd | 1 + README.md | 60 +++++++++++++++++++++++++++------------------------ 3 files changed, 52 insertions(+), 56 deletions(-) diff --git a/R/get_event.R b/R/get_event.R index 6118272..b8eef9a 100644 --- a/R/get_event.R +++ b/R/get_event.R @@ -6,6 +6,9 @@ #' @param include_regions Whether to include regions? Ask engineering if this can always be false #' @param start_date Start of date range to search events #' @param end_date End of date range to search events +#' @param confidences Confidence levels (1-4) of events (port visits only). +#' @param limit Limit of response size for each GFW API call. +#' @param offset Internal parameter to GFW pagination #' @param key Authorization token. Can be obtained with gfw_auth function #' @importFrom dplyr across #' @importFrom dplyr mutate @@ -26,7 +29,7 @@ get_event <- function(event_type='port_visit', start_date = NULL, end_date = NULL, confidences = NULL, - limit = 10000, + limit = 99999, offset = 0, key = gfw_auth() ){ @@ -51,36 +54,24 @@ get_event <- function(event_type='port_visit', # Process results if they exist if(length(all_entries) > 0){ - # Create progress bar - pb <- progress::progress_bar$new( - format = "Processing events: [:bar] :current/:total (:percent)", - total = length(all_entries) - ) - # Function to extract each entry to tibble - event_entry <- function(x){ - df_out <- tibble::tibble( - id = x$id, - type = x$type, - start = x$start, - end = x$end, - lat = x$position$lat, - lon = x$position$lon, - regions = list(x$regions), - boundingBox = list(x$boundingBox), - distances = list(x$distances), - vessel = list(x$vessel), - event_info = list(x[length(x)]) - ) - # Iterate progress bar - pb$tick() - Sys.sleep(0.01) - # return data - return(df_out) - } + # Convert list to dataframe + df_out <- tibble::tibble( + id = purrr::map_chr(all_entries, 'id'), + type = purrr::map_chr(all_entries, 'type'), + start = purrr::map_chr(all_entries, 'start'), + end = purrr::map_chr(all_entries, 'end'), + lat = purrr::map_dbl(purrr::map(all_entries, 'position'), 'lat'), + lon = purrr::map_dbl(purrr::map(all_entries, 'position'), 'lon'), + regions = purrr::map(all_entries, 'regions'), + boundingBox = purrr::map(all_entries, 'boundingBox'), + distances = purrr::map(all_entries, 'distances'), + vessel = purrr::map(all_entries, 'vessel'), + event_info = purrr::map(all_entries, length(all_entries[[1]])) # the event_info is always the last element + ) # Map function to each event to convert to data frame - event_df <- purrr::map_dfr(all_entries, event_entry) %>% + event_df <- df_out %>% dplyr::mutate(dplyr::across(c(.data$start, .data$end), make_datetime)) } else { diff --git a/README.Rmd b/README.Rmd index 846b4ed..145876a 100644 --- a/README.Rmd +++ b/README.Rmd @@ -201,6 +201,7 @@ get_event(event_type='fishing', ) ``` +When no events are available, the `get_event()` function returns an empty tibble. ## Map Visualization API diff --git a/README.md b/README.md index ed6f550..f27c907 100644 --- a/README.md +++ b/README.md @@ -209,34 +209,34 @@ fishing events for a list of 10 USA-flagged trawlers: usa_trawlers <- get_vessel_info( query = "flag = 'USA' AND geartype = 'trawlers'", search_type = "advanced", - dataset = "fishing_vessel" + dataset = "fishing_vessel", + key = key ) # Collapse vessel ids into a commas separated list to pass to Events API -usa_tralwer_ids <- paste0(usa_trawlers$id[1:10], collapse = ',') +usa_trawler_ids <- paste0(usa_trawlers$id[1:10], collapse = ',') ``` Now get the list of fishing events for these trawlers in January, 2020: ``` r get_event(event_type='fishing', - vessel = usa_tralwer_ids, + vessel = usa_trawler_ids, start_date = "2020-01-01", - end_date = "2020-02-01" + end_date = "2020-02-01", + key = key ) -#> [1] "Downloading 5 events from GFW" -#> # A tibble: 5 × 11 -#> id type start end lat lon regions -#> -#> 1 7deeba… fish… 2020-01-07 23:20:56 2020-01-08 02:47:56 28.0 -95.2 -#> 2 46b955… fish… 2020-01-10 04:59:35 2020-01-10 14:00:20 28.1 -95.0 -#> 3 5a5cc9… fish… 2020-01-12 18:25:47 2020-01-13 09:29:53 28.1 -94.0 -#> 4 64ad1a… fish… 2020-01-13 21:52:14 2020-01-14 01:12:14 28.1 -92.6 -#> 5 55cdf9… fish… 2020-01-26 17:41:27 2020-01-26 19:18:22 41.2 -69.1 -#> # … with 4 more variables: boundingBox , distances , vessel , -#> # event_info +#> [1] "Downloading 0 events from GFW" +#> # A tibble: 1 × 11 +#> id type start end lat lon regions boundingBox distances vessel +#> +#> 1 NA NA NA NA NA NA NA NA +#> # … with 1 more variable: event_info ``` +When no events are available, the `get_event()` function returns an +empty tibble. + ## Map Visualization API The `get_raster` function gets a raster from the [4Wings @@ -261,6 +261,7 @@ should specify: Here’s an example where we enter the geojson data manually: ``` r + region_json = '{"geojson":{"type":"Polygon","coordinates":[[[-76.11328125,-26.273714024406416],[-76.201171875,-26.980828590472093],[-76.376953125,-27.527758206861883],[-76.81640625,-28.30438068296276],[-77.255859375,-28.767659105691244],[-77.87109375,-29.152161283318918],[-78.486328125,-29.45873118535532],[-79.189453125,-29.61167011519739],[-79.892578125,-29.6880527498568],[-80.595703125,-29.61167011519739],[-81.5625,-29.382175075145277],[-82.177734375,-29.07537517955835],[-82.705078125,-28.6905876542507],[-83.232421875,-28.071980301779845],[-83.49609375,-27.683528083787756],[-83.759765625,-26.980828590472093],[-83.84765625,-26.35249785815401],[-83.759765625,-25.64152637306576],[-83.583984375,-25.16517336866393],[-83.232421875,-24.447149589730827],[-82.705078125,-23.966175871265037],[-82.177734375,-23.483400654325635],[-81.5625,-23.241346102386117],[-80.859375,-22.998851594142906],[-80.15625,-22.917922936146027],[-79.453125,-22.998851594142906],[-78.662109375,-23.1605633090483],[-78.134765625,-23.40276490540795],[-77.431640625,-23.885837699861995],[-76.9921875,-24.28702686537642],[-76.552734375,-24.846565348219727],[-76.2890625,-25.48295117535531],[-76.11328125,-26.273714024406416]]]}}' get_raster(spatial_resolution = 'low', @@ -278,6 +279,7 @@ If you want raster data from a particular EEZ, you can use the you specify the `region_source` as `'eez'`: ``` r + # use EEZ function to get EEZ code of Cote d'Ivoire code_eez <- get_region_id(region_name = 'CIV', region_source = 'eez', key = key) @@ -294,6 +296,7 @@ You could search for just one word in the name of the EEZ and then decide which one you want: ``` r + (get_region_id(region_name = 'French', region_source = 'eez', key = key)) #> # A tibble: 3 × 3 #> id iso3 label @@ -310,7 +313,7 @@ get_raster(spatial_resolution = 'low', region = 5677, region_source = 'eez', key = key) -#> Rows: 5433 Columns: 5 +#> Rows: 5420 Columns: 5 #> ── Column specification ──────────────────────────────────────────────────────── #> Delimiter: "," #> chr (1): flag @@ -318,26 +321,27 @@ get_raster(spatial_resolution = 'low', #> #> ℹ Use `spec()` to retrieve the full column specification for this data. #> ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message. -#> # A tibble: 5,433 × 5 +#> # A tibble: 5,420 × 5 #> Lat Lon `Time Range` flag `Apparent Fishing hours` #> -#> 1 44.7 -1.4 2021 FRA 1877. -#> 2 44.7 -1.3 2021 FRA 2353. -#> 3 46.7 -3.1 2021 FRA 1423. -#> 4 48.7 -5.8 2021 FRA 460. -#> 5 47.5 -4.3 2021 FRA 1137. -#> 6 47.6 -3.9 2021 FRA 1534. -#> 7 46.4 -2.4 2021 FRA 670. -#> 8 45.3 -2 2021 FRA 463. -#> 9 44.6 -1.5 2021 FRA 1152. -#> 10 46.2 -4.2 2021 ESP 184. -#> # … with 5,423 more rows +#> 1 46.2 -2.8 2021 FRA 778. +#> 2 44.8 -2 2021 ESP 117. +#> 3 45.9 -1.2 2021 FRA 59.2 +#> 4 47.2 -2.7 2021 GBR 76.4 +#> 5 44.6 -2 2021 FRA 708. +#> 6 49.8 -2 2021 GBR 3.57 +#> 7 49.5 -1 2021 FRA 502. +#> 8 48.1 -6.6 2021 FRA 319. +#> 9 48.9 -4.3 2021 FRA 86.5 +#> 10 46.1 -1.8 2021 FRA 393. +#> # … with 5,410 more rows ``` A similar approach can be used to search for a specific Marine Protected Area, in this case the Phoenix Island Protected Area (PIPA) ``` r + # use region id function to get MPA code of Phoenix Island Protected Area code_mpa <- get_region_id(region_name = 'Phoenix', region_source = 'mpa', key = key) From ad1cf7e80722b3faa4543d5353d7b032f53bf293 Mon Sep 17 00:00:00 2001 From: Tyler Date: Thu, 25 Aug 2022 14:11:05 -0400 Subject: [PATCH 07/20] Updated get_event to use pluck instead of double nested map and small tweak to imports in a few functions. --- DESCRIPTION | 4 ++-- NAMESPACE | 7 +++---- R/get_event.R | 15 +++++++-------- R/get_vessel_info.R | 4 ++-- README.Rmd | 6 +++++- README.md | 25 ++++++++++++++++++------- man/get_event.Rd | 8 +++++++- man/get_vessel_info.Rd | 4 +++- man/gfwr-package.Rd | 2 +- 9 files changed, 48 insertions(+), 27 deletions(-) diff --git a/DESCRIPTION b/DESCRIPTION index 924268b..6fd9995 100644 --- a/DESCRIPTION +++ b/DESCRIPTION @@ -3,7 +3,7 @@ Title: Interactions with GFW APIs providing data tibbles Version: 0.0.0.9000 Authors@R: c( person("Tyler", "Clavelle", , "tyler@globalfishingwatch.org", role = c("aut", "cre"), - comment = c(ORCID = "YOUR-ORCID-ID")), + comment = c(ORCID = "0000-0002-2155-2459")), person("Rocío", "Joo", , "rocio.joo@globalfishingwatch.org", role = c("aut"), comment = c(ORCID = "0000-0003-0319-4210")), @@ -12,7 +12,7 @@ Authors@R: c( comment = c(ORCID = "YOUR-ORCID-ID")) ) Description: This package connects to several Global Fishing Watch APIs to get vessel and events information in an R-friendly format. -License: Apache License (>= 2) +License: file LICENSE Encoding: UTF-8 Roxygen: list(markdown = TRUE) RoxygenNote: 7.2.0 diff --git a/NAMESPACE b/NAMESPACE index 0e3b8bd..3ca4da7 100644 --- a/NAMESPACE +++ b/NAMESPACE @@ -22,16 +22,15 @@ importFrom(httr2,resp_body_json) importFrom(httr2,resp_body_raw) importFrom(magrittr,"%>%") importFrom(magrittr,`%>%`) -importFrom(progress,progress_bar) importFrom(purrr,flatten) +importFrom(purrr,map) importFrom(purrr,map_chr) -importFrom(purrr,map_dfr) +importFrom(purrr,map_dbl) importFrom(purrr,pluck) importFrom(readr,read_csv) importFrom(rjson,toJSON) importFrom(rlang,.data) -importFrom(tibble,as_tibble) importFrom(tibble,enframe) +importFrom(tibble,tibble) importFrom(tidyr,unnest_wider) -importFrom(tidyselect,everything) importFrom(utils,unzip) diff --git a/R/get_event.R b/R/get_event.R index b8eef9a..7923d6c 100644 --- a/R/get_event.R +++ b/R/get_event.R @@ -12,14 +12,13 @@ #' @param key Authorization token. Can be obtained with gfw_auth function #' @importFrom dplyr across #' @importFrom dplyr mutate -#' @importFrom httr2 req_perform -#' @importFrom httr2 resp_body_json -#' @importFrom purrr map_dfr +#' @importFrom purrr map_chr +#' @importFrom purrr map_dbl +#' @importFrom purrr map +#' @importFrom purrr pluck #' @importFrom purrr flatten #' @importFrom rlang .data -#' @importFrom tibble as_tibble -#' @importFrom progress progress_bar -#' @importFrom tidyselect everything +#' @importFrom tibble tibble #' #' @export @@ -61,8 +60,8 @@ get_event <- function(event_type='port_visit', type = purrr::map_chr(all_entries, 'type'), start = purrr::map_chr(all_entries, 'start'), end = purrr::map_chr(all_entries, 'end'), - lat = purrr::map_dbl(purrr::map(all_entries, 'position'), 'lat'), - lon = purrr::map_dbl(purrr::map(all_entries, 'position'), 'lon'), + lat = purrr::map_dbl(all_entries, purrr::pluck, 'position','lat'), + lon = purrr::map_dbl(all_entries, purrr::pluck, 'position','lon'), regions = purrr::map(all_entries, 'regions'), boundingBox = purrr::map(all_entries, 'boundingBox'), distances = purrr::map(all_entries, 'distances'), diff --git a/R/get_vessel_info.R b/R/get_vessel_info.R index 9ae0a09..df0acde 100644 --- a/R/get_vessel_info.R +++ b/R/get_vessel_info.R @@ -5,7 +5,7 @@ #' @param dataset identity datasets to search against, default = 'all' #' @param limit max number of entries to return in each API response. All results will #' be returned regardless of limit -#' @param offset +#' @param offset Internal parameter to GFW pagination #' @param key Authorization token. Can be obtained with gfw_auth function #' @importFrom httr2 req_headers @@ -31,7 +31,7 @@ get_vessel_info <- function(query = NULL, search_type = NULL, dataset = "all", - limit = 10000, + limit = 99999, offset = 0, key = gfw_auth()) { endpoint <- get_identity_endpoint( diff --git a/README.Rmd b/README.Rmd index 145876a..6b7fdac 100644 --- a/README.Rmd +++ b/README.Rmd @@ -21,12 +21,16 @@ knitr::opts_chunk$set( [![Licence](https://img.shields.io/badge/license-Apache%202-blue)](https://opensource.org/licenses/Apache-2.0) -The `gfwr` R package is a simple wrapper for the Global Fishing Watch (GFW) [APIs](https://globalfishingwatch.org/our-apis/documentation#introduction). It provides convenient functions to freely pull GFW data directly into R in tidy formats. The package currently works with the following APIs: +The `gfwr` R package is a simple wrapper for the Global Fishing Watch (GFW) [APIs](https://globalfishingwatch.org/our-apis/documentation#introduction). It provides convenient functions to freely pull GFW data directly into R in tidy formats. + +The package currently works with the following APIs: * [Vessels API](https://globalfishingwatch.org/our-apis/documentation#vessels-api): vessel search and identity based on AIS self reported data * [Events API](https://globalfishingwatch.org/our-apis/documentation#events-api): encounters, loitering, port visits and fishing events based on AIS data * [Map Visualization (4Wings API)](https://globalfishingwatch.org/our-apis/documentation#map-visualization-4wings-api): apparent fishing effort based on AIS data +> **Note** +> See the [Terms of Use](https://globalfishingwatch.org/our-apis/documentation#reference-data) page for GFW APIs for information on our API licenses and rate limits. ## Installation diff --git a/README.md b/README.md index f27c907..17c498f 100644 --- a/README.md +++ b/README.md @@ -16,7 +16,9 @@ The `gfwr` R package is a simple wrapper for the Global Fishing Watch (GFW) [APIs](https://globalfishingwatch.org/our-apis/documentation#introduction). It provides convenient functions to freely pull GFW data directly into R -in tidy formats. The package currently works with the following APIs: +in tidy formats. + +The package currently works with the following APIs: - [Vessels API](https://globalfishingwatch.org/our-apis/documentation#vessels-api): @@ -29,6 +31,10 @@ in tidy formats. The package currently works with the following APIs: API)](https://globalfishingwatch.org/our-apis/documentation#map-visualization-4wings-api): apparent fishing effort based on AIS data +> **Note** See the [Terms of +> Use](https://globalfishingwatch.org/our-apis/documentation#reference-data) +> page for GFW APIs for information on our API licenses and rate limits. + ## Installation You can install the development version of `gfwr` like so: @@ -226,12 +232,17 @@ get_event(event_type='fishing', end_date = "2020-02-01", key = key ) -#> [1] "Downloading 0 events from GFW" -#> # A tibble: 1 × 11 -#> id type start end lat lon regions boundingBox distances vessel -#> -#> 1 NA NA NA NA NA NA NA NA -#> # … with 1 more variable: event_info +#> [1] "Downloading 5 events from GFW" +#> # A tibble: 5 × 11 +#> id type start end lat lon regions +#> +#> 1 b7f0e… fish… 2020-01-02 18:16:53 2020-01-02 19:06:52 44.6 -124. +#> 2 b2781… fish… 2020-01-05 02:00:04 2020-01-05 03:06:30 44.5 -124. +#> 3 91724… fish… 2020-01-05 03:57:26 2020-01-05 04:31:55 44.4 -124. +#> 4 3e1e3… fish… 2020-01-16 17:43:11 2020-01-16 18:37:02 44.6 -124. +#> 5 55cdf… fish… 2020-01-26 17:41:27 2020-01-26 19:18:22 41.2 -69.1 +#> # … with 4 more variables: boundingBox , distances , vessel , +#> # event_info ``` When no events are available, the `get_event()` function returns an diff --git a/man/get_event.Rd b/man/get_event.Rd index 14e1fa2..252321e 100644 --- a/man/get_event.Rd +++ b/man/get_event.Rd @@ -11,7 +11,7 @@ get_event( start_date = NULL, end_date = NULL, confidences = NULL, - limit = 10000, + limit = 99999, offset = 0, key = gfw_auth() ) @@ -27,6 +27,12 @@ get_event( \item{end_date}{End of date range to search events} +\item{confidences}{Confidence levels (1-4) of events (port visits only).} + +\item{limit}{Limit of response size for each GFW API call.} + +\item{offset}{Internal parameter to GFW pagination} + \item{key}{Authorization token. Can be obtained with gfw_auth function} } \description{ diff --git a/man/get_vessel_info.Rd b/man/get_vessel_info.Rd index 59e7137..f5424a8 100644 --- a/man/get_vessel_info.Rd +++ b/man/get_vessel_info.Rd @@ -8,7 +8,7 @@ get_vessel_info( query = NULL, search_type = NULL, dataset = "all", - limit = 10000, + limit = 99999, offset = 0, key = gfw_auth() ) @@ -23,6 +23,8 @@ get_vessel_info( \item{limit}{max number of entries to return in each API response. All results will be returned regardless of limit} +\item{offset}{Internal parameter to GFW pagination} + \item{key}{Authorization token. Can be obtained with gfw_auth function} } \description{ diff --git a/man/gfwr-package.Rd b/man/gfwr-package.Rd index b2eb35b..3d4181b 100644 --- a/man/gfwr-package.Rd +++ b/man/gfwr-package.Rd @@ -17,7 +17,7 @@ Useful links: } \author{ -\strong{Maintainer}: Tyler Clavelle \email{tyler@globalfishingwatch.org} (\href{https://orcid.org/YOUR-ORCID-ID}{ORCID}) +\strong{Maintainer}: Tyler Clavelle \email{tyler@globalfishingwatch.org} (\href{https://orcid.org/0000-0002-2155-2459}{ORCID}) Authors: \itemize{ From 58b7bbe9ddc08e7f17db14def2c38a0df9d5a3f5 Mon Sep 17 00:00:00 2001 From: Nate Miller Date: Tue, 30 Aug 2022 10:02:45 -0700 Subject: [PATCH 08/20] get_region_id to use region id and region name --- R/utils.R | 17 +++++++++++++++-- 1 file changed, 15 insertions(+), 2 deletions(-) diff --git a/R/utils.R b/R/utils.R index 250120e..dcbbf0d 100644 --- a/R/utils.R +++ b/R/utils.R @@ -145,14 +145,27 @@ get_region_id <- function(region_name, region_source = 'eez', key) { httr2::resp_body_json(.) %>% dplyr::bind_rows() - if (region_source == 'eez') { + # EEZ names + if (region_source == "eez" & is.character(region_name)) { result %>% dplyr::filter(agrepl(region_name, .$label) | agrepl(paste0('^',region_name), .$iso3)) %>% dplyr::mutate(id = as.numeric(id)) - } else if (region_source == 'mpa') { + } + # EEZ ids + else if (region_source == "eez" & is.numeric(region_name)) { + result %>% + dplyr::filter(id == {{ region_name }}) + } + # MPA names + else if (region_source == "mpa" & is.character(region_name)) { result %>% dplyr::filter(agrepl(region_name, .$label)) %>% dplyr::mutate(id = as.numeric(id)) + } + # MPA ids + else if (region_source == "mpa" & is.numeric(region_name)) { + result %>% + dplyr::filter(id == {{ region_name }}) } else { stop('Enter a valid region source') } From 68e8bb7bc14607d5041f693350e5a8f0bdceeb5b Mon Sep 17 00:00:00 2001 From: Nate Miller Date: Tue, 30 Aug 2022 11:07:56 -0700 Subject: [PATCH 09/20] unimportant documentation layout changes --- man/gist_error_body.Rd | 3 +++ man/make_char.Rd | 3 +++ man/make_datetime.Rd | 3 +++ man/paginate.Rd | 3 +++ 4 files changed, 12 insertions(+) diff --git a/man/gist_error_body.Rd b/man/gist_error_body.Rd index 8a0db2d..b294991 100644 --- a/man/gist_error_body.Rd +++ b/man/gist_error_body.Rd @@ -7,6 +7,9 @@ and display appropriately to user Taken from httr2 docs: https://httr2.r-lib.org/articles/wrapping-apis.html#sending-data} \usage{ gist_error_body(resp) +} +\value{ + } \description{ Helper function to parse error message data diff --git a/man/make_char.Rd b/man/make_char.Rd index 4160e60..da00bb1 100644 --- a/man/make_char.Rd +++ b/man/make_char.Rd @@ -5,6 +5,9 @@ \title{Basic function to make length 1 lists into characters} \usage{ make_char(col) +} +\value{ + } \description{ Basic function to make length 1 lists into characters diff --git a/man/make_datetime.Rd b/man/make_datetime.Rd index 604b8d6..59be83e 100644 --- a/man/make_datetime.Rd +++ b/man/make_datetime.Rd @@ -5,6 +5,9 @@ \title{Helper function to convert datetime responses} \usage{ make_datetime(x) +} +\value{ + } \description{ Helper function to convert datetime responses diff --git a/man/paginate.Rd b/man/paginate.Rd index b9af385..28d469a 100644 --- a/man/paginate.Rd +++ b/man/paginate.Rd @@ -5,6 +5,9 @@ \title{Pagination function for GFW API calls} \usage{ paginate(endpoint, key) +} +\value{ + } \description{ Pagination function for GFW API calls From 482ad22b3e2aff99bb8207915c0d7b40efe8fee8 Mon Sep 17 00:00:00 2001 From: Nate Miller Date: Tue, 30 Aug 2022 11:18:04 -0700 Subject: [PATCH 10/20] error message for missing search type --- R/get_vessel_info.R | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/R/get_vessel_info.R b/R/get_vessel_info.R index df0acde..a10a185 100644 --- a/R/get_vessel_info.R +++ b/R/get_vessel_info.R @@ -34,6 +34,11 @@ get_vessel_info <- function(query = NULL, limit = 99999, offset = 0, key = gfw_auth()) { + + if (!search_type %in% c("basic", "advanced", "id")) { + stop("Please specify 'basic', 'advanced' or 'id' for the argument `search_type`.") + } + endpoint <- get_identity_endpoint( dataset_type = dataset, search_type = search_type, From f7035d0d2e2d6cfebbb691dd5c02eb5eeee5e0a6 Mon Sep 17 00:00:00 2001 From: Nate Miller Date: Tue, 30 Aug 2022 11:43:55 -0700 Subject: [PATCH 11/20] update readme for new get_region_id functionality --- README.Rmd | 23 ++++++++++++++ README.md | 89 +++++++++++++++++++++++++++++++++++++----------------- 2 files changed, 84 insertions(+), 28 deletions(-) diff --git a/README.Rmd b/README.Rmd index 6b7fdac..0f30cdf 100644 --- a/README.Rmd +++ b/README.Rmd @@ -306,3 +306,26 @@ get_raster(spatial_resolution = 'low', key = key) ``` +The `get_region_id` function also works in reverse. If a region id as passed as +a `numeric` to the function as the `region_name`, the corresponding region label or iso3 can be returned. +This is especially useful when events are returned with regions. + +```{r example_region_id, eval=T} +# using same example as above +get_event(event_type = 'fishing', + vessel = usa_trawler_ids, + start_date = "2020-01-01", + end_date = "2020-02-01", + include_regions = TRUE, + key = key + ) %>% + # extract EEZ id code + dplyr::mutate(eez = as.character(purrr::map(purrr::map(regions, pluck, 'eez'), + paste0, collapse = ','))) %>% + dplyr::select(id, type, start, end, lat, lon, eez) %>% + dplyr::rowwise() %>% + dplyr::mutate(eez_name = get_region_id(region_name = as.numeric(eez), + region_source = 'eez', + key = key)$label) +``` + diff --git a/README.md b/README.md index 17c498f..d07422a 100644 --- a/README.md +++ b/README.md @@ -106,12 +106,13 @@ To get information of a vessel with MMSI = 224224000 using all datasets: ``` r get_vessel_info(query = 224224000, search_type = "basic", dataset = "all", key = key) -#> # A tibble: 1 × 16 -#> name callsign firstTransmissionDate flag id imo lastTransmissio… mmsi -#> -#> 1 1 EBSJ 2015-10-13T15:47:16Z ESP 3c99c… 8733… 2019-10-15T12:1… 2242… -#> # … with 8 more variables: msgCount , posCount , shipname , -#> # source , vesselType , years , dataset , score +#> # A tibble: 1 × 17 +#> name callsign firstTransmissionD… flag geartype id imo lastTransmissio… +#> +#> 1 1 EBSJ 2015-10-13T15:47:1… ESP NA 3c99… 8733… 2019-10-15T12:1… +#> # … with 9 more variables: mmsi , msgCount , posCount , +#> # shipname , source , vesselType , years , +#> # dataset , score ``` To combine different fields and do fuzzy matching to search the @@ -232,15 +233,14 @@ get_event(event_type='fishing', end_date = "2020-02-01", key = key ) -#> [1] "Downloading 5 events from GFW" -#> # A tibble: 5 × 11 -#> id type start end lat lon regions -#> -#> 1 b7f0e… fish… 2020-01-02 18:16:53 2020-01-02 19:06:52 44.6 -124. -#> 2 b2781… fish… 2020-01-05 02:00:04 2020-01-05 03:06:30 44.5 -124. -#> 3 91724… fish… 2020-01-05 03:57:26 2020-01-05 04:31:55 44.4 -124. -#> 4 3e1e3… fish… 2020-01-16 17:43:11 2020-01-16 18:37:02 44.6 -124. -#> 5 55cdf… fish… 2020-01-26 17:41:27 2020-01-26 19:18:22 41.2 -69.1 +#> [1] "Downloading 4 events from GFW" +#> # A tibble: 4 × 11 +#> id type start end lat lon regions +#> +#> 1 ed92fa… fish… 2020-01-04 20:52:15 2020-01-05 03:35:32 33.9 -78.3 +#> 2 51a7eb… fish… 2020-01-05 07:34:29 2020-01-05 18:46:05 33.9 -78.3 +#> 3 30e0aa… fish… 2020-01-06 23:03:11 2020-01-07 09:09:33 35.0 -76.0 +#> 4 15eff4… fish… 2020-01-07 11:39:32 2020-01-07 14:03:56 35.0 -76.0 #> # … with 4 more variables: boundingBox , distances , vessel , #> # event_info ``` @@ -324,7 +324,7 @@ get_raster(spatial_resolution = 'low', region = 5677, region_source = 'eez', key = key) -#> Rows: 5420 Columns: 5 +#> Rows: 5421 Columns: 5 #> ── Column specification ──────────────────────────────────────────────────────── #> Delimiter: "," #> chr (1): flag @@ -332,20 +332,20 @@ get_raster(spatial_resolution = 'low', #> #> ℹ Use `spec()` to retrieve the full column specification for this data. #> ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message. -#> # A tibble: 5,420 × 5 +#> # A tibble: 5,421 × 5 #> Lat Lon `Time Range` flag `Apparent Fishing hours` #> -#> 1 46.2 -2.8 2021 FRA 778. -#> 2 44.8 -2 2021 ESP 117. -#> 3 45.9 -1.2 2021 FRA 59.2 -#> 4 47.2 -2.7 2021 GBR 76.4 -#> 5 44.6 -2 2021 FRA 708. -#> 6 49.8 -2 2021 GBR 3.57 -#> 7 49.5 -1 2021 FRA 502. -#> 8 48.1 -6.6 2021 FRA 319. -#> 9 48.9 -4.3 2021 FRA 86.5 -#> 10 46.1 -1.8 2021 FRA 393. -#> # … with 5,410 more rows +#> 1 48.5 -8.1 2021 FRA 153. +#> 2 46.4 -4.1 2021 FRA 132. +#> 3 46.5 -2.9 2021 FRA 1418. +#> 4 46.9 -5.4 2021 ESP 245. +#> 5 47.9 -4.5 2021 FRA 549. +#> 6 48.5 -6.2 2021 FRA 395. +#> 7 47.3 -5.6 2021 FRA 554. +#> 8 46.9 -4.6 2021 FRA 113. +#> 9 45.5 -2.2 2021 FRA 387. +#> 10 44 -2.1 2021 ESP 500. +#> # … with 5,411 more rows ``` A similar approach can be used to search for a specific Marine Protected @@ -379,3 +379,36 @@ get_raster(spatial_resolution = 'low', region_source = 'trfmo', key = key) ``` + +The `get_region_id` function also works in reverse. If a region id as +passed as a `numeric` to the function as the `region_name`, the +corresponding region label or iso3 can be returned. This is especially +useful when events are returned with regions. + +``` r +# using same example as above +get_event(event_type = 'fishing', + vessel = usa_trawler_ids, + start_date = "2020-01-01", + end_date = "2020-02-01", + include_regions = TRUE, + key = key + ) %>% + # extract EEZ id code + dplyr::mutate(eez = as.character(purrr::map(purrr::map(regions, pluck, 'eez'), + paste0, collapse = ','))) %>% + dplyr::select(id, type, start, end, lat, lon, eez) %>% + dplyr::rowwise() %>% + dplyr::mutate(eez_name = get_region_id(region_name = as.numeric(eez), + region_source = 'eez', + key = key)$label) +#> [1] "Downloading 4 events from GFW" +#> # A tibble: 4 × 8 +#> # Rowwise: +#> id type start end lat lon eez eez_name +#> +#> 1 ed92… fish… 2020-01-04 20:52:15 2020-01-05 03:35:32 33.9 -78.3 8456 United … +#> 2 51a7… fish… 2020-01-05 07:34:29 2020-01-05 18:46:05 33.9 -78.3 8456 United … +#> 3 30e0… fish… 2020-01-06 23:03:11 2020-01-07 09:09:33 35.0 -76.0 8456 United … +#> 4 15ef… fish… 2020-01-07 11:39:32 2020-01-07 14:03:56 35.0 -76.0 8456 United … +``` From 8489ec3a81b0c050a7add1712fa8a513ad30ec5c Mon Sep 17 00:00:00 2001 From: Nate Miller Date: Tue, 30 Aug 2022 11:46:34 -0700 Subject: [PATCH 12/20] updated get_region_id documentation --- R/utils.R | 2 +- man/get_region_id.Rd | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/R/utils.R b/R/utils.R index dcbbf0d..25f5a99 100644 --- a/R/utils.R +++ b/R/utils.R @@ -120,7 +120,7 @@ paginate <- function(endpoint, key){ #' Function to pull numeric EEZ code using EEZ name #' @name get_region_id -#' @param region_name string, EEZ/MPA name +#' @param region_name string or numeric, EEZ/MPA name or EEZ/MPA id #' @param region_source string, source of region data ('eez' or 'mpa') #' @param key string, API token #' @export diff --git a/man/get_region_id.Rd b/man/get_region_id.Rd index b11d5d0..8bd3926 100644 --- a/man/get_region_id.Rd +++ b/man/get_region_id.Rd @@ -7,7 +7,7 @@ get_region_id(region_name, region_source = "eez", key) } \arguments{ -\item{region_name}{string, EEZ/MPA name} +\item{region_name}{string or numeric, EEZ/MPA name or EEZ/MPA id} \item{region_source}{string, source of region data ('eez' or 'mpa')} From 35b61618cc81f59390e1d03f4e9922dd01c572b1 Mon Sep 17 00:00:00 2001 From: rociojoo Date: Fri, 2 Sep 2022 15:30:47 -0600 Subject: [PATCH 13/20] typo in README --- README.Rmd | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.Rmd b/README.Rmd index 0f30cdf..0be5f2f 100644 --- a/README.Rmd +++ b/README.Rmd @@ -306,7 +306,7 @@ get_raster(spatial_resolution = 'low', key = key) ``` -The `get_region_id` function also works in reverse. If a region id as passed as +The `get_region_id` function also works in reverse. If a region id is passed as a `numeric` to the function as the `region_name`, the corresponding region label or iso3 can be returned. This is especially useful when events are returned with regions. From 28daa543ccc25cb4cdaeebda1446b280d13386e1 Mon Sep 17 00:00:00 2001 From: rociojoo Date: Fri, 2 Sep 2022 15:51:44 -0600 Subject: [PATCH 14/20] tweaked readme to run it without error, not evaluating GFW_TOKEN --- DESCRIPTION | 2 - README.Rmd | 10 +++- README.md | 102 +++++++++++++++++++++++++++-------------- man/gist_error_body.Rd | 3 -- man/make_char.Rd | 3 -- man/make_datetime.Rd | 3 -- man/paginate.Rd | 3 -- 7 files changed, 76 insertions(+), 50 deletions(-) diff --git a/DESCRIPTION b/DESCRIPTION index 6fd9995..297f5a5 100644 --- a/DESCRIPTION +++ b/DESCRIPTION @@ -25,8 +25,6 @@ Imports: rlang, tibble, tidyr, - tidyselect, - progress, rjson Suggests: glue diff --git a/README.Rmd b/README.Rmd index 0be5f2f..3d63773 100644 --- a/README.Rmd +++ b/README.Rmd @@ -36,7 +36,7 @@ The package currently works with the following APIs: You can install the development version of `gfwr` like so: -``` r +``` {r, eval = FALSE} devtools::install_github("GlobalFishingWatch/gfwr") ``` @@ -67,8 +67,10 @@ So you can do: ```{r, eval = T} key <- gfw_auth() +``` -# or this +or this +```{r, eval = F} key <- Sys.getenv("GFW_TOKEN") ``` @@ -329,3 +331,7 @@ get_event(event_type = 'fishing', key = key)$label) ``` +## Contributing + +We welcome all contributions to improve the package! Please read our +[Contribution Guide](https://github.com/GlobalFishingWatch/gfwr/blob/main/Contributing.md) and reach out! diff --git a/README.md b/README.md index d07422a..854f0b0 100644 --- a/README.md +++ b/README.md @@ -67,8 +67,11 @@ So you can do: ``` r key <- gfw_auth() +``` + +or this -# or this +``` r key <- Sys.getenv("GFW_TOKEN") ``` @@ -107,12 +110,13 @@ To get information of a vessel with MMSI = 224224000 using all datasets: get_vessel_info(query = 224224000, search_type = "basic", dataset = "all", key = key) #> # A tibble: 1 × 17 -#> name callsign firstTransmissionD… flag geartype id imo lastTransmissio… -#> -#> 1 1 EBSJ 2015-10-13T15:47:1… ESP NA 3c99… 8733… 2019-10-15T12:1… -#> # … with 9 more variables: mmsi , msgCount , posCount , -#> # shipname , source , vesselType , years , -#> # dataset , score +#> name callsign first…¹ flag geart…² id imo lastT…³ mmsi msgCo…⁴ posCo…⁵ +#> +#> 1 1 EBSJ 2015-1… ESP NA 3c99… 8733… 2019-1… 2242… 1887249 73677 +#> # … with 6 more variables: shipname , source , vesselType , +#> # years , dataset , score , and abbreviated variable names +#> # ¹​firstTransmissionDate, ²​geartype, ³​lastTransmissionDate, ⁴​msgCount, +#> # ⁵​posCount ``` To combine different fields and do fuzzy matching to search the @@ -233,14 +237,25 @@ get_event(event_type='fishing', end_date = "2020-02-01", key = key ) -#> [1] "Downloading 4 events from GFW" -#> # A tibble: 4 × 11 -#> id type start end lat lon regions -#> -#> 1 ed92fa… fish… 2020-01-04 20:52:15 2020-01-05 03:35:32 33.9 -78.3 -#> 2 51a7eb… fish… 2020-01-05 07:34:29 2020-01-05 18:46:05 33.9 -78.3 -#> 3 30e0aa… fish… 2020-01-06 23:03:11 2020-01-07 09:09:33 35.0 -76.0 -#> 4 15eff4… fish… 2020-01-07 11:39:32 2020-01-07 14:03:56 35.0 -76.0 +#> [1] "Downloading 15 events from GFW" +#> # A tibble: 15 × 11 +#> id type start end lat lon regions +#> +#> 1 5c468… fish… 2020-01-20 17:13:14 2020-01-20 18:30:39 54.9 -165. +#> 2 2a3e3… fish… 2020-01-21 02:23:48 2020-01-21 04:24:07 55.1 -165. +#> 3 bbf8e… fish… 2020-01-21 08:49:06 2020-01-21 10:14:13 54.9 -165. +#> 4 2ba57… fish… 2020-01-21 11:47:03 2020-01-21 21:58:03 54.8 -165. +#> 5 c8210… fish… 2020-01-22 01:44:38 2020-01-22 04:24:17 55.1 -165. +#> 6 1dbec… fish… 2020-01-22 05:58:49 2020-01-22 14:57:17 54.9 -165. +#> 7 3b80c… fish… 2020-01-22 16:51:57 2020-01-22 20:56:09 54.8 -165. +#> 8 7adb2… fish… 2020-01-24 18:50:49 2020-01-24 20:07:03 55.3 -165. +#> 9 e5b9b… fish… 2020-01-24 22:41:16 2020-01-24 23:24:35 55.5 -165. +#> 10 d60ea… fish… 2020-01-25 01:36:28 2020-01-25 02:09:00 55.5 -165. +#> 11 54534… fish… 2020-01-25 03:08:25 2020-01-25 04:10:28 55.5 -165. +#> 12 2447c… fish… 2020-01-25 22:16:21 2020-01-26 01:29:08 55.4 -164. +#> 13 5051a… fish… 2020-01-26 01:58:48 2020-01-26 06:09:33 55.4 -164. +#> 14 9a208… fish… 2020-01-26 09:25:52 2020-01-26 19:13:39 55.2 -164. +#> 15 4a757… fish… 2020-01-26 20:44:01 2020-01-26 22:40:06 55.3 -164. #> # … with 4 more variables: boundingBox , distances , vessel , #> # event_info ``` @@ -335,16 +350,16 @@ get_raster(spatial_resolution = 'low', #> # A tibble: 5,421 × 5 #> Lat Lon `Time Range` flag `Apparent Fishing hours` #> -#> 1 48.5 -8.1 2021 FRA 153. -#> 2 46.4 -4.1 2021 FRA 132. -#> 3 46.5 -2.9 2021 FRA 1418. -#> 4 46.9 -5.4 2021 ESP 245. -#> 5 47.9 -4.5 2021 FRA 549. -#> 6 48.5 -6.2 2021 FRA 395. -#> 7 47.3 -5.6 2021 FRA 554. -#> 8 46.9 -4.6 2021 FRA 113. -#> 9 45.5 -2.2 2021 FRA 387. -#> 10 44 -2.1 2021 ESP 500. +#> 1 48.2 -7.2 2021 FRA 136. +#> 2 47.9 -6.2 2021 FRA 117. +#> 3 46.3 -3.5 2021 FRA 141. +#> 4 47.4 -4.5 2021 FRA 428. +#> 5 47 -5.4 2021 ESP 336. +#> 6 49.3 -4.9 2021 FRA 237. +#> 7 47.5 -5.1 2021 FRA 656. +#> 8 46.3 -4.3 2021 FRA 468. +#> 9 45.7 -2 2021 FRA 120. +#> 10 48.2 -4.5 2021 FRA 381. #> # … with 5,411 more rows ``` @@ -380,7 +395,7 @@ get_raster(spatial_resolution = 'low', key = key) ``` -The `get_region_id` function also works in reverse. If a region id as +The `get_region_id` function also works in reverse. If a region id is passed as a `numeric` to the function as the `region_name`, the corresponding region label or iso3 can be returned. This is especially useful when events are returned with regions. @@ -402,13 +417,32 @@ get_event(event_type = 'fishing', dplyr::mutate(eez_name = get_region_id(region_name = as.numeric(eez), region_source = 'eez', key = key)$label) -#> [1] "Downloading 4 events from GFW" -#> # A tibble: 4 × 8 +#> [1] "Downloading 15 events from GFW" +#> # A tibble: 15 × 8 #> # Rowwise: -#> id type start end lat lon eez eez_name -#> -#> 1 ed92… fish… 2020-01-04 20:52:15 2020-01-05 03:35:32 33.9 -78.3 8456 United … -#> 2 51a7… fish… 2020-01-05 07:34:29 2020-01-05 18:46:05 33.9 -78.3 8456 United … -#> 3 30e0… fish… 2020-01-06 23:03:11 2020-01-07 09:09:33 35.0 -76.0 8456 United … -#> 4 15ef… fish… 2020-01-07 11:39:32 2020-01-07 14:03:56 35.0 -76.0 8456 United … +#> id type start end lat lon eez eez_n…¹ +#> +#> 1 5c46… fish… 2020-01-20 17:13:14 2020-01-20 18:30:39 54.9 -165. 8463 United… +#> 2 2a3e… fish… 2020-01-21 02:23:48 2020-01-21 04:24:07 55.1 -165. 8463 United… +#> 3 bbf8… fish… 2020-01-21 08:49:06 2020-01-21 10:14:13 54.9 -165. 8463 United… +#> 4 2ba5… fish… 2020-01-21 11:47:03 2020-01-21 21:58:03 54.8 -165. 8463 United… +#> 5 c821… fish… 2020-01-22 01:44:38 2020-01-22 04:24:17 55.1 -165. 8463 United… +#> 6 1dbe… fish… 2020-01-22 05:58:49 2020-01-22 14:57:17 54.9 -165. 8463 United… +#> 7 3b80… fish… 2020-01-22 16:51:57 2020-01-22 20:56:09 54.8 -165. 8463 United… +#> 8 7adb… fish… 2020-01-24 18:50:49 2020-01-24 20:07:03 55.3 -165. 8463 United… +#> 9 e5b9… fish… 2020-01-24 22:41:16 2020-01-24 23:24:35 55.5 -165. 8463 United… +#> 10 d60e… fish… 2020-01-25 01:36:28 2020-01-25 02:09:00 55.5 -165. 8463 United… +#> 11 5453… fish… 2020-01-25 03:08:25 2020-01-25 04:10:28 55.5 -165. 8463 United… +#> 12 2447… fish… 2020-01-25 22:16:21 2020-01-26 01:29:08 55.4 -164. 8463 United… +#> 13 5051… fish… 2020-01-26 01:58:48 2020-01-26 06:09:33 55.4 -164. 8463 United… +#> 14 9a20… fish… 2020-01-26 09:25:52 2020-01-26 19:13:39 55.2 -164. 8463 United… +#> 15 4a75… fish… 2020-01-26 20:44:01 2020-01-26 22:40:06 55.3 -164. 8463 United… +#> # … with abbreviated variable name ¹​eez_name ``` + +## Contributing + +We welcome all contributions to improve the package! Please read our +[Contribution +Guide](https://github.com/GlobalFishingWatch/gfwr/blob/main/Contributing.md) +and reach out! diff --git a/man/gist_error_body.Rd b/man/gist_error_body.Rd index b294991..8a0db2d 100644 --- a/man/gist_error_body.Rd +++ b/man/gist_error_body.Rd @@ -7,9 +7,6 @@ and display appropriately to user Taken from httr2 docs: https://httr2.r-lib.org/articles/wrapping-apis.html#sending-data} \usage{ gist_error_body(resp) -} -\value{ - } \description{ Helper function to parse error message data diff --git a/man/make_char.Rd b/man/make_char.Rd index da00bb1..4160e60 100644 --- a/man/make_char.Rd +++ b/man/make_char.Rd @@ -5,9 +5,6 @@ \title{Basic function to make length 1 lists into characters} \usage{ make_char(col) -} -\value{ - } \description{ Basic function to make length 1 lists into characters diff --git a/man/make_datetime.Rd b/man/make_datetime.Rd index 59be83e..604b8d6 100644 --- a/man/make_datetime.Rd +++ b/man/make_datetime.Rd @@ -5,9 +5,6 @@ \title{Helper function to convert datetime responses} \usage{ make_datetime(x) -} -\value{ - } \description{ Helper function to convert datetime responses diff --git a/man/paginate.Rd b/man/paginate.Rd index 28d469a..b9af385 100644 --- a/man/paginate.Rd +++ b/man/paginate.Rd @@ -5,9 +5,6 @@ \title{Pagination function for GFW API calls} \usage{ paginate(endpoint, key) -} -\value{ - } \description{ Pagination function for GFW API calls From fbea80f204fe8dcee5730ac24c82ddccccd32bbb Mon Sep 17 00:00:00 2001 From: Nate Miller Date: Fri, 2 Sep 2022 15:11:59 -0700 Subject: [PATCH 15/20] update documentation for gearType group_by --- R/get_raster.R | 2 +- README.Rmd | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/R/get_raster.R b/R/get_raster.R index 986dc69..4e142f6 100644 --- a/R/get_raster.R +++ b/R/get_raster.R @@ -2,7 +2,7 @@ #' #' @param spatial_resolution raster spatial resolution. Can be "low" = 0.1 degree or "high" = 0.01 degree #' @param temporal_resolution raster temporal resolution. Can be 'daily','monthly','yearly'. -#' @param group_by parameter to group by. Can be 'vessel_id', 'flag', 'geartype', 'flagAndGearType' +#' @param group_by parameter to group by. Can be 'vessel_id', 'flag', 'gearType', 'flagAndGearType' #' @param date_range Start and end of date range for raster #' @param region geojson or GFW region code, shape to filter raster #' @param region_source source of the region ('eez','mpa', 'trfmo' or 'user_json') diff --git a/README.Rmd b/README.Rmd index 3d63773..b567c2f 100644 --- a/README.Rmd +++ b/README.Rmd @@ -216,7 +216,7 @@ response to a data frame. In order to use it, you should specify: * The spatial resolution, which can be `low` (0.1 degree) or `high` (0.01 degree) * The temporal resolution, which can be `daily`, `monthly`, or `yearly`. -* The variable to group by: `vessel_id`, `flag`, `geartype`, or `flagAndGearType` +* The variable to group by: `vessel_id`, `flag`, `gearType`, or `flagAndGearType` * The date range * The `geojson` region or region code (such as an EEZ code) to filter the raster * The source for the specified region (currently, `eez`, `mpa`, or `user_json`) From 60c889867342a578ca1f3ebfa78fb7acda0aafc3 Mon Sep 17 00:00:00 2001 From: Nate Miller Date: Fri, 2 Sep 2022 15:15:07 -0700 Subject: [PATCH 16/20] update documentation and README with gearType --- README.md | 51 +++++++++++++++++++++++------------------------ man/get_raster.Rd | 2 +- 2 files changed, 26 insertions(+), 27 deletions(-) diff --git a/README.md b/README.md index 854f0b0..97ab111 100644 --- a/README.md +++ b/README.md @@ -110,13 +110,12 @@ To get information of a vessel with MMSI = 224224000 using all datasets: get_vessel_info(query = 224224000, search_type = "basic", dataset = "all", key = key) #> # A tibble: 1 × 17 -#> name callsign first…¹ flag geart…² id imo lastT…³ mmsi msgCo…⁴ posCo…⁵ -#> -#> 1 1 EBSJ 2015-1… ESP NA 3c99… 8733… 2019-1… 2242… 1887249 73677 -#> # … with 6 more variables: shipname , source , vesselType , -#> # years , dataset , score , and abbreviated variable names -#> # ¹​firstTransmissionDate, ²​geartype, ³​lastTransmissionDate, ⁴​msgCount, -#> # ⁵​posCount +#> name callsign firstTransmissionD… flag geartype id imo lastTransmissio… +#> +#> 1 1 EBSJ 2015-10-13T15:47:1… ESP NA 3c99… 8733… 2019-10-15T12:1… +#> # … with 9 more variables: mmsi , msgCount , posCount , +#> # shipname , source , vesselType , years , +#> # dataset , score ``` To combine different fields and do fuzzy matching to search the @@ -274,7 +273,7 @@ should specify: (0.01 degree) - The temporal resolution, which can be `daily`, `monthly`, or `yearly`. -- The variable to group by: `vessel_id`, `flag`, `geartype`, or +- The variable to group by: `vessel_id`, `flag`, `gearType`, or `flagAndGearType` - The date range - The `geojson` region or region code (such as an EEZ code) to filter @@ -420,24 +419,24 @@ get_event(event_type = 'fishing', #> [1] "Downloading 15 events from GFW" #> # A tibble: 15 × 8 #> # Rowwise: -#> id type start end lat lon eez eez_n…¹ -#> -#> 1 5c46… fish… 2020-01-20 17:13:14 2020-01-20 18:30:39 54.9 -165. 8463 United… -#> 2 2a3e… fish… 2020-01-21 02:23:48 2020-01-21 04:24:07 55.1 -165. 8463 United… -#> 3 bbf8… fish… 2020-01-21 08:49:06 2020-01-21 10:14:13 54.9 -165. 8463 United… -#> 4 2ba5… fish… 2020-01-21 11:47:03 2020-01-21 21:58:03 54.8 -165. 8463 United… -#> 5 c821… fish… 2020-01-22 01:44:38 2020-01-22 04:24:17 55.1 -165. 8463 United… -#> 6 1dbe… fish… 2020-01-22 05:58:49 2020-01-22 14:57:17 54.9 -165. 8463 United… -#> 7 3b80… fish… 2020-01-22 16:51:57 2020-01-22 20:56:09 54.8 -165. 8463 United… -#> 8 7adb… fish… 2020-01-24 18:50:49 2020-01-24 20:07:03 55.3 -165. 8463 United… -#> 9 e5b9… fish… 2020-01-24 22:41:16 2020-01-24 23:24:35 55.5 -165. 8463 United… -#> 10 d60e… fish… 2020-01-25 01:36:28 2020-01-25 02:09:00 55.5 -165. 8463 United… -#> 11 5453… fish… 2020-01-25 03:08:25 2020-01-25 04:10:28 55.5 -165. 8463 United… -#> 12 2447… fish… 2020-01-25 22:16:21 2020-01-26 01:29:08 55.4 -164. 8463 United… -#> 13 5051… fish… 2020-01-26 01:58:48 2020-01-26 06:09:33 55.4 -164. 8463 United… -#> 14 9a20… fish… 2020-01-26 09:25:52 2020-01-26 19:13:39 55.2 -164. 8463 United… -#> 15 4a75… fish… 2020-01-26 20:44:01 2020-01-26 22:40:06 55.3 -164. 8463 United… -#> # … with abbreviated variable name ¹​eez_name +#> id type start end lat lon eez +#> +#> 1 5c468b6c6e2c… fish… 2020-01-20 17:13:14 2020-01-20 18:30:39 54.9 -165. 8463 +#> 2 2a3e3ff1afda… fish… 2020-01-21 02:23:48 2020-01-21 04:24:07 55.1 -165. 8463 +#> 3 bbf8edd3f514… fish… 2020-01-21 08:49:06 2020-01-21 10:14:13 54.9 -165. 8463 +#> 4 2ba573356a51… fish… 2020-01-21 11:47:03 2020-01-21 21:58:03 54.8 -165. 8463 +#> 5 c821056d62e9… fish… 2020-01-22 01:44:38 2020-01-22 04:24:17 55.1 -165. 8463 +#> 6 1dbec6346493… fish… 2020-01-22 05:58:49 2020-01-22 14:57:17 54.9 -165. 8463 +#> 7 3b80cb45e530… fish… 2020-01-22 16:51:57 2020-01-22 20:56:09 54.8 -165. 8463 +#> 8 7adb26b0b0e5… fish… 2020-01-24 18:50:49 2020-01-24 20:07:03 55.3 -165. 8463 +#> 9 e5b9bb88d956… fish… 2020-01-24 22:41:16 2020-01-24 23:24:35 55.5 -165. 8463 +#> 10 d60ea178a45d… fish… 2020-01-25 01:36:28 2020-01-25 02:09:00 55.5 -165. 8463 +#> 11 54534316651a… fish… 2020-01-25 03:08:25 2020-01-25 04:10:28 55.5 -165. 8463 +#> 12 2447c8bd75ae… fish… 2020-01-25 22:16:21 2020-01-26 01:29:08 55.4 -164. 8463 +#> 13 5051a55b77c1… fish… 2020-01-26 01:58:48 2020-01-26 06:09:33 55.4 -164. 8463 +#> 14 9a2081cdaf9d… fish… 2020-01-26 09:25:52 2020-01-26 19:13:39 55.2 -164. 8463 +#> 15 4a7574a16def… fish… 2020-01-26 20:44:01 2020-01-26 22:40:06 55.3 -164. 8463 +#> # … with 1 more variable: eez_name ``` ## Contributing diff --git a/man/get_raster.Rd b/man/get_raster.Rd index 53ac9d2..74035a7 100644 --- a/man/get_raster.Rd +++ b/man/get_raster.Rd @@ -19,7 +19,7 @@ get_raster( \item{temporal_resolution}{raster temporal resolution. Can be 'daily','monthly','yearly'.} -\item{group_by}{parameter to group by. Can be 'vessel_id', 'flag', 'geartype', 'flagAndGearType'} +\item{group_by}{parameter to group by. Can be 'vessel_id', 'flag', 'gearType', 'flagAndGearType'} \item{date_range}{Start and end of date range for raster} From affa6175bfa49114b576f7389db12fb11b47fade Mon Sep 17 00:00:00 2001 From: Tyler Date: Fri, 9 Sep 2022 16:30:10 -0400 Subject: [PATCH 17/20] Removed limit and offset as function arguments. Slight documentation updates. --- R/get_event.R | 48 ++++++++++++---------- R/get_vessel_info.R | 9 +---- R/utils.R | 2 +- README.Rmd | 23 +++++++---- README.md | 91 +++++++++++++++++++++++++++--------------- man/get_event.Rd | 20 ++++++---- man/get_vessel_info.Rd | 7 ---- 7 files changed, 116 insertions(+), 84 deletions(-) diff --git a/R/get_event.R b/R/get_event.R index 7923d6c..9660708 100644 --- a/R/get_event.R +++ b/R/get_event.R @@ -1,15 +1,14 @@ #' #' Base function to get event from API and convert response to data frame #' -#' @param event_type Type of event to get data of. It can be "port_visit" or "fishing" +#' @param event_type Type of event to get data of. One of "port_visit", "encounter", "loitering, or "fishing" #' @param vessel VesselID. How to get this? #' @param include_regions Whether to include regions? Ask engineering if this can always be false #' @param start_date Start of date range to search events #' @param end_date End of date range to search events #' @param confidences Confidence levels (1-4) of events (port visits only). -#' @param limit Limit of response size for each GFW API call. -#' @param offset Internal parameter to GFW pagination #' @param key Authorization token. Can be obtained with gfw_auth function +#' @param quiet Boolean. Whether to print the number of events returned by the request. #' @importFrom dplyr across #' @importFrom dplyr mutate #' @importFrom purrr map_chr @@ -20,6 +19,13 @@ #' @importFrom rlang .data #' @importFrom tibble tibble #' +#' @details +#' There are currently four possible event types to request for a vessel. However, only likely vessels +#' can have fishing events (`event_type = 'fishing'`). An encounter event, which involves multiple vessels, will +#' be returned for each vessel involved in the encounter. For example, an encounter between a carrier vessel +#' and fishing vessel will be represented as two events, one for each vessel. These events will have identical `id` +#' values that are suffixed by a `.` and a number. +#' #' @export get_event <- function(event_type='port_visit', @@ -28,9 +34,8 @@ get_event <- function(event_type='port_visit', start_date = NULL, end_date = NULL, confidences = NULL, - limit = 99999, - offset = 0, - key = gfw_auth() + key = gfw_auth(), + quiet = FALSE ){ @@ -41,8 +46,8 @@ get_event <- function(event_type='port_visit', `start-date` = start_date, `end-date` = end_date, confidences = confidences, - limit = limit, - offset = offset + limit = 99999, + offset = 0 ) # API call; will paginate if neccessary, otherwise return list with one response @@ -73,21 +78,20 @@ get_event <- function(event_type='port_visit', event_df <- df_out %>% dplyr::mutate(dplyr::across(c(.data$start, .data$end), make_datetime)) + # Print out total events + total <- nrow(event_df) + + if(quiet == FALSE){ + print(paste("Downloading",total,"events from GFW")) + } + } else { - # Create empty dataframe to return when zero entries. - event_df <- tibble::tibble( - id = NA_character_, - type = NA_character_, - start = NA, - end = NA, - lat = NA_real_, - lon = NA_real_, - regions = NA, - boundingBox = NA, - distances = NA, - vessel = NA, - event_info = NA - ) + + if(quiet == FALSE){ + print("Your request returned zero results") + } + + return() } # Return results diff --git a/R/get_vessel_info.R b/R/get_vessel_info.R index df0acde..924737a 100644 --- a/R/get_vessel_info.R +++ b/R/get_vessel_info.R @@ -3,9 +3,6 @@ #' @param query search terms to identify vessel #' @param search_type type of search, may be 'basic','advanced', or 'id' #' @param dataset identity datasets to search against, default = 'all' -#' @param limit max number of entries to return in each API response. All results will -#' be returned regardless of limit -#' @param offset Internal parameter to GFW pagination #' @param key Authorization token. Can be obtained with gfw_auth function #' @importFrom httr2 req_headers @@ -31,15 +28,13 @@ get_vessel_info <- function(query = NULL, search_type = NULL, dataset = "all", - limit = 99999, - offset = 0, key = gfw_auth()) { endpoint <- get_identity_endpoint( dataset_type = dataset, search_type = search_type, query = query, - limit = limit, - offset = offset + limit = 99999, + offset = 0 ) response <- endpoint %>% diff --git a/R/utils.R b/R/utils.R index 250120e..0b9d87c 100644 --- a/R/utils.R +++ b/R/utils.R @@ -86,7 +86,7 @@ paginate <- function(endpoint, key){ # Current page values total <- response$total - print(paste("Downloading",total,"events from GFW")) + # print(paste("Downloading",total,"events from GFW")) next_off <- response$nextOffset diff --git a/README.Rmd b/README.Rmd index 6b7fdac..794b94d 100644 --- a/README.Rmd +++ b/README.Rmd @@ -29,7 +29,7 @@ The package currently works with the following APIs: * [Events API](https://globalfishingwatch.org/our-apis/documentation#events-api): encounters, loitering, port visits and fishing events based on AIS data * [Map Visualization (4Wings API)](https://globalfishingwatch.org/our-apis/documentation#map-visualization-4wings-api): apparent fishing effort based on AIS data -> **Note** +> **Note**: > See the [Terms of Use](https://globalfishingwatch.org/our-apis/documentation#reference-data) page for GFW APIs for information on our API licenses and rate limits. ## Installation @@ -72,8 +72,8 @@ key <- gfw_auth() key <- Sys.getenv("GFW_TOKEN") ``` -> **Note**: -`gfwr` functions are set to use `key = gfw_auth()` by default. +> **Note**: +> `gfwr` functions are set to use `key = gfw_auth()` by default. ## Vessels API @@ -85,8 +85,7 @@ identifies all vessels in the specified dataset that match * `advanced` search allows for the use of fuzzy matching with terms such as LIKE. The `id` search allows the user to search using a GFW vessel * `id` allows the user to specify the `vessel id` (generated by GFW) -> **Note**: -
+> **Note**: > `vessel id` is an internal ID generated by GFW to connect data accross APIs and involves a combination of vessel and tracking data information @@ -119,7 +118,8 @@ get_vessel_info(query = "8c7304226-6c71-edbe-0b63-c246734b3c01", search_type = "id", dataset = "carrier_vessel", key = key) ``` -To specify more than one `vessel id`: +To specify more than one `vessel id`: + > **Note**:
> No spaces or newlines are permitted between the `vessel ids` @@ -205,7 +205,16 @@ get_event(event_type='fishing', ) ``` -When no events are available, the `get_event()` function returns an empty tibble. +When no events are available, the `get_event()` function returns nothing. + +```{r example_event_type_4c, eval=T} +get_event(event_type='fishing', + vessel = usa_trawler_ids, + start_date = "2020-01-01", + end_date = "2020-01-01", + key = key + ) +``` ## Map Visualization API diff --git a/README.md b/README.md index 17c498f..fae2d8d 100644 --- a/README.md +++ b/README.md @@ -31,7 +31,8 @@ The package currently works with the following APIs: API)](https://globalfishingwatch.org/our-apis/documentation#map-visualization-4wings-api): apparent fishing effort based on AIS data -> **Note** See the [Terms of +> **Note**: +> See the [Terms of > Use](https://globalfishingwatch.org/our-apis/documentation#reference-data) > page for GFW APIs for information on our API licenses and rate limits. @@ -72,8 +73,8 @@ key <- gfw_auth() key <- Sys.getenv("GFW_TOKEN") ``` -> **Note**: `gfwr` functions are set to use `key = gfw_auth()` by -> default. +> **Note**: +> `gfwr` functions are set to use `key = gfw_auth()` by default. ## Vessels API @@ -90,9 +91,10 @@ There are three search types: `basic`, `advanced`, and `id`. vessel - `id` allows the user to specify the `vessel id` (generated by GFW) -> **Note**:
`vessel id` is an internal ID generated by GFW to -> connect data accross APIs and involves a combination of vessel and -> tracking data information +> **Note**: +> `vessel id` is an internal ID generated by GFW to connect data accross +> APIs and involves a combination of vessel and tracking data +> information The user can also specify which identity databases to use: `carrier_vessel`, `support_vessel`, `fishing_vessel`, or `all`. With the @@ -129,8 +131,10 @@ get_vessel_info(query = "8c7304226-6c71-edbe-0b63-c246734b3c01", search_type = "id", dataset = "carrier_vessel", key = key) ``` -To specify more than one `vessel id`: \> **Note**:
\> No spaces or -newlines are permitted between the `vessel ids` +To specify more than one `vessel id`: + +> **Note**:
No spaces or newlines are permitted between the +> `vessel ids` ``` r get_vessel_info(query = @@ -232,21 +236,44 @@ get_event(event_type='fishing', end_date = "2020-02-01", key = key ) -#> [1] "Downloading 5 events from GFW" -#> # A tibble: 5 × 11 -#> id type start end lat lon regions -#> -#> 1 b7f0e… fish… 2020-01-02 18:16:53 2020-01-02 19:06:52 44.6 -124. -#> 2 b2781… fish… 2020-01-05 02:00:04 2020-01-05 03:06:30 44.5 -124. -#> 3 91724… fish… 2020-01-05 03:57:26 2020-01-05 04:31:55 44.4 -124. -#> 4 3e1e3… fish… 2020-01-16 17:43:11 2020-01-16 18:37:02 44.6 -124. -#> 5 55cdf… fish… 2020-01-26 17:41:27 2020-01-26 19:18:22 41.2 -69.1 +#> [1] "Downloading 17 events from GFW" +#> # A tibble: 17 × 11 +#> id type start end lat lon regions +#> +#> 1 b7f0… fish… 2020-01-02 18:16:53 2020-01-02 19:06:52 44.6 -124. +#> 2 466f… fish… 2020-01-05 00:15:23 2020-01-05 02:00:53 26.3 -96.8 +#> 3 b278… fish… 2020-01-05 02:00:04 2020-01-05 03:06:30 44.5 -124. +#> 4 9172… fish… 2020-01-05 03:57:26 2020-01-05 04:31:55 44.4 -124. +#> 5 f46a… fish… 2020-01-05 11:35:49 2020-01-05 14:35:14 26.3 -96.8 +#> 6 c20f… fish… 2020-01-05 23:54:07 2020-01-06 01:02:35 26.2 -96.8 +#> 7 7749… fish… 2020-01-06 05:04:07 2020-01-06 06:19:03 26.3 -96.8 +#> 8 3e1e… fish… 2020-01-16 17:43:11 2020-01-16 18:37:02 44.6 -124. +#> 9 70be… fish… 2020-01-19 23:02:51 2020-01-20 01:14:22 27.6 -97.1 +#> 10 852e… fish… 2020-01-20 14:05:53 2020-01-20 14:47:52 27.6 -97.0 +#> 11 df4c… fish… 2020-01-23 10:51:40 2020-01-23 11:55:41 26.2 -96.8 +#> 12 2e9f… fish… 2020-01-23 22:12:16 2020-01-24 03:46:48 26.4 -96.8 +#> 13 1709… fish… 2020-01-26 11:00:19 2020-01-26 13:31:50 26.3 -96.8 +#> 14 8ff4… fish… 2020-01-27 04:38:31 2020-01-27 11:04:05 26.3 -96.8 +#> 15 f907… fish… 2020-01-27 13:57:34 2020-01-27 15:04:28 26.3 -96.8 +#> 16 f86b… fish… 2020-01-28 00:15:41 2020-01-28 09:00:41 26.3 -96.8 +#> 17 b847… fish… 2020-01-28 22:27:08 2020-01-29 04:15:08 26.3 -96.8 #> # … with 4 more variables: boundingBox , distances , vessel , #> # event_info ``` -When no events are available, the `get_event()` function returns an -empty tibble. +When no events are available, the `get_event()` function returns +nothing. + +``` r +get_event(event_type='fishing', + vessel = usa_trawler_ids, + start_date = "2020-01-01", + end_date = "2020-01-01", + key = key + ) +#> [1] "Your request returned zero results" +#> NULL +``` ## Map Visualization API @@ -324,7 +351,7 @@ get_raster(spatial_resolution = 'low', region = 5677, region_source = 'eez', key = key) -#> Rows: 5420 Columns: 5 +#> Rows: 5421 Columns: 5 #> ── Column specification ──────────────────────────────────────────────────────── #> Delimiter: "," #> chr (1): flag @@ -332,20 +359,20 @@ get_raster(spatial_resolution = 'low', #> #> ℹ Use `spec()` to retrieve the full column specification for this data. #> ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message. -#> # A tibble: 5,420 × 5 +#> # A tibble: 5,421 × 5 #> Lat Lon `Time Range` flag `Apparent Fishing hours` #> -#> 1 46.2 -2.8 2021 FRA 778. -#> 2 44.8 -2 2021 ESP 117. -#> 3 45.9 -1.2 2021 FRA 59.2 -#> 4 47.2 -2.7 2021 GBR 76.4 -#> 5 44.6 -2 2021 FRA 708. -#> 6 49.8 -2 2021 GBR 3.57 -#> 7 49.5 -1 2021 FRA 502. -#> 8 48.1 -6.6 2021 FRA 319. -#> 9 48.9 -4.3 2021 FRA 86.5 -#> 10 46.1 -1.8 2021 FRA 393. -#> # … with 5,410 more rows +#> 1 47.4 -4.2 2021 FRA 555. +#> 2 44.7 -1.4 2021 FRA 1877. +#> 3 49.7 -0.9 2021 FRA 906. +#> 4 44.6 -1.5 2021 FRA 1152. +#> 5 49.9 -0.4 2021 FRA 537. +#> 6 45.1 -1.4 2021 FRA 555. +#> 7 46.7 -3.8 2021 FRA 94.1 +#> 8 49.2 -3 2021 FRA 84.3 +#> 9 47.8 -6.4 2021 FRA 156. +#> 10 45.9 -1.4 2021 FRA 704. +#> # … with 5,411 more rows ``` A similar approach can be used to search for a specific Marine Protected diff --git a/man/get_event.Rd b/man/get_event.Rd index 252321e..22fac9b 100644 --- a/man/get_event.Rd +++ b/man/get_event.Rd @@ -11,13 +11,12 @@ get_event( start_date = NULL, end_date = NULL, confidences = NULL, - limit = 99999, - offset = 0, - key = gfw_auth() + key = gfw_auth(), + quiet = FALSE ) } \arguments{ -\item{event_type}{Type of event to get data of. It can be "port_visit" or "fishing"} +\item{event_type}{Type of event to get data of. One of "port_visit", "encounter", "loitering, or "fishing"} \item{vessel}{VesselID. How to get this?} @@ -29,12 +28,17 @@ get_event( \item{confidences}{Confidence levels (1-4) of events (port visits only).} -\item{limit}{Limit of response size for each GFW API call.} - -\item{offset}{Internal parameter to GFW pagination} - \item{key}{Authorization token. Can be obtained with gfw_auth function} + +\item{quiet}{Boolean. Whether to print the number of events returned by the request.} } \description{ Base function to get event from API and convert response to data frame } +\details{ +There are currently four possible event types to request for a vessel. However, only likely vessels +can have fishing events (\code{event_type = 'fishing'}). An encounter event, which involves multiple vessels, will +be returned for each vessel involved in the encounter. For example, an encounter between a carrier vessel +and fishing vessel will be represented as two events, one for each vessel. These events will have identical \code{id} +values that are suffixed by a \code{.} and a number. +} diff --git a/man/get_vessel_info.Rd b/man/get_vessel_info.Rd index f5424a8..86fb071 100644 --- a/man/get_vessel_info.Rd +++ b/man/get_vessel_info.Rd @@ -8,8 +8,6 @@ get_vessel_info( query = NULL, search_type = NULL, dataset = "all", - limit = 99999, - offset = 0, key = gfw_auth() ) } @@ -20,11 +18,6 @@ get_vessel_info( \item{dataset}{identity datasets to search against, default = 'all'} -\item{limit}{max number of entries to return in each API response. All results will -be returned regardless of limit} - -\item{offset}{Internal parameter to GFW pagination} - \item{key}{Authorization token. Can be obtained with gfw_auth function} } \description{ From 18dc2229347796ed34e92e190585cc627f889f3c Mon Sep 17 00:00:00 2001 From: Tyler Date: Fri, 16 Sep 2022 14:28:57 -0400 Subject: [PATCH 18/20] Updated documentation for get_event() function. --- R/get_event.R | 18 +++++++++++++----- man/get_event.Rd | 18 +++++++++++++----- 2 files changed, 26 insertions(+), 10 deletions(-) diff --git a/R/get_event.R b/R/get_event.R index 9660708..c43b7e6 100644 --- a/R/get_event.R +++ b/R/get_event.R @@ -20,11 +20,19 @@ #' @importFrom tibble tibble #' #' @details -#' There are currently four possible event types to request for a vessel. However, only likely vessels -#' can have fishing events (`event_type = 'fishing'`). An encounter event, which involves multiple vessels, will -#' be returned for each vessel involved in the encounter. For example, an encounter between a carrier vessel -#' and fishing vessel will be represented as two events, one for each vessel. These events will have identical `id` -#' values that are suffixed by a `.` and a number. +#' There are currently four available event types and these events are provide for three +#' vessel types - fishing, carrier, and support vessels. +#' Fishing events (`event_type = 'fishing'`) are specific to fishing vessels and loitering +#' events (`event_type = 'loitering'`) are specific to carrier vessels. Port visits +#' (`event_type = 'port_visit'`) and encounters (`event_type = 'encounter'`) are available +#' for all vessel types. For more details about the various event types, see the [GFW API documentation](https://globalfishingwatch.org/our-apis/documentation#data-caveat). +#' +#' Encounter events involve multiple vessels and one row is returned for each vessel involved in an encounter. +#' For example, an encounter between a carrier and fishing vessel (`carrier-fishing`) will have one +#' row for the fishing vessel and one for the carrier vessel. The `id` field for encounter events has +#' two components separated by a `.`. The first component is the unique id for the encounter event and will be +#' the same for all vessels involved in the encounter. The second component is an integer used to distinguish between +#' different vessels in the encounter. #' #' @export diff --git a/man/get_event.Rd b/man/get_event.Rd index 22fac9b..0cf3b10 100644 --- a/man/get_event.Rd +++ b/man/get_event.Rd @@ -36,9 +36,17 @@ get_event( Base function to get event from API and convert response to data frame } \details{ -There are currently four possible event types to request for a vessel. However, only likely vessels -can have fishing events (\code{event_type = 'fishing'}). An encounter event, which involves multiple vessels, will -be returned for each vessel involved in the encounter. For example, an encounter between a carrier vessel -and fishing vessel will be represented as two events, one for each vessel. These events will have identical \code{id} -values that are suffixed by a \code{.} and a number. +There are currently four available event types and these events are provide for three +vessel types - fishing, carrier, and support vessels. +Fishing events (\code{event_type = 'fishing'}) are specific to fishing vessels and loitering +events (\code{event_type = 'loitering'}) are specific to carrier vessels. Port visits +(\code{event_type = 'port_visit'}) and encounters (\code{event_type = 'encounter'}) are available +for all vessel types. For more details about the various event types, see the \href{https://globalfishingwatch.org/our-apis/documentation#data-caveat}{GFW API documentation}. + +Encounter events involve multiple vessels and one row is returned for each vessel involved in an encounter. +For example, an encounter between a carrier and fishing vessel (\code{carrier-fishing}) will have one +row for the fishing vessel and one for the carrier vessel. The \code{id} field for encounter events has +two components separated by a \code{.}. The first component is the unique id for the encounter event and will be +the same for all vessels involved in the encounter. The second component is an integer used to distinguish between +different vessels in the encounter. } From 0b45fd0d0074cc3fc4a48ae6f1dbf38e4c125cb3 Mon Sep 17 00:00:00 2001 From: Tyler Date: Fri, 16 Sep 2022 14:34:50 -0400 Subject: [PATCH 19/20] Rendered README to test branch. --- README.Rmd | 2 +- README.md | 75 +++++++++++++++--------------------------------------- 2 files changed, 22 insertions(+), 55 deletions(-) diff --git a/README.Rmd b/README.Rmd index d3bd2cc..f8335b1 100644 --- a/README.Rmd +++ b/README.Rmd @@ -136,7 +136,7 @@ get_vessel_info(query = ## Events API The `get_event` function allows you to get data on specific vessel activities from the [GFW Events API](https://globalfishingwatch.org/our-apis/documentation#events-api). Event types include: -apparent fishing events, potential transshipment events (two vessel encounters and loitering by refrigerated carrier vessels), and port visits. Find more information in our [caveat documentation](https://globalfishingwatch.org/our-apis/documentation#data-caveat). +apparent fishing events, potential transshipment events (two-vessel encounters and loitering by refrigerated carrier vessels), and port visits. Find more information in our [caveat documentation](https://globalfishingwatch.org/our-apis/documentation#data-caveat). ### Examples diff --git a/README.md b/README.md index 347b98c..f46ffdb 100644 --- a/README.md +++ b/README.md @@ -151,7 +151,7 @@ The `get_event` function allows you to get data on specific vessel activities from the [GFW Events API](https://globalfishingwatch.org/our-apis/documentation#events-api). Event types include: apparent fishing events, potential transshipment -events (two vessel encounters and loitering by refrigerated carrier +events (two-vessel encounters and loitering by refrigerated carrier vessels), and port visits. Find more information in our [caveat documentation](https://globalfishingwatch.org/our-apis/documentation#data-caveat). @@ -239,27 +239,11 @@ get_event(event_type='fishing', end_date = "2020-02-01", key = key ) -#> [1] "Downloading 17 events from GFW" -#> # A tibble: 17 × 11 -#> id type start end lat lon regions -#> -#> 1 b7f0… fish… 2020-01-02 18:16:53 2020-01-02 19:06:52 44.6 -124. -#> 2 466f… fish… 2020-01-05 00:15:23 2020-01-05 02:00:53 26.3 -96.8 -#> 3 b278… fish… 2020-01-05 02:00:04 2020-01-05 03:06:30 44.5 -124. -#> 4 9172… fish… 2020-01-05 03:57:26 2020-01-05 04:31:55 44.4 -124. -#> 5 f46a… fish… 2020-01-05 11:35:49 2020-01-05 14:35:14 26.3 -96.8 -#> 6 c20f… fish… 2020-01-05 23:54:07 2020-01-06 01:02:35 26.2 -96.8 -#> 7 7749… fish… 2020-01-06 05:04:07 2020-01-06 06:19:03 26.3 -96.8 -#> 8 3e1e… fish… 2020-01-16 17:43:11 2020-01-16 18:37:02 44.6 -124. -#> 9 70be… fish… 2020-01-19 23:02:51 2020-01-20 01:14:22 27.6 -97.1 -#> 10 852e… fish… 2020-01-20 14:05:53 2020-01-20 14:47:52 27.6 -97.0 -#> 11 df4c… fish… 2020-01-23 10:51:40 2020-01-23 11:55:41 26.2 -96.8 -#> 12 2e9f… fish… 2020-01-23 22:12:16 2020-01-24 03:46:48 26.4 -96.8 -#> 13 1709… fish… 2020-01-26 11:00:19 2020-01-26 13:31:50 26.3 -96.8 -#> 14 8ff4… fish… 2020-01-27 04:38:31 2020-01-27 11:04:05 26.3 -96.8 -#> 15 f907… fish… 2020-01-27 13:57:34 2020-01-27 15:04:28 26.3 -96.8 -#> 16 f86b… fish… 2020-01-28 00:15:41 2020-01-28 09:00:41 26.3 -96.8 -#> 17 b847… fish… 2020-01-28 22:27:08 2020-01-29 04:15:08 26.3 -96.8 +#> [1] "Downloading 1 events from GFW" +#> # A tibble: 1 × 11 +#> id type start end lat lon regions +#> +#> 1 009dfd… fish… 2020-01-24 01:00:19 2020-01-24 01:29:19 29.6 -93.8 #> # … with 4 more variables: boundingBox , distances , vessel , #> # event_info ``` @@ -365,16 +349,16 @@ get_raster(spatial_resolution = 'low', #> # A tibble: 5,421 × 5 #> Lat Lon `Time Range` flag `Apparent Fishing hours` #> -#> 1 47.4 -4.2 2021 FRA 555. -#> 2 44.7 -1.4 2021 FRA 1877. -#> 3 49.7 -0.9 2021 FRA 906. -#> 4 44.6 -1.5 2021 FRA 1152. -#> 5 49.9 -0.4 2021 FRA 537. -#> 6 45.1 -1.4 2021 FRA 555. -#> 7 46.7 -3.8 2021 FRA 94.1 -#> 8 49.2 -3 2021 FRA 84.3 -#> 9 47.8 -6.4 2021 FRA 156. -#> 10 45.9 -1.4 2021 FRA 704. +#> 1 49.6 -1.1 2021 FRA 868. +#> 2 49.8 -0.4 2021 BEL 89.9 +#> 3 49.4 -0.5 2021 FRA 118. +#> 4 49.3 -0.5 2021 FRA 15.6 +#> 5 47.3 -4.6 2021 FRA 167. +#> 6 47.3 -3.7 2021 FRA 995. +#> 7 45.6 -1.7 2021 FRA 1300. +#> 8 45.2 -1.8 2021 FRA 365. +#> 9 47.6 -3.3 2021 GBR 41.0 +#> 10 48.8 -2.3 2021 FRA 560. #> # … with 5,411 more rows ``` @@ -432,29 +416,12 @@ get_event(event_type = 'fishing', dplyr::mutate(eez_name = get_region_id(region_name = as.numeric(eez), region_source = 'eez', key = key)$label) -#> [1] "Downloading 17 events from GFW" -#> # A tibble: 17 × 8 +#> [1] "Downloading 1 events from GFW" +#> # A tibble: 1 × 8 #> # Rowwise: -#> id type start end lat lon eez -#> -#> 1 b7f0e1e9a5e… fish… 2020-01-02 18:16:53 2020-01-02 19:06:52 44.6 -124. 8456 -#> 2 466f42d8ffd… fish… 2020-01-05 00:15:23 2020-01-05 02:00:53 26.3 -96.8 8456 -#> 3 b2781be8028… fish… 2020-01-05 02:00:04 2020-01-05 03:06:30 44.5 -124. 8456 -#> 4 917245e7361… fish… 2020-01-05 03:57:26 2020-01-05 04:31:55 44.4 -124. 8456 -#> 5 f46ac6f9f23… fish… 2020-01-05 11:35:49 2020-01-05 14:35:14 26.3 -96.8 8456 -#> 6 c20f37b0381… fish… 2020-01-05 23:54:07 2020-01-06 01:02:35 26.2 -96.8 8456 -#> 7 7749235a84b… fish… 2020-01-06 05:04:07 2020-01-06 06:19:03 26.3 -96.8 8456 -#> 8 3e1e3181134… fish… 2020-01-16 17:43:11 2020-01-16 18:37:02 44.6 -124. 8456 -#> 9 70beeba723a… fish… 2020-01-19 23:02:51 2020-01-20 01:14:22 27.6 -97.1 8456 -#> 10 852e13cf11b… fish… 2020-01-20 14:05:53 2020-01-20 14:47:52 27.6 -97.0 8456 -#> 11 df4c364cc72… fish… 2020-01-23 10:51:40 2020-01-23 11:55:41 26.2 -96.8 8456 -#> 12 2e9fe27318b… fish… 2020-01-23 22:12:16 2020-01-24 03:46:48 26.4 -96.8 8456 -#> 13 170990893a9… fish… 2020-01-26 11:00:19 2020-01-26 13:31:50 26.3 -96.8 8456 -#> 14 8ff4e5df81b… fish… 2020-01-27 04:38:31 2020-01-27 11:04:05 26.3 -96.8 8456 -#> 15 f90721709fe… fish… 2020-01-27 13:57:34 2020-01-27 15:04:28 26.3 -96.8 8456 -#> 16 f86b7aff336… fish… 2020-01-28 00:15:41 2020-01-28 09:00:41 26.3 -96.8 8456 -#> 17 b847ad3ef72… fish… 2020-01-28 22:27:08 2020-01-29 04:15:08 26.3 -96.8 8456 -#> # … with 1 more variable: eez_name +#> id type start end lat lon eez eez_name +#> +#> 1 009d… fish… 2020-01-24 01:00:19 2020-01-24 01:29:19 29.6 -93.8 8456 United … ``` ## Contributing From 7a4f28b51198e0fcb34525f4453f3ad7d0ea997c Mon Sep 17 00:00:00 2001 From: Tyler Date: Fri, 30 Sep 2022 10:25:32 -0700 Subject: [PATCH 20/20] Added `NEWS.md` file to track release notes. --- NEWS.md | 25 +++++++++++++++++++++++++ README.Rmd | 6 ++++-- 2 files changed, 29 insertions(+), 2 deletions(-) create mode 100644 NEWS.md diff --git a/NEWS.md b/NEWS.md new file mode 100644 index 0000000..94a7528 --- /dev/null +++ b/NEWS.md @@ -0,0 +1,25 @@ +# gfwr 1.1.0 + +## New features: + +- In addition to your own JSON region, can now pass EEZ or MPA id to `get_raster` function to query specific region + +- `get_region_id` now takes id and returns label. This allows you to get the label for the id values returned by certain endpoints (e.g. `get_event`) + +- Considerable speed increases in `get_event` function + +## Bug fixes: + +- `get_event` prints `"Your request returned zero results"` and returns `NULL` when the API response contains no results instead of causing an error. + +## Other news: + +### Updated documentation + +- `get_raster`: requires `group_by` and appropriate parameter name is `gearType` + +- Added a `NEWS.md` file to track changes to the package. + +# gfwr 1.0.0 + +- Initial release of the `gfwr` package. It includes functions to access three GFW APIs - Vessels API, Events API, and Map Visualization (4Wings) API. diff --git a/README.Rmd b/README.Rmd index f8335b1..10a3225 100644 --- a/README.Rmd +++ b/README.Rmd @@ -101,8 +101,10 @@ This is generally recommended and is the option set by default. To get information of a vessel with MMSI = 224224000 using all datasets: ```{r example_vessel_info_1, eval=T} -get_vessel_info(query = 224224000, search_type = "basic", - dataset = "all", key = key) +get_vessel_info(query = 224224000, + search_type = "basic", + dataset = "all", + key = key) ``` To combine different fields and do fuzzy matching to search the `carrier vessel`