From da35ab5bef9a36020259b0becb6c21f834f40aa2 Mon Sep 17 00:00:00 2001 From: Nick Van der Auwermeulen Date: Fri, 20 Oct 2023 23:44:09 +0000 Subject: [PATCH] [Release] 0.1.3 What's changed: - Created Apps Script to process Google docs, PDF, and Gmail threads. See `apps_script/README.md`. - Added `third_party/g2docsmd-html` which is the base for converting files to markdown. - Updated the main `README.md` for clarity. - Created a `scripts/README.md` to better explain content processing. --- demos/palm/python/docs-agent/README.md | 22 +- .../python/docs-agent/apps_script/README.md | 164 +++ .../apps_script/drive_to_markdown.gs | 193 +++ .../python/docs-agent/apps_script/exportmd.gs | 1308 +++++++++++++++++ .../apps_script/gmail_to_markdown.gs | 137 ++ .../apps_script/helper_functions.gs | 183 +++ .../python/docs-agent/apps_script/main.gs | 27 + demos/palm/python/docs-agent/config.yaml | 2 +- demos/palm/python/docs-agent/poetry.lock | 813 +++++----- demos/palm/python/docs-agent/pyproject.toml | 2 +- .../palm/python/docs-agent/scripts/README.md | 76 + .../scripts/markdown_to_plain_text.py | 14 +- .../scripts/populate_vector_database.py | 4 +- .../third_party/g2docsmd-html/LICENSE | 202 +++ .../third_party/g2docsmd-html/exportmd.gs | 1244 ++++++++++++++++ .../001-initial-changes-for-docs-agent.patch | 153 ++ 16 files changed, 4157 insertions(+), 387 deletions(-) create mode 100644 demos/palm/python/docs-agent/apps_script/README.md create mode 100644 demos/palm/python/docs-agent/apps_script/drive_to_markdown.gs create mode 100644 demos/palm/python/docs-agent/apps_script/exportmd.gs create mode 100644 demos/palm/python/docs-agent/apps_script/gmail_to_markdown.gs create mode 100644 demos/palm/python/docs-agent/apps_script/helper_functions.gs create mode 100644 demos/palm/python/docs-agent/apps_script/main.gs create mode 100644 demos/palm/python/docs-agent/scripts/README.md create mode 100644 demos/palm/python/docs-agent/third_party/g2docsmd-html/LICENSE create mode 100644 demos/palm/python/docs-agent/third_party/g2docsmd-html/exportmd.gs create mode 100644 demos/palm/python/docs-agent/third_party/g2docsmd-html/patches/001-initial-changes-for-docs-agent.patch diff --git a/demos/palm/python/docs-agent/README.md b/demos/palm/python/docs-agent/README.md index f05c08dac..3ec9cfd38 100644 --- a/demos/palm/python/docs-agent/README.md +++ b/demos/palm/python/docs-agent/README.md @@ -1,6 +1,6 @@ # Docs Agent -The Docs Agent demo enables [PaLM API][genai-doc-site] users to launch a chat application +The Docs Agent project enables [PaLM API][genai-doc-site] users to launch a chat application on a Linux-based host machine using their own set of documents as a source dataset. **Note**: If you're interested in setting up and launching the Docs Agent sample app on your @@ -57,7 +57,7 @@ content from the source documents given user questions. Once the most relevant content is returned, the Docs Agent server uses the prompt structure shown in Figure 3 to augment the user question with a preset **condition** and a list of **context**. (When the Docs Agent server starts, the condition value is read from the -[`config.yaml`][condition-txt] file.) Then the Docs Agent server sends this prompt to a +[`config.yaml`][config-yaml] file.) Then the Docs Agent server sends this prompt to a PaLM 2 model using the PaLM API and receives a response generated by the model. ![Docs Agent prompt strcture](docs/images/docs-agent-prompt-structure-01.png) @@ -82,6 +82,9 @@ running on the host machine. The embeddings in this vector database enable the Docs Agent server to perform semantic search and retrieve context related to user questions for augmenting prompts. +For more information on the processing of Markdown files, see the [`README`][scripts-readme] +file in the `scripts` directory. + ![Document to embeddings](docs/images/docs-agent-embeddings-01.png) **Figure 4**. A document is split into small semantic chunks, which are then used to generate @@ -296,6 +299,13 @@ event of "like" for the response. The user may click this like button multiple times to toggle the state of the like button. But when examining the logs, only the final state of the like button will be considered for the response. +### Using Google Docs, PDF, or Gmail as input sources + +The project includes Apps Script files that allow you to convert various sources of content +(including Google Docs and PDF) from your Google Drive and Gmail into Markdown files. You can then +use these Markdown files as additional input sources for Docs Agent. For more information, see the +[`README`][apps-script-readme] file in the `apps_script` directory. + ## Issues identified The following issues have been identified and need to be worked on: @@ -427,7 +437,7 @@ To convert Markdown files to plain text files: cd $HOME/generative-ai-docs/demos/palm/python/docs-agent ``` -2. Open the `config.yaml` file using a text editor, for example: +2. Open the [`config.yaml`][config-yaml] file using a text editor, for example: ``` nano config.yaml @@ -542,7 +552,7 @@ allowing you to easily bring up and destory the Flask app instance. To customize settings in the Docs Agent chat app, do the following: -1. Edit the `config.yaml` file to update the following field: +1. Edit the [`config.yaml`][config-yaml] file to update the following field: ``` product_name: "My product" @@ -636,7 +646,6 @@ Meggin Kearney (`@Meggin`), and Kyo Lee (`@kyolee415`). [set-up-docs-agent]: #set-up-docs-agent [markdown-to-plain-text]: ./scripts/markdown_to_plain_text.py [populate-vector-database]: ./scripts/populate_vector_database.py -[condition-txt]: ./config.yaml [context-source-01]: http://eventhorizontelescope.org [fact-check-section]: #using-a-palm-2-model-to-fact-check-its-own-response [related-questions-section]: #using-a-palm-2-model-to-suggest-related-questions @@ -650,4 +659,7 @@ Meggin Kearney (`@Meggin`), and Kyo Lee (`@kyolee415`). [flutter-docs-src]: https://github.com/flutter/website/tree/main/src [flutter-docs-site]: https://docs.flutter.dev/ [poetry-known-issue]: https://github.com/python-poetry/poetry/issues/1917 +[apps-script-readme]: ./apps_script/README.md +[scripts-readme]: ./scripts/README.md +[config-yaml]: config.yaml [gen-ai-docs-repo]: https://github.com/google/generative-ai-docs diff --git a/demos/palm/python/docs-agent/apps_script/README.md b/demos/palm/python/docs-agent/apps_script/README.md new file mode 100644 index 000000000..a1198f70f --- /dev/null +++ b/demos/palm/python/docs-agent/apps_script/README.md @@ -0,0 +1,164 @@ +# Convert Google Docs, PDF, and Gmail to Markdown files + +The collection of scripts in this `apps_script` directory allows you to convert +the contents of Google Drive folders and Gmail to Markdown files that are +compatible with Docs Agent. + +The steps are: + +1. [Prepare a Google Drive folder](#1-prepare-a-google-driver-folder). +2. [Mount Google Drive on your host machine](#2-mount-google-drive-on-your-host-machine). +3. [Create an Apps Script project](#3-create-an-apps-script-project). +4. [Edit and run main.gs on Apps Script](#4-edit-and-run-main-gs-on-apps-script). +5. [Update config.yaml to include the mounted directory](#5-update-config-yaml-to-include-the-mounted-directory). + +## 1. Prepare a Google Drive folder + +First, create a new folder in Google Drive and add your Google Docs (which will be +used as source documents to Docs Agent) to the folder. + +Do the following: + +1. Browser to https://drive.google.com/. +1. Click **+ New** on the top left corner. +1. Click **New folder**. +1. Name your new folder (for example, `my source Google Docs`). +1. To enter the newly created folder, double click the folder. +1. Add (or move) your source Google Docs to this new folder. + +## 2. Mount Google Drive on your host machine + +Mount your Google Drive to your host machine, so that it becomes easy to access the +folders in Google Drive from your host machine (later in step 5). + +There are a variety of methods and tools available online that enable this setup +(for example, see [`google-drive-ocamlfuse`][google-drive-ocamlfuse] for Linux machines). + +## 3. Create an Apps Script project + +Create a new Apps Script project and copy all the `.gs` scripts in this +`apps_script` directory to your new Apps Script project. + +Do the following: + +1. Browse to https://script.google.com/. +1. Click **New Project**. +1. At the top of the page, click **Untitled Project** and enter a meaningful + title (for example, `gDocs to Docs Agent`). +1. Click the **+** icon next to **Files**. +1. Click **Script**. +1. Name the new script to be one of the `.gs` files in this `apps_script` directory + (for example, `drive_to_markdown`). +1. Copy the content of the `.gs` file to the new script on your Apps Script project. +1. To save, click the "Save project" icon in the toolbar. +1. Repeat the steps until all the `.gs` files are copied to your Apps Script project. +1. Click the **+** icon next to **Services**. +1. Scroll down and click **Drive API**. +1. Click **Add**. + +You are now ready to edit the parameters on the `main.gs` file to select a folder +in Google Drive and export emails from Gmail. + +![Apps Script project](../docs/images/apps-script-screenshot-01.png) + +**Figure 1**. A screenshot of an example Apps Script project. + +## 4. Edit and run main.gs on Apps Script + +Edit the `main.gs` file on your Apps Script project to select which functions +(features) you want to run. + +Do the following: + +1. Browse to your project on https://script.google.com/. + +1. Open the `main.gs` file. + +1. In the `main` function, comment out any functions that you don't want to run + (see Figure 1): + + * `convertDriveFolderToMDForDocsAgent(folderInput)`: This function converts + the contents of a Google Drive folder to Markdown files (currently only Google + Docs and PDF). Make sure to specify a valid Google Drive folder in the `folderInput` + variable. Use the name of the folder created in **step 1** above, for example: + + ``` + var folderInput = "my source Google Docs" + function main() { + convertDriveFolderToMDForDocsAgent(folderInput); + //exportEmailsToMarkdown(SEARCH_QUERY, folderOutput); + } + ``` + + * `exportEmailsToMarkdown(SEARCH_QUERY, folderOutput)`: This function converts + the emails returned from a Gmail search query into Markdown files. Make sure to + specify a search query in the `SEARCH_QUERY` variable. You can test this search + query directly in the Gmail search bar. Also, specify an output directory for the + resulting emails. + +1. To save, click the "Save project" icon in the toolbar. + +1. Click the "Run" icon in the toolbar. + + When this script runs successfully, the Execution log panel prints output similar + to the following: + + ``` + 9:55:59 PM Notice Execution completed + ``` + + Also, the script creates a new folder in your Google Drive and stores the converted + Markdown files in this folder. The name of this new folder has `-output` as a postfix. + For example, with the folder name `my source Google Docs`, the name of the new folder + is `my source Google Docs-output`. + + With Google Drive mounted on your host machine in step 2, you can now directly access + this folder from the host machine, for example: + + ``` + user@hostname:~/DriveFileStream/My Drive/my source Google Docs-output$ ls + Copy_of_My_Google_Docs_To_Be_Converted.md + ``` + +## 5. Update config.yaml to include the mounted directory + +Once you have your Google Drive mounted on the host machine, you can now +specify one of its folders as an input source directory for Docs Agent. + +Do the following: + +1. In the Docs Agent project, open the [`config.yaml`][config-yaml] file + with a text editor. + +1. Specify your mounted Google Drive folder as an `input` group, for example: + + ``` + input: + - path: "/usr/local/google/home/user/DriveFileStream/My Drive/my source Google Docs-output" + url_prefix: "docs.google.com" + ``` + + You **must** specify a value to the `url_prefix` field, such as `docs.google.com`. + Currently this value is used to generate hashes for the content. + +1. (**Optional**) Add an additional Google Drive folder for your exported emails, + for example: + + ``` + input: + - path: "/usr/local/google/home/user/DriveFileStream/My Drive/my source Google Docs-output" + url_prefix: "docs.google.com" + - path: "/usr/local/google/home/user/DriveFileStream/My Drive/psa-output" + url_prefix: "mail.google.com" + ``` + +1. Save the changes in the `config.yaml` file. + +You're all set with a new documentation source for Docs Agent. You can now follow the +instructions in the project's main [`README`][main-readme] file to launch the Docs Agent app. + + + +[config-yaml]: ../config.yaml +[main-readme]: ../README.md +[google-drive-ocamlfuse]: https://github.com/astrada/google-drive-ocamlfuse diff --git a/demos/palm/python/docs-agent/apps_script/drive_to_markdown.gs b/demos/palm/python/docs-agent/apps_script/drive_to_markdown.gs new file mode 100644 index 000000000..2b0ef9072 --- /dev/null +++ b/demos/palm/python/docs-agent/apps_script/drive_to_markdown.gs @@ -0,0 +1,193 @@ +/** + * Copyright 2023 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +function convertDriveFolderToMDForDocsAgent(folderName) { + //Checks if input folder exists or exits + if(folderExistsInput(folderName)){ + var file_count = 0; + var folders = DriveApp.getFoldersByName(folderName); + Logger.log("Output directory: "+ folderName + "-output"); + var folderOutput = folderName + "-output"; + var output_file_name = folderName + "-index"; + folderExistsOrCreate(folderOutput); + var folderOutputObj = DriveApp.getFoldersByName(folderOutput); + if (folderOutputObj.hasNext()){ + var folderOutputName = folderOutputObj.next(); + } + var sheet = checkIndexOutputOrCreate(output_file_name, folderOutputName); + var timeZone = Session.getScriptTimeZone(); + var date = Utilities.formatDate(new Date(), timeZone, "MM-dd-yyyy HH:mm:ss z"); + sheet.appendRow(["Created: ", date]) + sheet.appendRow(["Name","ID", "URL", "Markdown ID", "Markdown Output", "Date Created", "Last Updated", "Type", "Folder", "MD5 hash", "Status"]); + var foldersnext = folders.next(); + var myfiles = foldersnext.getFiles(); + var new_file_count = 0; + var unchanged_file_count = 0; + var updated_file_count = 0; + var gdoc_count = 0; + var pdf_count = 0; + var start_data_row = 2; + var status = "New content"; + + while (myfiles.hasNext()) { + var myfile = myfiles.next(); + var fname = sanitizeFileName(myfile.getName()); + var fdate = myfile.getLastUpdated(); + var furl = myfile.getUrl(); + var fid = myfile.getId(); + var ftype = myfile.getMimeType(); + var fcreate = myfile.getDateCreated(); + + //Function returns an array, assign each array value to seperate variables + var backup_results = returnBackupHash(sheet, "Backup", fid, start_data_row, 1, 9, 3); + if (backup_results != undefined && backup_results[0] != "no_results") { + var backup_fid = backup_results[0]; + var md5_backup = backup_results[1]; + var mdoutput_backup_id = backup_results[2]; + } + + if (ftype == "application/vnd.google-apps.document") { + Logger.log("File: " + fname + " is a Google doc."); + let gdoc = DocumentApp.openById(fid); + let gdoc_blob = gdoc.getBody().getText(); + var md5_hash = Utilities.computeDigest(Utilities.DigestAlgorithm.MD5,gdoc_blob, + Utilities.Charset.US_ASCII); + var hash_str = byteToStr(md5_hash); + if (backup_fid == fid && hash_str == md5_backup) { + Logger.log("File is unchanged. Skipping conversion."); + if (mdoutput_backup_id){ + var saved_file = DriveApp.getFileById(mdoutput_backup_id); + var saved_file_id = saved_file.getId(); + } + status = "Unchanged content"; + unchanged_file_count += 1; + var convert_file = false; + } + else if (backup_fid == fid && hash_str != md5_backup){ + status = "Updated content"; + updated_file_count += 1; + var convert_file = true; + } + else { + status = "New content"; + new_file_count += 1; + var convert_file = true; + } + if (convert_file){ + var frontmatter = "---" + "\n"; + frontmatter += "title: \"" + fname + "\"\n"; + frontmatter += "type: \"" + ftype + "\"\n"; + frontmatter += "id: \"" + fid + "\"\n"; + frontmatter += "created: \"" + fcreate + "\"\n"; + frontmatter += "updated: \"" + fdate + "\"\n"; + frontmatter += "URL: \"" + furl + "\"\n"; + frontmatter += "---" + "\n\n"; + var saved_file = convertDocumentToMarkdown(gdoc, folderOutputName, frontmatter); + var saved_file_id = saved_file.getId(); + Logger.log("Finished converting file: " + fname + " to markdown."); + Logger.log("Markdown file: " + saved_file); + status = "New content"; + gdoc_count += 1; + } + file_count += 1; + } + if (ftype == "application/pdf") { + // Converts PDFs - First to a temporary Google Doc and then use convertDocumentToMarkdown to convert to markdown with frontmatter + Logger.log("File: " + fname + " is a PDF."); + let pdfBlob = DriveApp.getFileById(fid).getBlob(); + let pdfblobText = pdfBlob.getDataAsString(); + var md5_hash = Utilities.computeDigest(Utilities.DigestAlgorithm.MD5,pdfblobText, + Utilities.Charset.US_ASCII); + var hash_str = byteToStr(md5_hash); + if (backup_fid == fid && hash_str == md5_backup) { + Logger.log("File is unchanged. Skipping conversion."); + if (mdoutput_backup_id){ + var saved_file = DriveApp.getFileById(mdoutput_backup_id); + var saved_file_id = saved_file.getId(); + } + status = "Unchanged content"; + unchanged_file_count += 1; + var convert_file = false; + } + else if (backup_fid == fid && hash_str != md5_backup){ + status = "Updated content"; + updated_file_count += 1; + var convert_file = true; + } + else { + status = "New content"; + new_file_count += 1; + var convert_file = true; + } + if (convert_file){ + let temp_doc_name = pdfBlob.getName() + "-temp"; + let temp_doc = {title: temp_doc_name, mimeType: pdfBlob.getContentType(), parents: [{id: folderOutputName.getId()}]} + let options = {ocr: true}; + let output = Drive.Files.insert(temp_doc, pdfBlob, options); + let output_id = output.getId(); + let gdoc = DocumentApp.openById(output_id); + var frontmatter = "---" + "\n"; + frontmatter += "title: \"" + fname + "\"\n"; + frontmatter += "type: \"" + ftype + "\"\n"; + frontmatter += "id: \"" + fid + "\"\n"; + frontmatter += "created: \"" + fcreate + "\"\n"; + frontmatter += "updated: \"" + fdate + "\"\n"; + frontmatter += "URL: \"" + furl + "\"\n"; + frontmatter += "---" + "\n\n"; + var saved_file = convertDocumentToMarkdown(gdoc, folderOutputName, frontmatter); + var saved_file_id = saved_file.getId(); + Logger.log("Finished converting file: "+ fname + " to markdown."); + Logger.log("Markdown file: " + saved_file); + Logger.log("Clearing temporary gdoc: " ); + let output_file = DriveApp.getFileById(output_id); + output_file.setTrashed(true); + status = "New content"; + pdf_count += 1; + } + file_count += 1; + } + let md_chip = createRichText(saved_file); + let original_chip = createRichText(myfile); + let folder_chip = createRichText(foldersnext); + metadata = [ + fname, + fid, + "original_chip", + saved_file_id, + "md_chip", + fcreate, + fdate, + ftype, + "folder_chip", + hash_str, + status, + ]; + row_number = file_count + start_data_row; + sheet.appendRow(metadata); + insertRichText(sheet, original_chip, "C", row_number); + insertRichText(sheet, md_chip, "E", row_number); + insertRichText(sheet, folder_chip, "I", row_number); + } + } + let conversion_count = pdf_count + gdoc_count + Logger.log("Converted a total of: " + gdoc_count + " Google Doc files."); + Logger.log("Converted a total of: " + pdf_count + " PDF files."); + Logger.log("Converted a grand total of: " + conversion_count + " files."); + Logger.log("New files: " + new_file_count) + Logger.log("Updated a total of: " + updated_file_count + " files.") + Logger.log("Files that haven't changed: " + unchanged_file_count); + Logger.log("Input directory had a total of: " + file_count + " files.") +} diff --git a/demos/palm/python/docs-agent/apps_script/exportmd.gs b/demos/palm/python/docs-agent/apps_script/exportmd.gs new file mode 100644 index 000000000..283872c79 --- /dev/null +++ b/demos/palm/python/docs-agent/apps_script/exportmd.gs @@ -0,0 +1,1308 @@ +/** + * Copyright 2023 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/* Original script is from: +https://github.com/lmmx/gdocs2md-html/blob/master/exportmd.gs +and commit: 0d86cfa +Parsing from mangini/gdocs2md. +Modified by clearf to add files to the google directory structure. +Modified by lmmx to write Markdown, going back to HTML-incorporation. + +Usage: + NB: don't use on top-level doc (in root Drive folder) See comment in setupScript function. + Adding this script to your doc: + - Tools > Script Manager > New + - Select "Blank Project", then paste this code in and save. + Running the script: + - Tools > Script Manager + - Select "convertDocumentToMarkdown" function. + - Click Run button. + - Converted doc will be added to a "Markdown" folder in the source document's directories. + - Images will be added to a subfolder of the "Markdown" folder. +*/ + +function onInstall(e) { + onOpen(e); +} + +function onOpen() { + // Add a menu with some items, some separators, and a sub-menu. + setupScript(); +// In future: +// DocumentApp.getUi().createAddonMenu(); + DocumentApp.getUi().createMenu('Markdown') + .addItem('View as markdown', 'markdownPopup') + .addSubMenu(DocumentApp.getUi().createMenu('Export \u2192 markdown') + .addItem('Export to local file', 'convertSingleDoc') + .addItem('Export entire folder to local file', 'convertFolder') + .addItem('Customise markdown conversion', 'changeDefaults')) + .addSeparator() + .addSubMenu(DocumentApp.getUi().createMenu('Toggle comment visibility') + .addItem('Image source URLs', 'toggleImageSourceStatus') + .addItem('All comments', 'toggleCommentStatus')) + .addItem("Add comment", 'addCommentDummy') + .addToUi(); +} + +function changeDefaults() { + var ui = DocumentApp.getUi(); + var default_settings = '{ use your imagination... }'; + var greeting = ui.alert('This should be set up to display defaults from variables passed to getDocComments etc., e.g. something like:\n\nDefault settings are:' + + '\ncomments - not checking deleted comments.\nDocument - this document (alternatively specify a document ID).' + + '\n\nClick OK to edit these, or cancel.', + ui.ButtonSet.OK_CANCEL); + ui.alert("There's not really need for this yet, so this won't proceed, regardless of what you just pressed."); + return; + + // Future: + if (greeting == ui.Button.CANCEL) { + ui.alert("Alright, never mind!"); + return; + } + // otherwise user clicked OK + // user clicked OK, to proceed with editing these defaults. Ask case by case whether to edit + + var response = ui.prompt('What is x (default y)?', ui.ButtonSet.YES_NO_CANCEL); + + // Example code from docs at https://developers.google.com/apps-script/reference/base/button-set + // Process the user's response. + if (response.getSelectedButton() == ui.Button.YES) { + Logger.log('The user\'s name is %s.', response.getResponseText()); + } else if (response.getSelectedButton() == ui.Button.NO) { + Logger.log('The user didn\'t want to provide a name.'); + } else { + Logger.log('The user clicked the close button in the dialog\'s title bar.'); + } +} + +function setupScript() { + var script_properties = PropertiesService.getScriptProperties(); + script_properties.setProperty("user_email", Drive.About.get().user.emailAddress); + + // manual way to do the following: + // script_properties.setProperty("folder_id", "INSERT_FOLDER_ID_HERE"); + // script_properties.setProperty("document_id", "INSERT_FILE_ID_HERE"); + + var doc_id = DocumentApp.getActiveDocument().getId(); + script_properties.setProperty("document_id", doc_id); + var doc_parents = DriveApp.getFileById(doc_id).getParents(); + var folders = doc_parents; + while (folders.hasNext()) { + var folder = folders.next(); + var folder_id = folder.getId(); + } + script_properties.setProperty("folder_id", folder_id); + script_properties.setProperty("image_folder_prefix", ""); // add if modifying image location +} + +function addCommentDummy() { + // Dummy function to be switched during development for addComment + DocumentApp.getUi() + .alert('Cancelling comment entry', + "There's not currently a readable anchor for Google Docs - you need to write your own!" + + + "\n\nThe infrastructure for using such an anchoring schema is sketched out in" + + " the exportmd.gs script's addComment function, for an anchor defined in anchor_props" + + + "\n\nSee github.com/lmmx/devnotes/wiki/Custom-Google-Docs-comment-anchoring-schema", + DocumentApp.getUi().ButtonSet.OK + ); + return; +} + +function addComment() { + + var doc_id = PropertiesService.getScriptProperties().getProperty('document_id'); + var user_email = PropertiesService.getScriptProperties().getProperty('email'); +/* Drive.Comments.insert({content: "hello world", + context: { + type: 'text/html', + value: 'hinges' + } + }, document_id); */ + var revision_list = Drive.Revisions.list(doc_id).items; + var recent_revision_id = revision_list[revision_list.length - 1].id; + var anchor_props = { + revision_id: recent_revision_id, + starting_offset: '', + offset_length: '', + total_chars: '' + } + insertComment(doc_id, 'hinges', 'Hello world!', my_email, anchor_props); +} + +function insertComment(fileId, selected_text, content, user_email, anchor_props) { + + // NB Deal with handling missing args + + /* + anchor_props is an object with 4 properties: + - revision_id, + - starting_offset, + - offset_length, + - total_chars + */ + + var context = Drive.newCommentContext(); + context.value = selected_text; + context.type = 'text/html'; + var comment = Drive.newComment(); + comment.kind = 'drive#comment'; + var author = Drive.newUser(); + author.kind = 'drive#user'; + author.displayName = user_email; + author.isAuthenticatedUser = true; + comment.author = author; + comment.content = type; + comment.context = context; + comment.status = 'open'; + comment.anchor = "{'r':" + + anchor_props.revision_id + + ",'a':[{'txt':{'o':" + + anchor_props.starting_offset + + ",'l':" + + anchor_props.offset_length + + ",'ml':" + + anchor_props.total_chars + + "}}]}"; + comment.fileId = fileId; + Drive.Comments.insert(comment, fileId); +} + +function decodeScriptSwitches(optional_storage_name) { + var property_name = (typeof(optional_storage_name) == 'string') ? optional_storage_name : 'switch_settings'; + var script_properties = PropertiesService.getScriptProperties(); + return script_properties + .getProperty(property_name) + .replace(/{|}/g,'') // Get the statements out of brackets... + .replace(',', ';'); // ...swap the separator for a semi-colon... + // ...evaluate the stored object string as statements upon string return and voila, switches interpreted +} + + +function getDocComments(comment_list_settings) { + var possible_settings = ['images', 'include_deleted']; + + // switches are processed and set on a script-wide property called "comment_switches" + var property_name = 'comment_switches'; + switchHandler(comment_list_settings, possible_settings, property_name); + + var script_properties = PropertiesService.getScriptProperties(); + var comment_switches = decodeScriptSwitches(property_name); + eval(comment_switches); + + var document_id = script_properties.getProperty("document_id"); + var comments_list = Drive.Comments.list(document_id, + {includeDeleted: include_deleted, + maxResults: 100 }); // 0 to 100, default 20 + // See https://developers.google.com/drive/v2/reference/comments/list for all options + var comment_array = []; + var image_sources = []; + // To collect all comments' image URLs to match against inlineImage class elements LINK_URL attribute + + for (var i = 0; i < comments_list.items.length; i++) { + var comment = comments_list.items[i]; + var comment_text = comment.content; + var comment_status = comment.status; + /* + images is a generic parameter passed in as a switch to + return image URL-containing comments only. + + If the parameter is provided, it's no longer undefined. + */ + var img_url_regex = /(https?:\/\/.+?\.(png|gif|jpe?g))/; + var has_img_url = img_url_regex.test(comment_text); + + if (images && !has_img_url) continue; // no image URL, don't store comment + if (has_img_url) image_sources.push(RegExp.$1); + comment_array.push(comment); + } + script_properties.setProperty('image_source_URLs', image_sources) + return comment_array; +} + +function isValidAttrib(attribute) { // Sanity check function, called per element in array + + // Possible list of attributes to check against (leaving out unchanging ones like kind) + possible_attrs = [ + 'selfLink', + 'commentId', + 'createdDate', + 'modifiedDate', + 'author', + 'htmlContent', + 'content', + 'deleted', + 'status', + 'context', + 'anchor', + 'fileId', + 'fileTitle', + 'replies', + 'author' + ]; + + // Check if attribute(s) provided can be used to match/filter comments: + + if (typeof(attribute) == 'string' || typeof(attribute) == 'object') { + // Either a string/object (1-tuple) + + // Generated with Javascript, gist: https://gist.github.com/lmmx/451b301e1d78ed2c10b4 + + // Return false from the function if any of the attributes specified are not in the above list + + // If an object, the name is the key, otherwise it's just the string + if (attribute.constructor === Object) { + var att_keys = []; + for (var att_key in attribute) { + if (attribute.hasOwnProperty(att_key)) { + att_keys.push(att_key); + } + } + for (var n=0; n < att_keys.length; n++) { + var attribute_name = att_keys[n]; + var is_valid_attrib = (possible_attrs.indexOf(attribute_name) > -1); + + // The attribute needs to be one of the possible attributes listed above, match its given value(s), + // else returning false will throw an error from onAttribError when within getCommentAttributes + return is_valid_attrib; + } + } else if (typeof(attribute) == 'string') { + var attribute_name = attribute; + var is_valid_attrib = (possible_attrs.indexOf(attribute_name) > -1); + return is_valid_attrib; + // Otherwise is a valid (string) attribute + } else if (attribute.constructor === Array) { + return false; // Again, if within getCommentAttributes this will cause an error - shouldn't pass an array + } else { + // Wouldn't expect this to happen, so give a custom error message + Logger.log('Unknown type (assumed impossible) passed to isValidAttrib: ', attribute, attribute.constructor); + throw new TypeError('Unknown passed to isValidAttrib - this should be receiving 1-tuples only, see logs for details.'); + } + } else return false; // Neither string/object / array of strings &/or objects - not a valid attribute +} + +function getCommentAttributes(attributes, comment_list_settings) { + + // A filter function built on Comments.list, for a given list of attributes + // Objects' values are ignored here, only their property titles are used to filter comments. + + + /* + - attributes: array of attributes to filter/match on + - comment_list_settings: (optional) object with properties corresponding to switches in getDocComments + + This function outputs an array of the same length as the comment list, containing + values for all fields matched/filtered on. + */ + + + /* + * All possible comment attributes are listed at: + * https://developers.google.com/drive/v2/reference/comments#properties + */ + + // Firstly, describe the type in a message to be thrown in case of TypeError: + + var attrib_def_message = "'attributes' should be a string (the attribute to get for each comment), " + + "an object (a key-value pair for attribute and desired value), " + + "or an array of objects (each with key-value pairs)"; + + function onAttribError(message) { + Logger.log(message); + throw new TypeError(message); + } + + // If (optional) comment_list_settings isn't set, make a getDocComments call with switches left blank. + if (typeof(comment_list_settings) == 'undefined') var comment_list_settings = {}; + if (typeof(attributes) == 'undefined') onAttribError(attrib_def_message); // no variables specified + + if (isValidAttrib(attributes)) { // This will be true if there's only one attribute, not provided in an array + + /* + Make a 1-tuple (array of 1) from either an object or a string, + i.e. a single attribute, with or without a defined value respectively. + */ + + var attributes = Array(attributes); + + } else if (attributes.constructor === Array) { + + // Check each item in the array is a valid attribute specification + for (var l = 0; l < attributes.length; l++) { + if (! isValidAttrib(attributes[l]) ) { + onAttribError('Error in attribute ' + + (l+1) + ' of ' + attributes.length + + '\n\n' + + attrib_def_message); + } + } + + } else { // Neither attribute nor array of attributes + throw new TypeError(attrib_def_message); + } + + // Attributes now holds an array of string and/or objects specifying a comment match and/or filter query + + var comment_list = getDocComments(comment_list_settings); + var comment_attrib_lists = []; + for (var i in comment_list) { + var comment = comment_list[i]; + var comment_attrib_list = []; + for (var j in attributes) { + var comment_attribute = comment_list[i][attributes[j]]; + comment_attrib_list.push(comment_attribute); + } + comment_attrib_lists.push(comment_attrib_list); + } + // The array comment_attrib_lists is now full of the requested attributes, + // of length equal to that of attributes + return comment_attrib_lists; +} + +// Example function to use getCommentAttributes: + +function filterComments(attributes, comment_list_settings) { + var comment_attributes = getCommentAttributes(attributes, comment_list_settings); + var m = attribs.indexOf('commentId') // no need to keep track of commentID array position + comm_attribs.map(function(attrib_pair) { + if (attrib_pair[1]); + }) +} + +function toggleCommentStatus(comment_switches){ + // Technically just image URL-containing comments, not sources just yet + var attribs = ['commentId', 'status']; + var comm_attribs = getCommentAttributes(attribs, comment_switches); + var rearrangement = []; + comm_attribs.map( + function(attrib_pair) { // for every comment return with the images_only / images: true comments.list setting, + switch (attrib_pair[1]){ // check the status of each + case 'open': + rearrangement.push([attrib_pair[0],'resolved']); + break; + case 'resolved': + rearrangement.push([attrib_pair[0],'open']); + break; + } + } + ); + var script_properties = PropertiesService.getScriptProperties(); + var doc_id = script_properties.getProperty("document_id"); + rearrangement.map( + function(new_attrib_pair) { // for every comment ID with flipped status + Drive.Comments.patch('{"status": "' + + new_attrib_pair[1] + + '"}', doc_id, new_attrib_pair[0]) + } + ); + return; +} + +function toggleImageSourceStatus(){ + toggleCommentStatus({images: true}); +} + +function flipResolved() { + // Flip the status of resolved comments to open, and open comments to resolved (respectful = true) + // I.e. make resolved URL-containing comments visible, without losing track of normal comments' status + + // To force all comments' statuses to switch between resolved and open en masse set respectful to false + + var switch_settings = {}; + switch_settings.respectful = true; + switch_settings.images_only = false; // If true, only switch status of comments with an image URL + switch_settings.switch_deleted_comments = false; // If true, also switch status of deleted comments + + var comments_list = getDocComments( + { images: switch_settings.images_only, + include_deleted: switch_settings.switch_deleted_comments }); + + // Note: these parameters are unnecessary if both false (in their absence assumed false) + // but included for ease of later reuse + + if (switch_settings.respectful) { + // flip between + } else { + // flip all based on status of first in list + } +} + +function markdownPopup() { + var css_style = ''; + + // The above was written with js since doesn't work: + // https://gist.github.com/lmmx/ec084fc351528395f2bb + + var mdstring = stringMiddleMan(); + + var htmlstring = + '' + + css_style + + '
'; + + var html5 = HtmlService.createHtmlOutput(htmlstring) + .setSandboxMode(HtmlService.SandboxMode.IFRAME) + .setWidth(800) + .setHeight(500); + + DocumentApp.getUi() + .showModalDialog(html5, 'Markdown output'); +} + +function stringMiddleMan() { + var returned_string; + convertSingleDoc({"return_string": true}); // for some reason needs the scope to be already set... + // could probably rework to use mdstring rather than returned_string, cut out middle man function + return this.returned_string; +} + +function convertSingleDoc(optional_switches) { + var script_properties = PropertiesService.getScriptProperties(); + // renew comments list on every export + var doc_comments = getDocComments(); + var image_urls = getDocComments({images: true}); // NB assumed false - any value will do + script_properties.setProperty("comments", doc_comments); + script_properties.setProperty("image_srcs", image_urls); + var folder_id = script_properties.getProperty("folder_id"); + var document_id = script_properties.getProperty("document_id"); + var source_folder = DriveApp.getFolderById(folder_id); + var markdown_folders = source_folder.getFoldersByName("Markdown"); + + var markdown_folder; + if (markdown_folders.hasNext()) { + markdown_folder = markdown_folders.next(); + } else { + // Create a Markdown folder if it doesn't exist. + markdown_folder = source_folder.createFolder("Markdown") + } + + convertDocumentToMarkdown(DocumentApp.openById(document_id), markdown_folder, optional_switches); +} + +function convertFolder() { + var script_properties = PropertiesService.getScriptProperties(); + var folder_id = script_properties.getProperty("folder_id"); + var source_folder = DriveApp.getFolderById(folder_id); + var markdown_folders = source_folder.getFoldersByName("Markdown"); + + + var markdown_folder; + if (markdown_folders.hasNext()) { + markdown_folder = markdown_folders.next(); + } else { + // Create a Markdown folder if it doesn't exist. + markdown_folder = source_folder.createFolder("Markdown"); + } + + // Only try to convert google docs files. + var gdoc_files = source_folder.getFilesByType("application/vnd.google-apps.document"); + + // For every file in this directory + while(gdoc_files.hasNext()) { + var gdoc_file = gdoc_files.next() + + var filename = gdoc_file.getName(); + var md_files = markdown_folder.getFilesByName(filename + ".md"); + var update_file = false; + + if (md_files.hasNext()) { + var md_file = md_files.next(); + + if (md_files.hasNext()){ // There are multiple markdown files; delete and rerun + update_file = true; + } else if (md_file.getLastUpdated() < gdoc_file.getLastUpdated()) { + update_file = true; + } + } else { + // There is no folder and the conversion needs to be rerun + update_file = true; + } + + if (update_file) { + convertDocumentToMarkdown(DocumentApp.openById(gdoc_file.getId()), markdown_folder); + } + } +} + +function switchHandler(input_switches, potential_switches, optional_storage_name) { + + // Firstly, if no input switches were set, make an empty input object + if (typeof(input_switches) == 'undefined') input_switches = {}; + + // Use optional storage name if it's defined (must be a string), else use default variable name "switch_settings" + var property_name = (typeof(optional_storage_name) == 'string') ? optional_storage_name : 'switch_settings'; + + // Make a blank object to be populated and stored as the script-wide property named after property_name + var switch_settings = {}; + + for (var i in potential_switches) { + var potential_switch = potential_switches[i]; + + // If each switch has been set (in input_switches), evaluate it, else assume it's switched off (false): + + if (input_switches.propertyIsEnumerable(potential_switch)) { + + // Evaluates a string representing a statement which sets switch_settings properties from input_switches + // e.g. "switch_settings.images = true" when input_switches = {images: true} + + eval('switch_settings.' + potential_switch + " = " + input_switches[potential_switch]); + + } else { + + // Alternatively, the evaluated statement sets anything absent from the input_switches object as false + // e.g. "switch_settings.images = false" when input_switches = {} and potential_switches = ['images'] + + eval('switch_settings.' + potential_switch + " = false"); + } + } + + PropertiesService.getScriptProperties().setProperty(property_name, switch_settings); + + /* + Looks bad but more sensible than repeatedly checking if arg undefined. + + Sets every variable named in the potential_switches array to false if + it wasn't passed into the input_switches object, otherwise evaluates. + + Any arguments not passed in are false, but so are any explicitly passed in as false: + all parameters are therefore Boolean until otherwise specified. + */ + +} + +function convertDocumentToMarkdown(document, destination_folder, frontmatter_input, optional_switches) { + // if returning a string, force_save_images will make the script continue - experimental + var possible_switches = ['return_string', 'force_save_images']; + var property_name = 'conversion_switches'; + switchHandler(optional_switches, possible_switches, property_name); + + // TODO switch off image storage if force_save_images is true - not necessary for normal behaviour + var script_properties = PropertiesService.getScriptProperties(); + var comment_switches = decodeScriptSwitches(property_name); + eval(comment_switches); + + var image_prefix = script_properties.getProperty("image_folder_prefix"); + var numChildren = document.getActiveSection().getNumChildren(); + if (frontmatter_input != "") { + var text = frontmatter_input; + } + else { + var text = "" + } + var md_filename = sanitizeFileName(document.getName()) + ".md"; + var image_foldername = document.getName()+"_images"; + var inSrc = false; + var inClass = false; + var globalImageCounter = 0; + var globalListCounters = {}; + // edbacher: added a variable for indent in src
 block. Let style sheet do margin.
+  var srcIndent = "";
+
+  var postHasImages = false;
+
+  var files = [];
+
+  // Walk through all the child elements of the doc.
+  for (var i = 0; i < numChildren; i++) {
+    var child = document.getActiveSection().getChild(i);
+    var result = processParagraph(i, child, inSrc, globalImageCounter, globalListCounters, image_prefix + image_foldername);
+    globalImageCounter += (result && result.images) ? result.images.length : 0;
+    if (result!==null) {
+      if (result.sourceGlossary==="start" && !inSrc) {
+        inSrc=true;
+        text+="
\n";
+      } else if (result.sourceGlossary==="end" && inSrc) {
+        inSrc=false;
+        text+="
\n\n"; + } else if (result.sourceFigCap==="start" && !inSrc) { + inSrc=true; + text+="
\n";
+      } else if (result.sourceFigCap==="end" && inSrc) {
+        inSrc=false;
+        text+="
\n\n"; + } else if (result.source==="start" && !inSrc) { + inSrc=true; + text+="
\n";
+      } else if (result.source==="end" && inSrc) {
+        inSrc=false;
+        text+="
\n\n"; + } else if (result.inClass==="start" && !inClass) { + inClass=true; + text+="
\n";
+      } else if (result.inClass==="end" && inClass) {
+        inClass=false;
+        text+="
\n\n"; + } else if (inClass) { + text+=result.text+"\n\n"; + } else if (inSrc) { + text+=(srcIndent+escapeHTML(result.text)+"\n"); + } else if (result.text && result.text.length>0) { + text+=result.text+"\n\n"; + } + + if (result.images && result.images.length>0) { + for (var j=0; j/g, '>'); +} + +function standardQMarks(text) { + return text.replace(/\u2018|\u8216|\u2019|\u8217/g,"'").replace(/\u201c|\u8220|\u201d|\u8221/g, '"') +} + +// Process each child element (not just paragraphs). +function processParagraph(index, element, inSrc, imageCounter, listCounters, image_path) { + // First, check for things that require no processing. + if (element.getType() === DocumentApp.ElementType.UNSUPPORTED) { + return null; + } + if (element.getNumChildren()==0) { + return null; + } + // Skip on TOC. + if (element.getType() === DocumentApp.ElementType.TABLE_OF_CONTENTS) { + return {"text": "[[TOC]]"}; + } + + // Set up for real results. + var result = {}; + var pOut = ""; + var textElements = []; + var imagePrefix = "image_"; + + // Handle Table elements. Pretty simple-minded now, but works for simple tables. + // Note that Markdown does not process within block-level HTML, so it probably + // doesn't make sense to add markup within tables. + if (element.getType() === DocumentApp.ElementType.TABLE) { + textElements.push("\n"); + var nCols = element.getChild(0).getNumCells(); + for (var i = 0; i < element.getNumChildren(); i++) { + textElements.push(" \n"); + // process this row + for (var j = 0; j < nCols; j++) { + textElements.push(" \n"); + } + textElements.push(" \n"); + } + textElements.push("
" + element.getChild(i).getChild(j).getText() + "
\n"); + } + + // Need to handle this element type, return null for now + if (element.getType() === DocumentApp.ElementType.CODE_SNIPPET) { + return null + } + + // Process various types (ElementType). + for (var i = 0; i < element.getNumChildren(); i++) { + var t = element.getChild(i).getType(); + + if (t === DocumentApp.ElementType.TABLE_ROW) { + // do nothing: already handled TABLE_ROW + } else if (t === DocumentApp.ElementType.TEXT) { + var txt = element.getChild(i); + pOut += txt.getText(); + textElements.push(txt); + } else if (t === DocumentApp.ElementType.INLINE_IMAGE) { + var imglink = element.getChild(i).getLinkUrl(); + result.images = result.images || []; + var blob = element.getChild(i).getBlob() + var contentType = blob.getContentType(); + var extension = ""; + if (/\/png$/.test(contentType)) { + extension = ".png"; + } else if (/\/gif$/.test(contentType)) { + extension = ".gif"; + } else if (/\/jpe?g$/.test(contentType)) { + extension = ".jpg"; + } else { + throw "Unsupported image type: "+contentType; + } + + var name = imagePrefix + imageCounter + extension; + blob.setName(name); + + imageCounter++; + if (!return_string || force_save_images) { + textElements.push('![](' + image_path + '/' + name + ')'); + } else { + textElements.push('![](' + imglink + ')'); + } + //result.images.push( { + // "bytes": blob.getBytes(), + // "type": contentType, + // "name": name}); + + result.images.push({ "blob" : blob } ) + + // Need to fix this case TODO + } else if (t === DocumentApp.ElementType.INLINE_DRAWING) { + + imageCounter++; + if (!return_string || force_save_images) { + textElements.push('![](' + "drawing" + '/' + " name" + ')'); + } else { + textElements.push('![](' + "drawing" + ')'); + } + //result.images.push( { + // "bytes": blob.getBytes(), + // "type": contentType, + // "name": name}); + + // result.images.push({ "blob" : blob } ) + + } + else if (t === DocumentApp.ElementType.PAGE_BREAK) { + // ignore + } else if (t === DocumentApp.ElementType.HORIZONTAL_RULE) { + textElements.push('* * *\n'); + } else if (t === DocumentApp.ElementType.FOOTNOTE) { + textElements.push(' ('+element.getChild(i).getFootnoteContents().getText()+')'); + // Fixes for new elements + } else if (t === DocumentApp.ElementType.DATE) { + textElements.push(' ('+element.getChild(i)+')'); + } else if (t === DocumentApp.ElementType.RICH_LINK) { + textElements.push(' ('+element.getChild(i).getUrl()+')'); + } else if (t === DocumentApp.ElementType.PERSON) { + textElements.push(element.getChild(i).getName() + ', '); + } else if (t === DocumentApp.ElementType.UNSUPPORTED) { + textElements.push(' '); + } else { + throw "Paragraph "+index+" of type "+element.getType()+" has an unsupported child: " + +t+" "+(element.getChild(i)["getText"] ? element.getChild(i).getText():'')+" index="+index; + } + } + + if (textElements.length==0) { + // Isn't result empty now? + return result; + } + +// Fix for unrecognized command getIndentFirstLine + var ind_f = 0; + var ind_s = 0; + var ind_e = 0; + if (t === DocumentApp.ElementType.PARAGRAPH) { + + var ind_f = element.getIndentFirstLine(); + var ind_s = element.getIndentStart(); + var ind_e = element.getIndentEnd(); + } + var i_fse = [ind_f,ind_s,ind_e]; + var indents = {}; + for (indt=0;indt 0) indents[indname] = eval(indname); + // lazy test, null (no indent) is not greater than zero, but becomes set if indent 'undone' + } + var inIndent = (Object.keys(indents).length > 0); + + // evb: Add glossary and figure caption too. (And abbreviations: gloss and fig-cap.) + // process source code block: + if (/^\s*---\s+gloss\s*$/.test(pOut) || /^\s*---\s+source glossary\s*$/.test(pOut)) { + result.sourceGlossary = "start"; + } else if (/^\s*---\s+fig-cap\s*$/.test(pOut) || /^\s*---\s+source fig-cap\s*$/.test(pOut)) { + result.sourceFigCap = "start"; + } else if (/^\s*---\s+src\s*$/.test(pOut) || /^\s*---\s+source code\s*$/.test(pOut)) { + result.source = "start"; + } else if (/^\s*---\s+class\s+([^ ]+)\s*$/.test(pOut)) { + result.inClass = "start"; + result.className = RegExp.$1.replace(/\./g,' '); + } else if (/^\s*---\s*$/.test(pOut)) { + result.source = "end"; + result.sourceGlossary = "end"; + result.sourceFigCap = "end"; + result.inClass = "end"; + } else if (/^\s*---\s+jsperf\s*([^ ]+)\s*$/.test(pOut)) { + result.text = ''; + } else { + + prefix = findPrefix(inSrc, element, listCounters); + + var pOut = ""; + for (var i=0; i): + if (gt === DocumentApp.GlyphType.BULLET + || gt === DocumentApp.GlyphType.HOLLOW_BULLET + || gt === DocumentApp.GlyphType.SQUARE_BULLET) { + prefix += "* "; + } else { + // Ordered list (
    ): + var key = listItem.getListId() + '.' + listItem.getNestingLevel(); + var counter = listCounters[key] || 0; + counter++; + listCounters[key] = counter; + prefix += counter+". "; + } + } + } + return prefix; +} + +function processTextElement(inSrc, txt) { + if (typeof(txt) === 'string') { + return txt; + } + + var pOut = txt.getText(); + if (! txt.getTextAttributeIndices) { + return pOut; + } + +// Logger.log("Initial String: " + pOut) + + // CRC introducing reformatted_txt to let us apply rational formatting that we can actually parse + var reformatted_txt = txt.copy(); + reformatted_txt.deleteText(0,pOut.length-1); + reformatted_txt = reformatted_txt.setText(pOut); + + var attrs = txt.getTextAttributeIndices(); + var lastOff = pOut.length; + // We will run through this loop multiple times for the things we care about. + // Font + // URL + // Then for alignment + // Then for bold + // Then for italic. + + // FONTs + var lastOff = pOut.length; // loop goes backwards, so this holds + for (var i=attrs.length-1; i>=0; i--) { + var off=attrs[i]; + var font=txt.getFontFamily(off) + if (font) { + while (i>=1 && txt.getFontFamily(attrs[i-1])==font) { + // detect fonts that are in multiple pieces because of errors on formatting: + i-=1; + off=attrs[i]; + } + reformatted_txt.setFontFamily(off, lastOff-1, font); + } + lastOff=off; + } + + // URL + // XXX TODO actually convert to URL text here. + var lastOff=pOut.length; + for (var i=attrs.length-1; i>=0; i--) { + var off=attrs[i]; + var url=txt.getLinkUrl(off); + if (url) { + while (i>=1 && txt.getLinkUrl(attrs[i-1]) == url) { + // detect urls that are in multiple pieces because of errors on formatting: + i-=1; + off=attrs[i]; + } + reformatted_txt.setLinkUrl(off, lastOff-1, url); + } + lastOff=off; + } + + // alignment + var lastOff=pOut.length; + for (var i=attrs.length-1; i>=0; i--) { + var off=attrs[i]; + var alignment=txt.getTextAlignment(off); + if (alignment) { // + while (i>=1 && txt.getTextAlignment(attrs[i-1]) == alignment) { + i-=1; + off=attrs[i]; + } + reformatted_txt.setTextAlignment(off, lastOff-1, alignment); + } + lastOff=off; + } + + // strike + var lastOff=pOut.length; + for (var i=attrs.length-1; i>=0; i--) { + var off=attrs[i]; + var strike=txt.isStrikethrough(off); + if (strike) { + while (i>=1 && txt.isStrikethrough(attrs[i-1])) { + i-=1; + off=attrs[i]; + } + reformatted_txt.setStrikethrough(off, lastOff-1, strike); + } + lastOff=off; + } + + // bold + var lastOff=pOut.length; + for (var i=attrs.length-1; i>=0; i--) { + var off=attrs[i]; + var bold=txt.isBold(off); + if (bold) { + while (i>=1 && txt.isBold(attrs[i-1])) { + i-=1; + off=attrs[i]; + } + reformatted_txt.setBold(off, lastOff-1, bold); + } + lastOff=off; + } + + // italics + var lastOff=pOut.length; + for (var i=attrs.length-1; i>=0; i--) { + var off=attrs[i]; + var italic=txt.isItalic(off); + if (italic) { + while (i>=1 && txt.isItalic(attrs[i-1])) { + i-=1; + off=attrs[i]; + } + reformatted_txt.setItalic(off, lastOff-1, italic); + } + lastOff=off; + } + + + var mOut=""; // Modified out string + var harmonized_attrs = reformatted_txt.getTextAttributeIndices(); + reformatted_txt.getTextAttributeIndices(); // @lmmx: is this a typo...? + pOut = reformatted_txt.getText(); + + + // Markdown is farily picky about how it will let you intersperse spaces around words and strong/italics chars. This regex (hopefully) clears this up + // Match any number of \*, followed by spaces/word boundaries against anything that is not the \*, followed by boundaries, spaces and * again. + // Test case at http://jsfiddle.net/ovqLv0s9/2/ + + var reAlignStars = /(\*+)(\s*\b)([^\*]+)(\b\s*)(\*+)/g; + + var lastOff=pOut.length; + for (var i=harmonized_attrs.length-1; i>=0; i--) { + var off=harmonized_attrs[i]; + + var raw_text = pOut.substring(off, lastOff) + + var d1 = ""; // @lmmx: build up a modifier prefix + var d2 = ""; // @lmmx: ...and suffix + + var end_font; + + var mark_bold = false; + var mark_italic = false; + var mark_code = false; + var mark_sup = false; + var mark_sub = false; + var mark_strike = false; + + // The end of the text block is a special case. + if (lastOff == pOut.length) { + end_font = reformatted_txt.getFontFamily(lastOff - 1) + if (end_font) { + if (!inSrc && end_font===end_font.COURIER_NEW) { + mark_code = true; + } + } + if (reformatted_txt.isBold(lastOff -1)) { + mark_bold = true; + } + if (reformatted_txt.isItalic(lastOff - 1)) { + // edbacher: changed this to handle bold italic properly. + mark_italic = true; + } + if (reformatted_txt.isStrikethrough(lastOff - 1)) { + mark_strike = true; + } + if (reformatted_txt.getTextAlignment(lastOff - 1)===DocumentApp.TextAlignment.SUPERSCRIPT) { + mark_sup = true; + } + if (reformatted_txt.getTextAlignment(lastOff - 1)===DocumentApp.TextAlignment.SUBSCRIPT) { + mark_sub = true; + } + } else { + end_font = reformatted_txt.getFontFamily(lastOff -1 ) + if (end_font) { + if (!inSrc && end_font===end_font.COURIER_NEW && reformatted_txt.getFontFamily(lastOff) != end_font) { + mark_code=true; + } + } + if (reformatted_txt.isBold(lastOff - 1) && !reformatted_txt.isBold(lastOff) ) { + mark_bold=true; + } + if (reformatted_txt.isStrikethrough(lastOff - 1) && !reformatted_txt.isStrikethrough(lastOff)) { + mark_strike=true; + } + if (reformatted_txt.isItalic(lastOff - 1) && !reformatted_txt.isItalic(lastOff)) { + mark_italic=true; + } + if (reformatted_txt.getTextAlignment(lastOff - 1)===DocumentApp.TextAlignment.SUPERSCRIPT) { + if (reformatted_txt.getTextAlignment(lastOff)!==DocumentApp.TextAlignment.SUPERSCRIPT) { + mark_sup = true; + } + } + if (reformatted_txt.getTextAlignment(lastOff - 1)===DocumentApp.TextAlignment.SUBSCRIPT) { + if (reformatted_txt.getTextAlignment(lastOff)!==DocumentApp.TextAlignment.SUBSCRIPT) { + mark_sub = true; + } + } + } + + if (mark_code) { + d2 = '`'; // shouldn't these go last? or will it interfere w/ reAlignStars? + } + if (mark_bold) { + d2 = "**" + d2; + } + if (mark_italic) { + d2 = "*" + d2; + } + if (mark_strike) { + d2 = "" + d2; + } + if (mark_sup) { + d2 = '' + d2; + } + if (mark_sub) { + d2 = '' + d2; + } + + mark_bold = mark_italic = mark_code = mark_sup = mark_sub = mark_strike = false; + + var font=reformatted_txt.getFontFamily(off); + if (off == 0) { + if (font) { + if (!inSrc && font===font.COURIER_NEW) { + mark_code = true; + } + } + if (reformatted_txt.isBold(off)) { + mark_bold = true; + } + if (reformatted_txt.isItalic(off)) { + mark_italic = true; + } + if (reformatted_txt.isStrikethrough(off)) { + mark_strike = true; + } + if (reformatted_txt.getTextAlignment(off)===DocumentApp.TextAlignment.SUPERSCRIPT) { + mark_sup = true; + } + if (reformatted_txt.getTextAlignment(off)===DocumentApp.TextAlignment.SUBSCRIPT) { + mark_sub = true; + } + } else { + if (font) { + if (!inSrc && font===font.COURIER_NEW && reformatted_txt.getFontFamily(off - 1) != font) { + mark_code=true; + } + } + if (reformatted_txt.isBold(off) && !reformatted_txt.isBold(off -1) ) { + mark_bold=true; + } + if (reformatted_txt.isItalic(off) && !reformatted_txt.isItalic(off - 1)) { + mark_italic=true; + } + if (reformatted_txt.isStrikethrough(off) && !reformatted_txt.isStrikethrough(off - 1)) { + mark_strike=true; + } + if (reformatted_txt.getTextAlignment(off)===DocumentApp.TextAlignment.SUPERSCRIPT) { + if (reformatted_txt.getTextAlignment(off - 1)!==DocumentApp.TextAlignment.SUPERSCRIPT) { + mark_sup = true; + } + } + if (reformatted_txt.getTextAlignment(off)===DocumentApp.TextAlignment.SUBSCRIPT) { + if (reformatted_txt.getTextAlignment(off - 1)!==DocumentApp.TextAlignment.SUBSCRIPT) { + mark_sub = true; + } + } + } + + + if (mark_code) { + d1 = '`'; + } + + if (mark_bold) { + d1 = d1 + "**"; + } + + if (mark_italic) { + d1 = d1 + "*"; + } + + if (mark_sup) { + d1 = d1 + ''; + } + + if (mark_sub) { + d1 = d1 + ''; + } + + if (mark_strike) { + d1 = d1 + ''; + } + + var url=reformatted_txt.getLinkUrl(off); + if (url) { + mOut = d1 + '['+ raw_text +']('+url+')' + d2 + mOut; + } else { + var new_text = d1 + raw_text + d2; + new_text = new_text.replace(reAlignStars, "$2$1$3$5$4"); + mOut = new_text + mOut; + } + + lastOff=off; +// Logger.log("Modified String: " + mOut) + } + + mOut = pOut.substring(0, off) + mOut; + return mOut; +} \ No newline at end of file diff --git a/demos/palm/python/docs-agent/apps_script/gmail_to_markdown.gs b/demos/palm/python/docs-agent/apps_script/gmail_to_markdown.gs new file mode 100644 index 000000000..3263ef100 --- /dev/null +++ b/demos/palm/python/docs-agent/apps_script/gmail_to_markdown.gs @@ -0,0 +1,137 @@ +/** + * Copyright 2023 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +function exportEmailsToMarkdown(search, folderName) { + //Checks if input folder exists or exits + if(folderExistsOrCreate(folderName)){ + var output_file_name = folderName + "-index"; + var folderOutputObj = DriveApp.getFoldersByName(folderName); + if (folderOutputObj.hasNext()){ + var folderOutputName = folderOutputObj.next(); + } + var sheet = checkIndexOutputOrCreate(output_file_name, folderOutputName); + console.log(`Searching for: "${search}"`); + var start = 0; + var max = 500; + var threads = GmailApp.search(search, start, max); + var threadMax = threads.length; + if (threads!=null){ + console.log(threadMax + " threads found."); + } else { + console.warn("No threads found with the search criteria"); + return; + } + let timeZone = Session.getScriptTimeZone(); + let created_date = Utilities.formatDate(new Date(), timeZone, "MM-dd-yyyy HH:mm:ss z"); + sheet.appendRow(["Created: ", created_date]) + sheet.appendRow(["Date", "From", "Subject", "To", "Markdown ID", "Markdown URL", "Full date", "MD5 hash", "Status"]); + var start_data_row = 2; + var status = "New content"; + var newEmails = 0; + var unchangedEmails = 0; + for (var threadCount in threads) { + var msgs = threads[threadCount].getMessages(); + Logger.log("Processing thread " + threadCount + " of " + threadMax); + for (var msgCount in msgs) { + var msg = msgs[msgCount]; + var subject = msg.getSubject().replace(/"/g, "\\\"");; + // Removes replies and forwards - Can mostly be noise. + if(!subject.toLowerCase().includes("re:") && + !subject.toLowerCase().includes("fwd:") && + !subject.toLowerCase().includes("forwarded message")){ + // Values to get and store messages + var date = msg.getDate(); + let from_author = msg.getFrom().replace(/"/g, "\\\""); + var hash_content = from_author + date + subject; + let sanitized_subject = sanitizeString(subject); + let date_format = Utilities.formatDate(date, "PST", "MM-dd-yyyy"); + let to = msg.getTo(); + let to_array = to.split(", "); + for (i in to_array) { + to_array[i] = "\"" + to_array[i].replace(/^" "/, "").replace(/"/g, "\\\"") + "\""; + } + let md5_hash = Utilities.computeDigest(Utilities.DigestAlgorithm.MD5,hash_content, + Utilities.Charset.US_ASCII); + let hash_str = byteToStr(md5_hash); + //Function returns an array, assign each array value to seperate variables. For emails, only need to retrieve + // backup markdown ids + var backup_results = returnBackupHash(sheet, "Backup", hash_str, start_data_row, 7, 4, 5); + if (backup_results != undefined && backup_results[0] != "no_results") { + Logger.log("Email is already in markdown format. Skipping conversion."); + var status = "Unchanged content"; + var markdown_id = backup_results[1]; + if (markdown_id){ + var md_file = DriveApp.getFileById(markdown_id); + } + unchangedEmails += 1; + } + else { + var status = "New content"; + let message = msg.getPlainBody(); + let filename = sanitizeFileName(date_format + subject + ".md"); + // Initialize blank text since file will get updated with URL + let email_md = ""; + // Add count here for emails + newEmails += 1; + Logger.log("Email number: " + newEmails + "| Saving email to: " + filename); + var body = "# " + subject + "\n"; + // Cleans the reply part of emails + body += regexToCleanCharsMD(sanitizeBody(message.replace(/^>/g,""))) + "\n"; + var destinationFolder = DriveApp.getFoldersByName(folderOutputName).next(); + // Initialize blank file to retrieve URL which is then added to the frontmatter + var destinationFile = destinationFolder.createFile(filename, email_md , MimeType.PLAIN_TEXT); + // Create metadata for the object + var markdown_id = destinationFile.getId(); + var md_file = DriveApp.getFileById(markdown_id); + let md_url = md_file.getUrl(); + let frontmatter = "---" + "\n"; + frontmatter += "title: \"" + sanitized_subject + "\"\n"; + frontmatter += "type: \"" + "email" + "\"\n"; + frontmatter += "URL: \"" + md_url + "\"\n"; + frontmatter += "created: \"" + date + "\"\n"; + frontmatter += "from: \"" + from_author + "\"\n"; + frontmatter += "to: \[" + to_array + "\]\n"; + frontmatter += "---" + "\n\n"; + email_md = frontmatter + body; + var encoded = Utilities.base64Encode(email_md); + var byteDataArray = Utilities.base64Decode(encoded); + var textAsBlob = Utilities.newBlob(byteDataArray); + Drive.Files.update(null,markdown_id, textAsBlob); + } + let md_chip = createRichText(md_file); + metadata = [ + date_format, + from_author, + sanitized_subject, + to, + markdown_id, + "md_chip", + date, + hash_str, + status, + ]; + sheet.appendRow(metadata); + var emailTotal = newEmails + unchangedEmails; + let row_number = emailTotal + start_data_row; + insertRichText(sheet, md_chip, "F", row_number); + } + } + } + Logger.log("Saved a total of " + newEmails + " new emails."); + Logger.log("There is a total of " + unchangedEmails + " unchanged emails."); + Logger.log("Grand total of " + emailTotal + " emails."); + } +} \ No newline at end of file diff --git a/demos/palm/python/docs-agent/apps_script/helper_functions.gs b/demos/palm/python/docs-agent/apps_script/helper_functions.gs new file mode 100644 index 000000000..129e74329 --- /dev/null +++ b/demos/palm/python/docs-agent/apps_script/helper_functions.gs @@ -0,0 +1,183 @@ +/** + * Copyright 2023 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +// Checks to see if a folder already exists in the drive +function folderExists(folderName) { + const folderIterator = DriveApp.getRootFolder().getFoldersByName(folderName); + if(folderIterator.hasNext()) { + return true; + } + else { + return false; + } +} + +// Checks to see if a folder already exists in the drive and exits if it doesn't. Useful for input directories +function folderExistsInput(folderName){ + if (folderExists(folderName)) { + Logger.log("Folder exists: "+ folderName); + return true; + } + else { + Logger.log("Folder does not exist: "+ folderName + ". Please make sure the directory exists."); + return false; + } +} + +// Checks to see if folder exists or creates it. Useful for output directories +function folderExistsOrCreate(folderName){ + if(folderExists(folderName)) { + Logger.log("Folder exists: "+ folderName); + return true; + } + else { + Logger.log("Folder does not exist: "+ folderName + ". Creating the directory."); + DriveApp.createFolder(folderName); + return true; + } +} +// Checks to see if a file exists in a folder +function checkFileExists(fileName,folderName){ + let folder = DriveApp.getFoldersByName(folderName); + if(!folder.hasNext()){ + } + else{ + var file = folder.next().getFilesByName(fileName); + if(!file.hasNext()){ + return true; + } + else{ + return false; + } + } +} + +// Function to check if an index output sheet exists or creates it. Returns the file object +// Specify the file output name and outputdirectory +function checkIndexOutputOrCreate(fileName, folderOutput) { + var timeZone = Session.getScriptTimeZone(); + var date = Utilities.formatDate(new Date(), timeZone, "MM-dd-yyyy hh:mm:ss"); + let file = {title: fileName, mimeType: MimeType.GOOGLE_SHEETS, parents: [{id: folderOutput.getId()}]} + let params = "title='" + fileName + "' and parents in '" + folderOutput.getId() + "'"; + let file_search = DriveApp.searchFiles(params); + if (file_search.hasNext()) { + let fileId = file_search.next().getId(); + var sheet = SpreadsheetApp.openById(fileId); + Logger.log("File index: " + fileName + " exists."); + var sheet_index = sheet.getSheetByName("Index"); + if (sheet.getSheetByName("Backup")){ + var sheet_backup = sheet.getSheetByName("Backup"); + sheet.deleteSheet(sheet_backup); + } + var sheet_backup = sheet.insertSheet("Backup", 1); + var sheet_backup_open = sheet.getSheetByName("Backup"); + sheet_index.getDataRange().copyTo(sheet_backup_open.getRange(1,1)); + if (sheet_index != null){ + sheet.deleteSheet(sheet_index); + } + sheet.insertSheet("Index", 0); + sheet_index = sheet.getSheetByName("Index"); + sheet_index.addDeveloperMetadata("Date", date); + } + else { + Logger.log("File index: " + fileName + " does not exist."); + let output = Drive.Files.insert(file).id; + var sheet = SpreadsheetApp.openById(output); + var sheet_1 = sheet.getSheetByName("Sheet1"); + sheet.insertSheet("Index", 0); + var sheet_index = sheet.getSheetByName("Index") + sheet_index.addDeveloperMetadata("Date", date); + sheet.deleteSheet(sheet_1); + } + return sheet; +} + +// Function to convert byte array into a string +function byteToStr(byteInput){ + let signatureStr = ''; + for (i = 0; i < byteInput.length; i++) { + let byte = byteInput[i]; + if (byte < 0) + byte += 256; + let byteStr = byte.toString(16); + if (byteStr.length == 1) byteStr = '0' + byteStr; + signatureStr += byteStr; + } +return signatureStr; +} + +// Function to remove special characters for file names +function sanitizeFileName(fileName){ + let clean_filename = fileName.replace(/\[/g, "_").replace(/\]/g, "_").replace(/\(/g, "_").replace(/\)/g, "_").replace(/^_/g, "").replace(/,/g, "_").replace(/ /g, "_").replace(/:/g, "").replace(/`/g, "").replace(/\'/g, "").replace(/&/g, "and").replace(//g, "").replace(/’/g, ""); +return clean_filename; +} + +// Function to remove special characters for file names +function sanitizeString(string){ + let clean_string = string.replace(/\[/g, "").replace(/\]/g, "").replace(/\(/g, "").replace(/\)/g, "").replace(/^_/g, "").replace(/,/g, " ").replace(/:/g, "").replace(/`/g, "").replace(/\'/g, "").replace(/&/g, "and").replace(//g, ""); +return clean_string; +} + +function sanitizeBody(string){ + let clean_body = string.replace(/’/g, "'").replace(/^M/g, ""); +return clean_body; +} + +function regexToCleanCharsMD(string){ + let clean_string = string.replace(/(\*+)(\s*\b)([^\*]+)(\b\s*)(\*+)/g, "$2$1$3$5$4"); +return clean_string; +} + +// Function to check if a backup sheet exists and return a hash if the file exists +// Specify the sheet name where the backup is saved, default is "Backup" +// From your backup sheet specify the column that contains the MD5 hash +// and the columns for which you return values +function returnBackupHash(sheet, sheet_name, fid, start_data_row, pos_id, pos_1_col, pos_2_col){ + if (sheet.getSheetByName(sheet_name)){ + let backup_sheet = sheet.getSheetByName(sheet_name); + if(backup_sheet.getLastRow()> start_data_row){ + let backup_values = backup_sheet.getDataRange().getValues(); + for (let row_count = start_data_row; row_count < backup_sheet.getLastRow(); row_count++) { + let row_id = backup_values[row_count][pos_id]; + let pos_1_value = backup_values[row_count][pos_1_col]; + //Retrieve id of existing markdown conversion + let pos_2_value = backup_values[row_count][pos_2_col]; + if (row_id == fid){ + var results = [row_id, pos_1_value, pos_2_value]; + break; + } + else { + var results = ["no_results"]; + } + } + return results; + } + } +} + +// Creates a richText item with item. +function createRichText (item){ + let title = item.getName(); + let url = item.getUrl(); + let richText = SpreadsheetApp.newRichTextValue().setText(title).setLinkUrl(url).build(); + return richText; +} + +// Insert a richText item in a specific cell +function insertRichText (sheetItem, item, column, row){ + let range = sheetItem.getRange(column + row); + range.setRichTextValue(item); +} diff --git a/demos/palm/python/docs-agent/apps_script/main.gs b/demos/palm/python/docs-agent/apps_script/main.gs new file mode 100644 index 000000000..2fe88de33 --- /dev/null +++ b/demos/palm/python/docs-agent/apps_script/main.gs @@ -0,0 +1,27 @@ +/** + * Copyright 2023 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +// Defines the gmail search query for saving emails to markdown +var SEARCH_QUERY = 'subject: psa to:my-mailing-list@example.com'; +// Defines the directory to output the emails in markdown format +var folderOutput = "PSA-output" +// Defines the directory that has your docs content +var folderInput = "input-folder" + +function main() { + convertDriveFolderToMDForDocsAgent(folderInput); + exportEmailsToMarkdown(SEARCH_QUERY, folderOutput); +} \ No newline at end of file diff --git a/demos/palm/python/docs-agent/config.yaml b/demos/palm/python/docs-agent/config.yaml index 5b10185bb..21f2e78bc 100644 --- a/demos/palm/python/docs-agent/config.yaml +++ b/demos/palm/python/docs-agent/config.yaml @@ -71,7 +71,7 @@ input: # models are unable to provide responses. # condition_text: "You are a helpful chatbot answering questions from users. Read -the following context firstand answer the question at the end:" +the following context first and answer the question at the end:" fact_check_question: "Can you compare the following text to the context provided in this prompt and write a short message that warns the readers about which part diff --git a/demos/palm/python/docs-agent/poetry.lock b/demos/palm/python/docs-agent/poetry.lock index b1c17c48c..c5ddf9eeb 100644 --- a/demos/palm/python/docs-agent/poetry.lock +++ b/demos/palm/python/docs-agent/poetry.lock @@ -13,13 +13,13 @@ files = [ [[package]] name = "annotated-types" -version = "0.5.0" +version = "0.6.0" description = "Reusable constraint types to use with typing.Annotated" optional = false -python-versions = ">=3.7" +python-versions = ">=3.8" files = [ - {file = "annotated_types-0.5.0-py3-none-any.whl", hash = "sha256:58da39888f92c276ad970249761ebea80ba544b77acddaa1a4d6cf78287d45fd"}, - {file = "annotated_types-0.5.0.tar.gz", hash = "sha256:47cdc3490d9ac1506ce92c7aaa76c579dc3509ff11e098fc867e5130ab7be802"}, + {file = "annotated_types-0.6.0-py3-none-any.whl", hash = "sha256:0641064de18ba7a25dee8f96403ebc39113d0cb953a01429249d5c7564666a43"}, + {file = "annotated_types-0.6.0.tar.gz", hash = "sha256:563339e807e53ffd9c267e99fc6d9ea23eb8443c08f112651963e24e22f84a5d"}, ] [[package]] @@ -56,13 +56,14 @@ files = [ [[package]] name = "array-record" -version = "0.4.1" +version = "0.5.0" description = "A file format that achieves a new frontier of IO efficiency" optional = false python-versions = ">=3.9" files = [ - {file = "array_record-0.4.1-py310-none-any.whl", hash = "sha256:6a0c8ed6fdfaaf2cecd3d5c6b9c13e116ad3299649611c8fd184d64557fbaba8"}, - {file = "array_record-0.4.1-py39-none-any.whl", hash = "sha256:a74e9c0075860777b79e4b3ac278f67add270acf78520d3b9cf8c325aef42951"}, + {file = "array_record-0.5.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:7940fda50bbd1ac6d5f0c3c3d4c885dc886bfb3622a123a8e8fd798582303d8a"}, + {file = "array_record-0.5.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:ac09a423710864f6225ebc45a307c30b419fde4470d2e5f48ba2b1e99bbe84d3"}, + {file = "array_record-0.5.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:07330e56e7bb1354b5fb9f974dc3c42e4a3cd6e8dcf4c6e6a8b84db14ee9a8ed"}, ] [package.dependencies] @@ -196,33 +197,29 @@ lxml = ["lxml"] [[package]] name = "black" -version = "23.9.1" +version = "23.10.0" description = "The uncompromising code formatter." optional = false python-versions = ">=3.8" files = [ - {file = "black-23.9.1-cp310-cp310-macosx_10_16_arm64.whl", hash = "sha256:d6bc09188020c9ac2555a498949401ab35bb6bf76d4e0f8ee251694664df6301"}, - {file = "black-23.9.1-cp310-cp310-macosx_10_16_universal2.whl", hash = "sha256:13ef033794029b85dfea8032c9d3b92b42b526f1ff4bf13b2182ce4e917f5100"}, - {file = "black-23.9.1-cp310-cp310-macosx_10_16_x86_64.whl", hash = "sha256:75a2dc41b183d4872d3a500d2b9c9016e67ed95738a3624f4751a0cb4818fe71"}, - {file = "black-23.9.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:13a2e4a93bb8ca74a749b6974925c27219bb3df4d42fc45e948a5d9feb5122b7"}, - {file = "black-23.9.1-cp310-cp310-win_amd64.whl", hash = "sha256:adc3e4442eef57f99b5590b245a328aad19c99552e0bdc7f0b04db6656debd80"}, - {file = "black-23.9.1-cp311-cp311-macosx_10_16_arm64.whl", hash = "sha256:8431445bf62d2a914b541da7ab3e2b4f3bc052d2ccbf157ebad18ea126efb91f"}, - {file = "black-23.9.1-cp311-cp311-macosx_10_16_universal2.whl", hash = "sha256:8fc1ddcf83f996247505db6b715294eba56ea9372e107fd54963c7553f2b6dfe"}, - {file = "black-23.9.1-cp311-cp311-macosx_10_16_x86_64.whl", hash = "sha256:7d30ec46de88091e4316b17ae58bbbfc12b2de05e069030f6b747dfc649ad186"}, - {file = "black-23.9.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:031e8c69f3d3b09e1aa471a926a1eeb0b9071f80b17689a655f7885ac9325a6f"}, - {file = "black-23.9.1-cp311-cp311-win_amd64.whl", hash = "sha256:538efb451cd50f43aba394e9ec7ad55a37598faae3348d723b59ea8e91616300"}, - {file = "black-23.9.1-cp38-cp38-macosx_10_16_arm64.whl", hash = "sha256:638619a559280de0c2aa4d76f504891c9860bb8fa214267358f0a20f27c12948"}, - {file = "black-23.9.1-cp38-cp38-macosx_10_16_universal2.whl", hash = "sha256:a732b82747235e0542c03bf352c126052c0fbc458d8a239a94701175b17d4855"}, - {file = "black-23.9.1-cp38-cp38-macosx_10_16_x86_64.whl", hash = "sha256:cf3a4d00e4cdb6734b64bf23cd4341421e8953615cba6b3670453737a72ec204"}, - {file = "black-23.9.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:cf99f3de8b3273a8317681d8194ea222f10e0133a24a7548c73ce44ea1679377"}, - {file = "black-23.9.1-cp38-cp38-win_amd64.whl", hash = "sha256:14f04c990259576acd093871e7e9b14918eb28f1866f91968ff5524293f9c573"}, - {file = "black-23.9.1-cp39-cp39-macosx_10_16_arm64.whl", hash = "sha256:c619f063c2d68f19b2d7270f4cf3192cb81c9ec5bc5ba02df91471d0b88c4c5c"}, - {file = "black-23.9.1-cp39-cp39-macosx_10_16_universal2.whl", hash = "sha256:6a3b50e4b93f43b34a9d3ef00d9b6728b4a722c997c99ab09102fd5efdb88325"}, - {file = "black-23.9.1-cp39-cp39-macosx_10_16_x86_64.whl", hash = "sha256:c46767e8df1b7beefb0899c4a95fb43058fa8500b6db144f4ff3ca38eb2f6393"}, - {file = "black-23.9.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:50254ebfa56aa46a9fdd5d651f9637485068a1adf42270148cd101cdf56e0ad9"}, - {file = "black-23.9.1-cp39-cp39-win_amd64.whl", hash = "sha256:403397c033adbc45c2bd41747da1f7fc7eaa44efbee256b53842470d4ac5a70f"}, - {file = "black-23.9.1-py3-none-any.whl", hash = "sha256:6ccd59584cc834b6d127628713e4b6b968e5f79572da66284532525a042549f9"}, - {file = "black-23.9.1.tar.gz", hash = "sha256:24b6b3ff5c6d9ea08a8888f6977eae858e1f340d7260cf56d70a49823236b62d"}, + {file = "black-23.10.0-cp310-cp310-macosx_10_16_arm64.whl", hash = "sha256:f8dc7d50d94063cdfd13c82368afd8588bac4ce360e4224ac399e769d6704e98"}, + {file = "black-23.10.0-cp310-cp310-macosx_10_16_x86_64.whl", hash = "sha256:f20ff03f3fdd2fd4460b4f631663813e57dc277e37fb216463f3b907aa5a9bdd"}, + {file = "black-23.10.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:d3d9129ce05b0829730323bdcb00f928a448a124af5acf90aa94d9aba6969604"}, + {file = "black-23.10.0-cp310-cp310-win_amd64.whl", hash = "sha256:960c21555be135c4b37b7018d63d6248bdae8514e5c55b71e994ad37407f45b8"}, + {file = "black-23.10.0-cp311-cp311-macosx_10_16_arm64.whl", hash = "sha256:30b78ac9b54cf87bcb9910ee3d499d2bc893afd52495066c49d9ee6b21eee06e"}, + {file = "black-23.10.0-cp311-cp311-macosx_10_16_x86_64.whl", hash = "sha256:0e232f24a337fed7a82c1185ae46c56c4a6167fb0fe37411b43e876892c76699"}, + {file = "black-23.10.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:31946ec6f9c54ed7ba431c38bc81d758970dd734b96b8e8c2b17a367d7908171"}, + {file = "black-23.10.0-cp311-cp311-win_amd64.whl", hash = "sha256:c870bee76ad5f7a5ea7bd01dc646028d05568d33b0b09b7ecfc8ec0da3f3f39c"}, + {file = "black-23.10.0-cp38-cp38-macosx_10_16_arm64.whl", hash = "sha256:6901631b937acbee93c75537e74f69463adaf34379a04eef32425b88aca88a23"}, + {file = "black-23.10.0-cp38-cp38-macosx_10_16_x86_64.whl", hash = "sha256:481167c60cd3e6b1cb8ef2aac0f76165843a374346aeeaa9d86765fe0dd0318b"}, + {file = "black-23.10.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:f74892b4b836e5162aa0452393112a574dac85e13902c57dfbaaf388e4eda37c"}, + {file = "black-23.10.0-cp38-cp38-win_amd64.whl", hash = "sha256:47c4510f70ec2e8f9135ba490811c071419c115e46f143e4dce2ac45afdcf4c9"}, + {file = "black-23.10.0-cp39-cp39-macosx_10_16_arm64.whl", hash = "sha256:76baba9281e5e5b230c9b7f83a96daf67a95e919c2dfc240d9e6295eab7b9204"}, + {file = "black-23.10.0-cp39-cp39-macosx_10_16_x86_64.whl", hash = "sha256:a3c2ddb35f71976a4cfeca558848c2f2f89abc86b06e8dd89b5a65c1e6c0f22a"}, + {file = "black-23.10.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:db451a3363b1e765c172c3fd86213a4ce63fb8524c938ebd82919bf2a6e28c6a"}, + {file = "black-23.10.0-cp39-cp39-win_amd64.whl", hash = "sha256:7fb5fc36bb65160df21498d5a3dd330af8b6401be3f25af60c6ebfe23753f747"}, + {file = "black-23.10.0-py3-none-any.whl", hash = "sha256:e223b731a0e025f8ef427dd79d8cd69c167da807f5710add30cdf131f13dd62e"}, + {file = "black-23.10.0.tar.gz", hash = "sha256:31b9f87b277a68d0e99d2905edae08807c007973eaa609da5f0c62def6b7c0bd"}, ] [package.dependencies] @@ -242,13 +239,13 @@ uvloop = ["uvloop (>=0.15.2)"] [[package]] name = "blinker" -version = "1.6.2" +version = "1.6.3" description = "Fast, simple object-to-object and broadcast signaling" optional = false python-versions = ">=3.7" files = [ - {file = "blinker-1.6.2-py3-none-any.whl", hash = "sha256:c3d739772abb7bc2860abf5f2ec284223d9ad5c76da018234f6f50d6f31ab1f0"}, - {file = "blinker-1.6.2.tar.gz", hash = "sha256:4afd3de66ef3a9f8067559fb7a1cbe555c17dcbe15971b05d1b625c3e7abe213"}, + {file = "blinker-1.6.3-py3-none-any.whl", hash = "sha256:296320d6c28b006eb5e32d4712202dbcdcbf5dc482da298c2f44881c43884aaa"}, + {file = "blinker-1.6.3.tar.gz", hash = "sha256:152090d27c1c5c722ee7e48504b02d76502811ce02e1523553b4cf8c8b3d3a8d"}, ] [[package]] @@ -374,13 +371,13 @@ files = [ [[package]] name = "chex" -version = "0.1.83" +version = "0.1.84" description = "Chex: Testing made fun, in JAX!" optional = false python-versions = ">=3.9" files = [ - {file = "chex-0.1.83-py3-none-any.whl", hash = "sha256:d51485018b064c0cca2b9db8e96a74368ab4bea96f023fa95b916ada1ec27b0e"}, - {file = "chex-0.1.83.tar.gz", hash = "sha256:8b8eb8d435594ff62f54be919b4b40795a6d2c35b800876f02f17f47643ceb6f"}, + {file = "chex-0.1.84-py3-none-any.whl", hash = "sha256:a41603ed1a3c1d59c15aa017238b36b2437c82ef0a69d5bc4b1140fb32610ea5"}, + {file = "chex-0.1.84.tar.gz", hash = "sha256:380ebf7a57ec65311656e1924261ba4c5c4860c5f9f5f71b906de3279f46b451"}, ] [package.dependencies] @@ -711,13 +708,13 @@ files = [ [[package]] name = "etils" -version = "1.5.0" +version = "1.5.1" description = "Collection of common python utils" optional = false python-versions = ">=3.9" files = [ - {file = "etils-1.5.0-py3-none-any.whl", hash = "sha256:e28bf37a0ee20e81d6cf301ce64574763cc4ade8580a2b3276b790993c167d12"}, - {file = "etils-1.5.0.tar.gz", hash = "sha256:8f4d7cd4ced7a888489cf52a1470457695a9988fa05e767c44a3a5b0f395dd43"}, + {file = "etils-1.5.1-py3-none-any.whl", hash = "sha256:2c1bfa2817eb4881cb509097f1e65ac6160126ba74ec47b3bb47ee678628d8c8"}, + {file = "etils-1.5.1.tar.gz", hash = "sha256:b530c0d1b2ed1b8da1af367d4b97891e680a6a4658d4190183210bbbb8cf1fb9"}, ] [package.dependencies] @@ -737,8 +734,8 @@ ecolab = ["etils[enp]", "etils[epy]", "jupyter", "mediapy", "numpy"] edc = ["etils[epy]"] enp = ["etils[epy]", "numpy"] epath = ["etils[epy]", "fsspec", "importlib_resources", "typing_extensions", "zipp"] -epath-gcs = ["gcsfs"] -epath-s3 = ["s3fs"] +epath-gcs = ["etils[epath]", "gcsfs"] +epath-s3 = ["etils[epath]", "s3fs"] epy = ["typing_extensions"] etqdm = ["absl-py", "etils[epy]", "tqdm"] etree = ["etils[array-types]", "etils[enp]", "etils[epy]", "etils[etqdm]"] @@ -777,20 +774,20 @@ tests = ["asttokens (>=2.1.0)", "coverage", "coverage-enable-subprocess", "ipyth [[package]] name = "fastapi" -version = "0.103.2" +version = "0.104.0" description = "FastAPI framework, high performance, easy to learn, fast to code, ready for production" optional = false -python-versions = ">=3.7" +python-versions = ">=3.8" files = [ - {file = "fastapi-0.103.2-py3-none-any.whl", hash = "sha256:3270de872f0fe9ec809d4bd3d4d890c6d5cc7b9611d721d6438f9dacc8c4ef2e"}, - {file = "fastapi-0.103.2.tar.gz", hash = "sha256:75a11f6bfb8fc4d2bec0bd710c2d5f2829659c0e8c0afd5560fdda6ce25ec653"}, + {file = "fastapi-0.104.0-py3-none-any.whl", hash = "sha256:456482c1178fb7beb2814b88e1885bc49f9a81f079665016feffe3e1c6a7663e"}, + {file = "fastapi-0.104.0.tar.gz", hash = "sha256:9c44de45693ae037b0c6914727a29c49a40668432b67c859a87851fc6a7b74c6"}, ] [package.dependencies] anyio = ">=3.7.1,<4.0.0" pydantic = ">=1.7.4,<1.8 || >1.8,<1.8.1 || >1.8.1,<2.0.0 || >2.0.0,<2.0.1 || >2.0.1,<2.1.0 || >2.1.0,<3.0.0" starlette = ">=0.27.0,<0.28.0" -typing-extensions = ">=4.5.0" +typing-extensions = ">=4.8.0" [package.extras] all = ["email-validator (>=2.0.0)", "httpx (>=0.23.0)", "itsdangerous (>=1.1.0)", "jinja2 (>=2.11.2)", "orjson (>=3.2.1)", "pydantic-extra-types (>=2.0.0)", "pydantic-settings (>=2.0.0)", "python-multipart (>=0.0.5)", "pyyaml (>=5.3.1)", "ujson (>=4.0.1,!=4.0.2,!=4.1.0,!=4.2.0,!=4.3.0,!=5.0.0,!=5.1.0)", "uvicorn[standard] (>=0.12.0)"] @@ -992,13 +989,13 @@ grpcio-gcp = ["grpcio-gcp (>=0.2.2,<1.0.dev0)"] [[package]] name = "google-auth" -version = "2.23.2" +version = "2.23.3" description = "Google Authentication Library" optional = false python-versions = ">=3.7" files = [ - {file = "google-auth-2.23.2.tar.gz", hash = "sha256:5a9af4be520ba33651471a0264eead312521566f44631cbb621164bc30c8fd40"}, - {file = "google_auth-2.23.2-py2.py3-none-any.whl", hash = "sha256:c2e253347579d483004f17c3bd0bf92e611ef6c7ba24d41c5c59f2e7aeeaf088"}, + {file = "google-auth-2.23.3.tar.gz", hash = "sha256:6864247895eea5d13b9c57c9e03abb49cb94ce2dc7c58e91cba3248c7477c9e3"}, + {file = "google_auth-2.23.3-py2.py3-none-any.whl", hash = "sha256:a8f4608e65c244ead9e0538f181a96c6e11199ec114d41f1d7b1bffa96937bda"}, ] [package.dependencies] @@ -1064,13 +1061,13 @@ six = "*" [[package]] name = "googleapis-common-protos" -version = "1.60.0" +version = "1.61.0" description = "Common protobufs used in Google APIs" optional = false python-versions = ">=3.7" files = [ - {file = "googleapis-common-protos-1.60.0.tar.gz", hash = "sha256:e73ebb404098db405ba95d1e1ae0aa91c3e15a71da031a2eeb6b2e23e7bc3708"}, - {file = "googleapis_common_protos-1.60.0-py2.py3-none-any.whl", hash = "sha256:69f9bbcc6acde92cab2db95ce30a70bd2b81d20b12eff3f1aabaffcbe8a93918"}, + {file = "googleapis-common-protos-1.61.0.tar.gz", hash = "sha256:8a64866a97f6304a7179873a465d6eee97b7a24ec6cfd78e0f575e96b821240b"}, + {file = "googleapis_common_protos-1.61.0-py2.py3-none-any.whl", hash = "sha256:22f1915393bb3245343f6efe87f6fe868532efc12aa26b391b15132e1279f1c0"}, ] [package.dependencies] @@ -1242,32 +1239,36 @@ files = [ [[package]] name = "h5py" -version = "3.9.0" +version = "3.10.0" description = "Read and write HDF5 files from Python" optional = false python-versions = ">=3.8" files = [ - {file = "h5py-3.9.0-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:eb7bdd5e601dd1739698af383be03f3dad0465fe67184ebd5afca770f50df9d6"}, - {file = "h5py-3.9.0-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:78e44686334cbbf2dd21d9df15823bc38663f27a3061f6a032c68a3e30c47bf7"}, - {file = "h5py-3.9.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:f68b41efd110ce9af1cbe6fa8af9f4dcbadace6db972d30828b911949e28fadd"}, - {file = "h5py-3.9.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:12aa556d540f11a2cae53ea7cfb94017353bd271fb3962e1296b342f6550d1b8"}, - {file = "h5py-3.9.0-cp310-cp310-win_amd64.whl", hash = "sha256:d97409e17915798029e297a84124705c8080da901307ea58f29234e09b073ddc"}, - {file = "h5py-3.9.0-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:551e358db05a874a0f827b22e95b30092f2303edc4b91bb62ad2f10e0236e1a0"}, - {file = "h5py-3.9.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:6822a814b9d8b8363ff102f76ea8d026f0ca25850bb579d85376029ee3e73b93"}, - {file = "h5py-3.9.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:54f01202cdea754ab4227dd27014bdbd561a4bbe4b631424fd812f7c2ce9c6ac"}, - {file = "h5py-3.9.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:64acceaf6aff92af091a4b83f6dee3cf8d3061f924a6bb3a33eb6c4658a8348b"}, - {file = "h5py-3.9.0-cp311-cp311-win_amd64.whl", hash = "sha256:804c7fb42a34c8ab3a3001901c977a5c24d2e9c586a0f3e7c0a389130b4276fc"}, - {file = "h5py-3.9.0-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:8d9492391ff5c3c80ec30ae2fe82a3f0efd1e750833739c25b0d090e3be1b095"}, - {file = "h5py-3.9.0-cp38-cp38-macosx_11_0_arm64.whl", hash = "sha256:9da9e7e63376c32704e37ad4cea2dceae6964cee0d8515185b3ab9cbd6b947bc"}, - {file = "h5py-3.9.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:a4e20897c88759cbcbd38fb45b507adc91af3e0f67722aa302d71f02dd44d286"}, - {file = "h5py-3.9.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:dbf5225543ca35ce9f61c950b73899a82be7ba60d58340e76d0bd42bf659235a"}, - {file = "h5py-3.9.0-cp38-cp38-win_amd64.whl", hash = "sha256:36408f8c62f50007d14e000f9f3acf77e103b9e932c114cbe52a3089e50ebf94"}, - {file = "h5py-3.9.0-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:23e74b878bbe1653ab34ca49b83cac85529cd0b36b9d625516c5830cc5ca2eac"}, - {file = "h5py-3.9.0-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:3f457089c5d524b7998e3649bc63240679b8fb0a3859ea53bbb06841f3d755f1"}, - {file = "h5py-3.9.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:a6284061f3214335e1eec883a6ee497dbe7a79f19e6a57fed2dd1f03acd5a8cb"}, - {file = "h5py-3.9.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:95f7a745efd0d56076999b52e8da5fad5d30823bac98b59c68ae75588d09991a"}, - {file = "h5py-3.9.0-cp39-cp39-win_amd64.whl", hash = "sha256:79bbca34696c6f9eeeb36a91776070c49a060b2879828e2c8fa6c58b8ed10dd1"}, - {file = "h5py-3.9.0.tar.gz", hash = "sha256:e604db6521c1e367c6bd7fad239c847f53cc46646f2d2651372d05ae5e95f817"}, + {file = "h5py-3.10.0-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:b963fb772964fc1d1563c57e4e2e874022ce11f75ddc6df1a626f42bd49ab99f"}, + {file = "h5py-3.10.0-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:012ab448590e3c4f5a8dd0f3533255bc57f80629bf7c5054cf4c87b30085063c"}, + {file = "h5py-3.10.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:781a24263c1270a62cd67be59f293e62b76acfcc207afa6384961762bb88ea03"}, + {file = "h5py-3.10.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:f42e6c30698b520f0295d70157c4e202a9e402406f50dc08f5a7bc416b24e52d"}, + {file = "h5py-3.10.0-cp310-cp310-win_amd64.whl", hash = "sha256:93dd840bd675787fc0b016f7a05fc6efe37312a08849d9dd4053fd0377b1357f"}, + {file = "h5py-3.10.0-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:2381e98af081b6df7f6db300cd88f88e740649d77736e4b53db522d8874bf2dc"}, + {file = "h5py-3.10.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:667fe23ab33d5a8a6b77970b229e14ae3bb84e4ea3382cc08567a02e1499eedd"}, + {file = "h5py-3.10.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:90286b79abd085e4e65e07c1bd7ee65a0f15818ea107f44b175d2dfe1a4674b7"}, + {file = "h5py-3.10.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:6c013d2e79c00f28ffd0cc24e68665ea03ae9069e167087b2adb5727d2736a52"}, + {file = "h5py-3.10.0-cp311-cp311-win_amd64.whl", hash = "sha256:92273ce69ae4983dadb898fd4d3bea5eb90820df953b401282ee69ad648df684"}, + {file = "h5py-3.10.0-cp312-cp312-macosx_10_9_x86_64.whl", hash = "sha256:3c97d03f87f215e7759a354460fb4b0d0f27001450b18b23e556e7856a0b21c3"}, + {file = "h5py-3.10.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:86df4c2de68257b8539a18646ceccdcf2c1ce6b1768ada16c8dcfb489eafae20"}, + {file = "h5py-3.10.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:ba9ab36be991119a3ff32d0c7cbe5faf9b8d2375b5278b2aea64effbeba66039"}, + {file = "h5py-3.10.0-cp312-cp312-win_amd64.whl", hash = "sha256:2c8e4fda19eb769e9a678592e67eaec3a2f069f7570c82d2da909c077aa94339"}, + {file = "h5py-3.10.0-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:492305a074327e8d2513011fa9fffeb54ecb28a04ca4c4227d7e1e9616d35641"}, + {file = "h5py-3.10.0-cp38-cp38-macosx_11_0_arm64.whl", hash = "sha256:9450464b458cca2c86252b624279115dcaa7260a40d3cb1594bf2b410a2bd1a3"}, + {file = "h5py-3.10.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:fd6f6d1384a9f491732cee233b99cd4bfd6e838a8815cc86722f9d2ee64032af"}, + {file = "h5py-3.10.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:3074ec45d3dc6e178c6f96834cf8108bf4a60ccb5ab044e16909580352010a97"}, + {file = "h5py-3.10.0-cp38-cp38-win_amd64.whl", hash = "sha256:212bb997a91e6a895ce5e2f365ba764debeaef5d2dca5c6fb7098d66607adf99"}, + {file = "h5py-3.10.0-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:5dfc65ac21fa2f630323c92453cadbe8d4f504726ec42f6a56cf80c2f90d6c52"}, + {file = "h5py-3.10.0-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:d4682b94fd36ab217352be438abd44c8f357c5449b8995e63886b431d260f3d3"}, + {file = "h5py-3.10.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:aece0e2e1ed2aab076c41802e50a0c3e5ef8816d60ece39107d68717d4559824"}, + {file = "h5py-3.10.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:43a61b2c2ad65b1fabc28802d133eed34debcc2c8b420cb213d3d4ef4d3e2229"}, + {file = "h5py-3.10.0-cp39-cp39-win_amd64.whl", hash = "sha256:ae2f0201c950059676455daf92700eeb57dcf5caaf71b9e1328e6e6593601770"}, + {file = "h5py-3.10.0.tar.gz", hash = "sha256:d93adc48ceeb33347eb24a634fb787efc7ae4644e6ea4ba733d099605045c049"}, ] [package.dependencies] @@ -1275,46 +1276,47 @@ numpy = ">=1.17.3" [[package]] name = "httptools" -version = "0.6.0" +version = "0.6.1" description = "A collection of framework independent HTTP protocol utils." optional = false -python-versions = ">=3.5.0" -files = [ - {file = "httptools-0.6.0-cp310-cp310-macosx_10_9_universal2.whl", hash = "sha256:818325afee467d483bfab1647a72054246d29f9053fd17cc4b86cda09cc60339"}, - {file = "httptools-0.6.0-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:72205730bf1be875003692ca54a4a7c35fac77b4746008966061d9d41a61b0f5"}, - {file = "httptools-0.6.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:33eb1d4e609c835966e969a31b1dedf5ba16b38cab356c2ce4f3e33ffa94cad3"}, - {file = "httptools-0.6.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:6bdc6675ec6cb79d27e0575750ac6e2b47032742e24eed011b8db73f2da9ed40"}, - {file = "httptools-0.6.0-cp310-cp310-musllinux_1_1_aarch64.whl", hash = "sha256:463c3bc5ef64b9cf091be9ac0e0556199503f6e80456b790a917774a616aff6e"}, - {file = "httptools-0.6.0-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:82f228b88b0e8c6099a9c4757ce9fdbb8b45548074f8d0b1f0fc071e35655d1c"}, - {file = "httptools-0.6.0-cp310-cp310-win_amd64.whl", hash = "sha256:0781fedc610293a2716bc7fa142d4c85e6776bc59d617a807ff91246a95dea35"}, - {file = "httptools-0.6.0-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:721e503245d591527cddd0f6fd771d156c509e831caa7a57929b55ac91ee2b51"}, - {file = "httptools-0.6.0-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:274bf20eeb41b0956e34f6a81f84d26ed57c84dd9253f13dcb7174b27ccd8aaf"}, - {file = "httptools-0.6.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:259920bbae18740a40236807915def554132ad70af5067e562f4660b62c59b90"}, - {file = "httptools-0.6.0-cp311-cp311-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:03bfd2ae8a2d532952ac54445a2fb2504c804135ed28b53fefaf03d3a93eb1fd"}, - {file = "httptools-0.6.0-cp311-cp311-musllinux_1_1_aarch64.whl", hash = "sha256:f959e4770b3fc8ee4dbc3578fd910fab9003e093f20ac8c621452c4d62e517cb"}, - {file = "httptools-0.6.0-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:6e22896b42b95b3237eccc42278cd72c0df6f23247d886b7ded3163452481e38"}, - {file = "httptools-0.6.0-cp311-cp311-win_amd64.whl", hash = "sha256:38f3cafedd6aa20ae05f81f2e616ea6f92116c8a0f8dcb79dc798df3356836e2"}, - {file = "httptools-0.6.0-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:47043a6e0ea753f006a9d0dd076a8f8c99bc0ecae86a0888448eb3076c43d717"}, - {file = "httptools-0.6.0-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:35a541579bed0270d1ac10245a3e71e5beeb1903b5fbbc8d8b4d4e728d48ff1d"}, - {file = "httptools-0.6.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:65d802e7b2538a9756df5acc062300c160907b02e15ed15ba035b02bce43e89c"}, - {file = "httptools-0.6.0-cp37-cp37m-musllinux_1_1_aarch64.whl", hash = "sha256:26326e0a8fe56829f3af483200d914a7cd16d8d398d14e36888b56de30bec81a"}, - {file = "httptools-0.6.0-cp37-cp37m-musllinux_1_1_x86_64.whl", hash = "sha256:e41ccac9e77cd045f3e4ee0fc62cbf3d54d7d4b375431eb855561f26ee7a9ec4"}, - {file = "httptools-0.6.0-cp37-cp37m-win_amd64.whl", hash = "sha256:4e748fc0d5c4a629988ef50ac1aef99dfb5e8996583a73a717fc2cac4ab89932"}, - {file = "httptools-0.6.0-cp38-cp38-macosx_10_9_universal2.whl", hash = "sha256:cf8169e839a0d740f3d3c9c4fa630ac1a5aaf81641a34575ca6773ed7ce041a1"}, - {file = "httptools-0.6.0-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:5dcc14c090ab57b35908d4a4585ec5c0715439df07be2913405991dbb37e049d"}, - {file = "httptools-0.6.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:0d0b0571806a5168013b8c3d180d9f9d6997365a4212cb18ea20df18b938aa0b"}, - {file = "httptools-0.6.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:0fb4a608c631f7dcbdf986f40af7a030521a10ba6bc3d36b28c1dc9e9035a3c0"}, - {file = "httptools-0.6.0-cp38-cp38-musllinux_1_1_aarch64.whl", hash = "sha256:93f89975465133619aea8b1952bc6fa0e6bad22a447c6d982fc338fbb4c89649"}, - {file = "httptools-0.6.0-cp38-cp38-musllinux_1_1_x86_64.whl", hash = "sha256:73e9d66a5a28b2d5d9fbd9e197a31edd02be310186db423b28e6052472dc8201"}, - {file = "httptools-0.6.0-cp38-cp38-win_amd64.whl", hash = "sha256:22c01fcd53648162730a71c42842f73b50f989daae36534c818b3f5050b54589"}, - {file = "httptools-0.6.0-cp39-cp39-macosx_10_9_universal2.whl", hash = "sha256:3f96d2a351b5625a9fd9133c95744e8ca06f7a4f8f0b8231e4bbaae2c485046a"}, - {file = "httptools-0.6.0-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:72ec7c70bd9f95ef1083d14a755f321d181f046ca685b6358676737a5fecd26a"}, - {file = "httptools-0.6.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:b703d15dbe082cc23266bf5d9448e764c7cb3fcfe7cb358d79d3fd8248673ef9"}, - {file = "httptools-0.6.0-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:82c723ed5982f8ead00f8e7605c53e55ffe47c47465d878305ebe0082b6a1755"}, - {file = "httptools-0.6.0-cp39-cp39-musllinux_1_1_aarch64.whl", hash = "sha256:b0a816bb425c116a160fbc6f34cece097fd22ece15059d68932af686520966bd"}, - {file = "httptools-0.6.0-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:dea66d94e5a3f68c5e9d86e0894653b87d952e624845e0b0e3ad1c733c6cc75d"}, - {file = "httptools-0.6.0-cp39-cp39-win_amd64.whl", hash = "sha256:23b09537086a5a611fad5696fc8963d67c7e7f98cb329d38ee114d588b0b74cd"}, - {file = "httptools-0.6.0.tar.gz", hash = "sha256:9fc6e409ad38cbd68b177cd5158fc4042c796b82ca88d99ec78f07bed6c6b796"}, +python-versions = ">=3.8.0" +files = [ + {file = "httptools-0.6.1-cp310-cp310-macosx_10_9_universal2.whl", hash = "sha256:d2f6c3c4cb1948d912538217838f6e9960bc4a521d7f9b323b3da579cd14532f"}, + {file = "httptools-0.6.1-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:00d5d4b68a717765b1fabfd9ca755bd12bf44105eeb806c03d1962acd9b8e563"}, + {file = "httptools-0.6.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:639dc4f381a870c9ec860ce5c45921db50205a37cc3334e756269736ff0aac58"}, + {file = "httptools-0.6.1-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:e57997ac7fb7ee43140cc03664de5f268813a481dff6245e0075925adc6aa185"}, + {file = "httptools-0.6.1-cp310-cp310-musllinux_1_1_aarch64.whl", hash = "sha256:0ac5a0ae3d9f4fe004318d64b8a854edd85ab76cffbf7ef5e32920faef62f142"}, + {file = "httptools-0.6.1-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:3f30d3ce413088a98b9db71c60a6ada2001a08945cb42dd65a9a9fe228627658"}, + {file = "httptools-0.6.1-cp310-cp310-win_amd64.whl", hash = "sha256:1ed99a373e327f0107cb513b61820102ee4f3675656a37a50083eda05dc9541b"}, + {file = "httptools-0.6.1-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:7a7ea483c1a4485c71cb5f38be9db078f8b0e8b4c4dc0210f531cdd2ddac1ef1"}, + {file = "httptools-0.6.1-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:85ed077c995e942b6f1b07583e4eb0a8d324d418954fc6af913d36db7c05a5a0"}, + {file = "httptools-0.6.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:8b0bb634338334385351a1600a73e558ce619af390c2b38386206ac6a27fecfc"}, + {file = "httptools-0.6.1-cp311-cp311-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:7d9ceb2c957320def533671fc9c715a80c47025139c8d1f3797477decbc6edd2"}, + {file = "httptools-0.6.1-cp311-cp311-musllinux_1_1_aarch64.whl", hash = "sha256:4f0f8271c0a4db459f9dc807acd0eadd4839934a4b9b892f6f160e94da309837"}, + {file = "httptools-0.6.1-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:6a4f5ccead6d18ec072ac0b84420e95d27c1cdf5c9f1bc8fbd8daf86bd94f43d"}, + {file = "httptools-0.6.1-cp311-cp311-win_amd64.whl", hash = "sha256:5cceac09f164bcba55c0500a18fe3c47df29b62353198e4f37bbcc5d591172c3"}, + {file = "httptools-0.6.1-cp312-cp312-macosx_10_9_universal2.whl", hash = "sha256:75c8022dca7935cba14741a42744eee13ba05db00b27a4b940f0d646bd4d56d0"}, + {file = "httptools-0.6.1-cp312-cp312-macosx_10_9_x86_64.whl", hash = "sha256:48ed8129cd9a0d62cf4d1575fcf90fb37e3ff7d5654d3a5814eb3d55f36478c2"}, + {file = "httptools-0.6.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:6f58e335a1402fb5a650e271e8c2d03cfa7cea46ae124649346d17bd30d59c90"}, + {file = "httptools-0.6.1-cp312-cp312-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:93ad80d7176aa5788902f207a4e79885f0576134695dfb0fefc15b7a4648d503"}, + {file = "httptools-0.6.1-cp312-cp312-musllinux_1_1_aarch64.whl", hash = "sha256:9bb68d3a085c2174c2477eb3ffe84ae9fb4fde8792edb7bcd09a1d8467e30a84"}, + {file = "httptools-0.6.1-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:b512aa728bc02354e5ac086ce76c3ce635b62f5fbc32ab7082b5e582d27867bb"}, + {file = "httptools-0.6.1-cp312-cp312-win_amd64.whl", hash = "sha256:97662ce7fb196c785344d00d638fc9ad69e18ee4bfb4000b35a52efe5adcc949"}, + {file = "httptools-0.6.1-cp38-cp38-macosx_10_9_universal2.whl", hash = "sha256:8e216a038d2d52ea13fdd9b9c9c7459fb80d78302b257828285eca1c773b99b3"}, + {file = "httptools-0.6.1-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:3e802e0b2378ade99cd666b5bffb8b2a7cc8f3d28988685dc300469ea8dd86cb"}, + {file = "httptools-0.6.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:4bd3e488b447046e386a30f07af05f9b38d3d368d1f7b4d8f7e10af85393db97"}, + {file = "httptools-0.6.1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:fe467eb086d80217b7584e61313ebadc8d187a4d95bb62031b7bab4b205c3ba3"}, + {file = "httptools-0.6.1-cp38-cp38-musllinux_1_1_aarch64.whl", hash = "sha256:3c3b214ce057c54675b00108ac42bacf2ab8f85c58e3f324a4e963bbc46424f4"}, + {file = "httptools-0.6.1-cp38-cp38-musllinux_1_1_x86_64.whl", hash = "sha256:8ae5b97f690badd2ca27cbf668494ee1b6d34cf1c464271ef7bfa9ca6b83ffaf"}, + {file = "httptools-0.6.1-cp38-cp38-win_amd64.whl", hash = "sha256:405784577ba6540fa7d6ff49e37daf104e04f4b4ff2d1ac0469eaa6a20fde084"}, + {file = "httptools-0.6.1-cp39-cp39-macosx_10_9_universal2.whl", hash = "sha256:95fb92dd3649f9cb139e9c56604cc2d7c7bf0fc2e7c8d7fbd58f96e35eddd2a3"}, + {file = "httptools-0.6.1-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:dcbab042cc3ef272adc11220517278519adf8f53fd3056d0e68f0a6f891ba94e"}, + {file = "httptools-0.6.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:0cf2372e98406efb42e93bfe10f2948e467edfd792b015f1b4ecd897903d3e8d"}, + {file = "httptools-0.6.1-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:678fcbae74477a17d103b7cae78b74800d795d702083867ce160fc202104d0da"}, + {file = "httptools-0.6.1-cp39-cp39-musllinux_1_1_aarch64.whl", hash = "sha256:e0b281cf5a125c35f7f6722b65d8542d2e57331be573e9e88bc8b0115c4a7a81"}, + {file = "httptools-0.6.1-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:95658c342529bba4e1d3d2b1a874db16c7cca435e8827422154c9da76ac4e13a"}, + {file = "httptools-0.6.1-cp39-cp39-win_amd64.whl", hash = "sha256:7ebaec1bf683e4bf5e9fbb49b8cc36da482033596a415b3e4ebab5a4c0d7ec5e"}, + {file = "httptools-0.6.1.tar.gz", hash = "sha256:c6e26c30455600b95d94b1b836085138e82f177351454ee841c148f93a9bad5a"}, ] [package.extras] @@ -1462,63 +1464,70 @@ files = [ [[package]] name = "jax" -version = "0.4.17" +version = "0.4.19" description = "Differentiate, compile, and transform Numpy code." optional = false python-versions = ">=3.9" files = [ - {file = "jax-0.4.17-py3-none-any.whl", hash = "sha256:c3ab72ea2f1c5d8ccf2561e79f6562fb2964629f3e55b3ac1c11c48b64c20336"}, - {file = "jax-0.4.17.tar.gz", hash = "sha256:d7508a69e87835f534cb07a2f21d79cc1cb8c4cfdcf7fb010927267ef7355f1d"}, + {file = "jax-0.4.19-py3-none-any.whl", hash = "sha256:168462f53774d727eede0e3bfbc90b9946c708483564d3179b8e44bb54846b46"}, + {file = "jax-0.4.19.tar.gz", hash = "sha256:29f87f9a50964d3ca5eeb2973de3462f0e8b4eca6d46027894a0e9a903420601"}, ] [package.dependencies] ml-dtypes = ">=0.2.0" -numpy = ">=1.22" +numpy = [ + {version = ">=1.23.2", markers = "python_version >= \"3.11\""}, + {version = ">=1.22", markers = "python_version < \"3.11\""}, +] opt-einsum = "*" -scipy = ">=1.7" +scipy = ">=1.9" [package.extras] australis = ["protobuf (>=3.13,<4)"] -ci = ["jaxlib (==0.4.16)"] -cpu = ["jaxlib (==0.4.17)"] -cuda = ["jaxlib (==0.4.17+cuda11.cudnn86)"] -cuda11-cudnn86 = ["jaxlib (==0.4.17+cuda11.cudnn86)"] -cuda11-local = ["jaxlib (==0.4.17+cuda11.cudnn86)"] -cuda11-pip = ["jaxlib (==0.4.17+cuda11.cudnn86)", "nvidia-cublas-cu11 (>=11.11)", "nvidia-cuda-cupti-cu11 (>=11.8)", "nvidia-cuda-nvcc-cu11 (>=11.8)", "nvidia-cuda-runtime-cu11 (>=11.8)", "nvidia-cudnn-cu11 (>=8.8)", "nvidia-cufft-cu11 (>=10.9)", "nvidia-cusolver-cu11 (>=11.4)", "nvidia-cusparse-cu11 (>=11.7)"] -cuda12-local = ["jaxlib (==0.4.17+cuda12.cudnn89)"] -cuda12-pip = ["jaxlib (==0.4.17+cuda12.cudnn89)", "nvidia-cublas-cu12 (>=12.2.5.6)", "nvidia-cuda-cupti-cu12 (>=12.2.142)", "nvidia-cuda-nvcc-cu12 (>=12.2.140)", "nvidia-cuda-runtime-cu12 (>=12.2.140)", "nvidia-cudnn-cu12 (>=8.9)", "nvidia-cufft-cu12 (>=11.0.8.103)", "nvidia-cusolver-cu12 (>=11.5.2)", "nvidia-cusparse-cu12 (>=12.1.2.141)"] +ci = ["jaxlib (==0.4.18)"] +cpu = ["jaxlib (==0.4.19)"] +cuda = ["jaxlib (==0.4.19+cuda11.cudnn86)"] +cuda11-cudnn86 = ["jaxlib (==0.4.19+cuda11.cudnn86)"] +cuda11-local = ["jaxlib (==0.4.19+cuda11.cudnn86)"] +cuda11-pip = ["jaxlib (==0.4.19+cuda11.cudnn86)", "nvidia-cublas-cu11 (>=11.11)", "nvidia-cuda-cupti-cu11 (>=11.8)", "nvidia-cuda-nvcc-cu11 (>=11.8)", "nvidia-cuda-runtime-cu11 (>=11.8)", "nvidia-cudnn-cu11 (>=8.8)", "nvidia-cufft-cu11 (>=10.9)", "nvidia-cusolver-cu11 (>=11.4)", "nvidia-cusparse-cu11 (>=11.7)", "nvidia-nccl-cu11 (>=2.18.3)"] +cuda12-local = ["jaxlib (==0.4.19+cuda12.cudnn89)"] +cuda12-pip = ["jaxlib (==0.4.19+cuda12.cudnn89)", "nvidia-cublas-cu12 (>=12.2.5.6)", "nvidia-cuda-cupti-cu12 (>=12.2.142)", "nvidia-cuda-nvcc-cu12 (>=12.2.140)", "nvidia-cuda-runtime-cu12 (>=12.2.140)", "nvidia-cudnn-cu12 (>=8.9)", "nvidia-cufft-cu12 (>=11.0.8.103)", "nvidia-cusolver-cu12 (>=11.5.2)", "nvidia-cusparse-cu12 (>=12.1.2.141)", "nvidia-nccl-cu12 (>=2.18.3)", "nvidia-nvjitlink-cu12 (>=12.2)"] minimum-jaxlib = ["jaxlib (==0.4.14)"] -tpu = ["jaxlib (==0.4.17)", "libtpu-nightly (==0.1.dev20231003)", "requests"] +tpu = ["jaxlib (==0.4.19)", "libtpu-nightly (==0.1.dev20231018)", "requests"] [[package]] name = "jaxlib" -version = "0.4.17" +version = "0.4.19" description = "XLA library for JAX" optional = false python-versions = ">=3.9" files = [ - {file = "jaxlib-0.4.17-cp310-cp310-macosx_10_14_x86_64.whl", hash = "sha256:d4be1ac4bf1be1ae1cd8f5f4da414a6d0de8de36cf2effdb5758d4d677896078"}, - {file = "jaxlib-0.4.17-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:392c779f902c43e1a0af49159daffef9b5af952aba001463f98cf95a59ef17ff"}, - {file = "jaxlib-0.4.17-cp310-cp310-manylinux2014_x86_64.whl", hash = "sha256:160fce68b82a79a6c522652e8dd9a10aac9c00d1599cb7e166671ad909aa139e"}, - {file = "jaxlib-0.4.17-cp310-cp310-win_amd64.whl", hash = "sha256:61b3788c6cfe46f307e6e67d4a942de72cf34711ff349f4f11500cdf6dc67199"}, - {file = "jaxlib-0.4.17-cp311-cp311-macosx_10_14_x86_64.whl", hash = "sha256:6c3524150bd85098f291fac81f73e285f3e095dbbb49751647cc27bed5327a78"}, - {file = "jaxlib-0.4.17-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:2e0d84a756b47ef6db52a6532b1f242cb8dc9035c102c60075470d65e71f7afb"}, - {file = "jaxlib-0.4.17-cp311-cp311-manylinux2014_x86_64.whl", hash = "sha256:e04a0b8bb18ac24e25c15ed03be771815566f118c16f585ffe2e0f75bf7c064d"}, - {file = "jaxlib-0.4.17-cp311-cp311-win_amd64.whl", hash = "sha256:73173f1aff8d277110d32bdd5e073dc7d50e6618b5567b3bfbc53864b4613439"}, - {file = "jaxlib-0.4.17-cp312-cp312-macosx_10_14_x86_64.whl", hash = "sha256:909971337aabf5f2724a84c3166cea454b37024908d830695dc6b4ba4440676f"}, - {file = "jaxlib-0.4.17-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:98d42a402201fd0cb332bad4177b20c942d1acd1487581ee0c3cb5ef6766531a"}, - {file = "jaxlib-0.4.17-cp312-cp312-manylinux2014_x86_64.whl", hash = "sha256:593aa9d1c940b9215968878561ad59feee2438386c3868d6524ff4ca730cfdf1"}, - {file = "jaxlib-0.4.17-cp312-cp312-win_amd64.whl", hash = "sha256:a4384cc7187f4f10749c6c623211d1a6b55575f921c00af38ff8f05fd3f7ecfd"}, - {file = "jaxlib-0.4.17-cp39-cp39-macosx_10_14_x86_64.whl", hash = "sha256:84b6cd54143ffe2ce45d5bcf2f9eafa1f9b4cf51dab0cc8e7703622fb624549d"}, - {file = "jaxlib-0.4.17-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:a521d8323ef4d8155efc11b788f29dd3794cbb80f83533da957921a058fd3abe"}, - {file = "jaxlib-0.4.17-cp39-cp39-manylinux2014_x86_64.whl", hash = "sha256:c933a6fb74f9fc16a2610566b32ff0135077ae9032e50f695653377d2cfbc9e5"}, - {file = "jaxlib-0.4.17-cp39-cp39-win_amd64.whl", hash = "sha256:44a2bff9966fe3b5783595d3214d3598bea48a2aa502de9d7d44c2ca39426929"}, + {file = "jaxlib-0.4.19-cp310-cp310-macosx_10_14_x86_64.whl", hash = "sha256:82a28c519497aedf2c42d4c0a106f301f0b33bc80cf58d7fb59a6e2555b23746"}, + {file = "jaxlib-0.4.19-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:3bb747bdd28b29f3442c67c5e132b2ecde445fa09aed9bbc3ad2d75af172911f"}, + {file = "jaxlib-0.4.19-cp310-cp310-manylinux2014_aarch64.whl", hash = "sha256:05272c33ea6a2b83c8c6842b7f944fc0a7be82f85666ddbf76391fee308e60b3"}, + {file = "jaxlib-0.4.19-cp310-cp310-manylinux2014_x86_64.whl", hash = "sha256:92c9d8f823c4b2d69cb7994a8db4a0f9948f26d4add18d0d156105b9f0420729"}, + {file = "jaxlib-0.4.19-cp310-cp310-win_amd64.whl", hash = "sha256:a0f02c69b8406cccedec8b77ab27f492bbee92dfb2eb817a3a124b86e5a7f43d"}, + {file = "jaxlib-0.4.19-cp311-cp311-macosx_10_14_x86_64.whl", hash = "sha256:80e2c8c6493eda17b0d86a96bb55a0315131d5812eb71ed9a1ded06f0b2e5154"}, + {file = "jaxlib-0.4.19-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:cc23808da5960904fd2679b58e3735461e607a6ab2ec00ad4ca2ccd0ca922733"}, + {file = "jaxlib-0.4.19-cp311-cp311-manylinux2014_aarch64.whl", hash = "sha256:64a2fffc69e66863cda0f2bf5443d92abb4d37f2613273d8d05229635be539a3"}, + {file = "jaxlib-0.4.19-cp311-cp311-manylinux2014_x86_64.whl", hash = "sha256:9be343cf05cc35ba0d8c39769cb63ebea0283767502595566f55100e9aaa0cfc"}, + {file = "jaxlib-0.4.19-cp311-cp311-win_amd64.whl", hash = "sha256:675362fa803e7f64b2cf3c371852136be72a6223622c0f6aa254351bf18f1152"}, + {file = "jaxlib-0.4.19-cp312-cp312-macosx_10_14_x86_64.whl", hash = "sha256:287ce521f6bd2ad61c1d7fa2fc5fd229a613a43e3f5e31d5bb98f705d4535267"}, + {file = "jaxlib-0.4.19-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:680313acb5d4f446303effa475ec88f3cfc3edbe0d00d077f459fcbee5d4a850"}, + {file = "jaxlib-0.4.19-cp312-cp312-manylinux2014_aarch64.whl", hash = "sha256:f664f293465cf009147ff102ae877e24ea4b6513454d53d4633b8ac1f5130e6f"}, + {file = "jaxlib-0.4.19-cp312-cp312-manylinux2014_x86_64.whl", hash = "sha256:0594e692036e5789d6b5f6d8e2dff5f5a3f8deccd9404760a455e1e6ac064579"}, + {file = "jaxlib-0.4.19-cp312-cp312-win_amd64.whl", hash = "sha256:9c724eb7f35d5e9de0d1b992048c7ca134ff6b63fc59f2c0fddb096da225c49c"}, + {file = "jaxlib-0.4.19-cp39-cp39-macosx_10_14_x86_64.whl", hash = "sha256:9834f520d2ea3dd0a4c6d30c151d2a1cb38865d584cbc22ecf0d7fbcea040ac0"}, + {file = "jaxlib-0.4.19-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:4e6bf688eaa535b66ac3fad88de7fa1a6334375f04039253b7a4f79507bee11b"}, + {file = "jaxlib-0.4.19-cp39-cp39-manylinux2014_aarch64.whl", hash = "sha256:96510b738c73a6dee2361bb58298400f1633d544c5941ef61a54932a7c63f288"}, + {file = "jaxlib-0.4.19-cp39-cp39-manylinux2014_x86_64.whl", hash = "sha256:f1b4eb5aeb6e2b4a959db91cc0c7e005fe35e1875a2dcdc62b922c0a6fc934f1"}, + {file = "jaxlib-0.4.19-cp39-cp39-win_amd64.whl", hash = "sha256:3750000f5859464e9cce0d51b79832528cd1cf4117847bb248b514ad00a58a75"}, ] [package.dependencies] ml-dtypes = ">=0.2.0" numpy = ">=1.22" -scipy = ">=1.7" +scipy = ">=1.9" [package.extras] cuda11-pip = ["nvidia-cublas-cu11 (>=11.11)", "nvidia-cuda-cupti-cu11 (>=11.8)", "nvidia-cuda-nvcc-cu11 (>=11.8)", "nvidia-cuda-runtime-cu11 (>=11.8)", "nvidia-cudnn-cu11 (>=8.8)", "nvidia-cufft-cu11 (>=10.9)", "nvidia-cusolver-cu11 (>=11.4)", "nvidia-cusparse-cu11 (>=11.7)"] @@ -1668,17 +1677,17 @@ files = [ [[package]] name = "markdown" -version = "3.4.4" +version = "3.5" description = "Python implementation of John Gruber's Markdown." optional = false -python-versions = ">=3.7" +python-versions = ">=3.8" files = [ - {file = "Markdown-3.4.4-py3-none-any.whl", hash = "sha256:a4c1b65c0957b4bd9e7d86ddc7b3c9868fb9670660f6f99f6d1bca8954d5a941"}, - {file = "Markdown-3.4.4.tar.gz", hash = "sha256:225c6123522495d4119a90b3a3ba31a1e87a70369e03f14799ea9c0d7183a3d6"}, + {file = "Markdown-3.5-py3-none-any.whl", hash = "sha256:4afb124395ce5fc34e6d9886dab977fd9ae987fc6e85689f08278cf0c69d4bf3"}, + {file = "Markdown-3.5.tar.gz", hash = "sha256:a807eb2e4778d9156c8f07876c6e4d50b5494c5665c4834f67b06459dfd877b3"}, ] [package.extras] -docs = ["mdx-gh-links (>=0.2)", "mkdocs (>=1.0)", "mkdocs-nature (>=0.4)"] +docs = ["mdx-gh-links (>=0.2)", "mkdocs (>=1.5)", "mkdocs-gen-files", "mkdocs-literate-nav", "mkdocs-nature (>=0.6)", "mkdocs-section-index", "mkdocstrings[python]"] testing = ["coverage", "pyyaml"] [[package]] @@ -1978,21 +1987,21 @@ files = [ [[package]] name = "networkx" -version = "3.1" +version = "3.2" description = "Python package for creating and manipulating graphs and networks" optional = false -python-versions = ">=3.8" +python-versions = ">=3.9" files = [ - {file = "networkx-3.1-py3-none-any.whl", hash = "sha256:4f33f68cb2afcf86f28a45f43efc27a9386b535d567d2127f8f61d51dec58d36"}, - {file = "networkx-3.1.tar.gz", hash = "sha256:de346335408f84de0eada6ff9fafafff9bcda11f0a0dfaa931133debb146ab61"}, + {file = "networkx-3.2-py3-none-any.whl", hash = "sha256:8b25f564bd28f94ac821c58b04ae1a3109e73b001a7d476e4bb0d00d63706bf8"}, + {file = "networkx-3.2.tar.gz", hash = "sha256:bda29edf392d9bfa5602034c767d28549214ec45f620081f0b74dc036a1fbbc1"}, ] [package.extras] -default = ["matplotlib (>=3.4)", "numpy (>=1.20)", "pandas (>=1.3)", "scipy (>=1.8)"] -developer = ["mypy (>=1.1)", "pre-commit (>=3.2)"] -doc = ["nb2plots (>=0.6)", "numpydoc (>=1.5)", "pillow (>=9.4)", "pydata-sphinx-theme (>=0.13)", "sphinx (>=6.1)", "sphinx-gallery (>=0.12)", "texext (>=0.6.7)"] -extra = ["lxml (>=4.6)", "pydot (>=1.4.2)", "pygraphviz (>=1.10)", "sympy (>=1.10)"] -test = ["codecov (>=2.1)", "pytest (>=7.2)", "pytest-cov (>=4.0)"] +default = ["matplotlib (>=3.5)", "numpy (>=1.22)", "pandas (>=1.4)", "scipy (>=1.9,!=1.11.0,!=1.11.1)"] +developer = ["changelist (==0.4)", "mypy (>=1.1)", "pre-commit (>=3.2)", "rtoml"] +doc = ["nb2plots (>=0.7)", "nbconvert (<7.9)", "numpydoc (>=1.6)", "pillow (>=9.4)", "pydata-sphinx-theme (>=0.14)", "sphinx (>=7)", "sphinx-gallery (>=0.14)", "texext (>=0.6.7)"] +extra = ["lxml (>=4.6)", "pydot (>=1.4.2)", "pygraphviz (>=1.11)", "sympy (>=1.10)"] +test = ["pytest (>=7.2)", "pytest-cov (>=4.0)"] [[package]] name = "nltk" @@ -2021,43 +2030,43 @@ twitter = ["twython"] [[package]] name = "numpy" -version = "1.26.0" +version = "1.26.1" description = "Fundamental package for array computing in Python" optional = false python-versions = "<3.13,>=3.9" files = [ - {file = "numpy-1.26.0-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:f8db2f125746e44dce707dd44d4f4efeea8d7e2b43aace3f8d1f235cfa2733dd"}, - {file = "numpy-1.26.0-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:0621f7daf973d34d18b4e4bafb210bbaf1ef5e0100b5fa750bd9cde84c7ac292"}, - {file = "numpy-1.26.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:51be5f8c349fdd1a5568e72713a21f518e7d6707bcf8503b528b88d33b57dc68"}, - {file = "numpy-1.26.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:767254ad364991ccfc4d81b8152912e53e103ec192d1bb4ea6b1f5a7117040be"}, - {file = "numpy-1.26.0-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:436c8e9a4bdeeee84e3e59614d38c3dbd3235838a877af8c211cfcac8a80b8d3"}, - {file = "numpy-1.26.0-cp310-cp310-win32.whl", hash = "sha256:c2e698cb0c6dda9372ea98a0344245ee65bdc1c9dd939cceed6bb91256837896"}, - {file = "numpy-1.26.0-cp310-cp310-win_amd64.whl", hash = "sha256:09aaee96c2cbdea95de76ecb8a586cb687d281c881f5f17bfc0fb7f5890f6b91"}, - {file = "numpy-1.26.0-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:637c58b468a69869258b8ae26f4a4c6ff8abffd4a8334c830ffb63e0feefe99a"}, - {file = "numpy-1.26.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:306545e234503a24fe9ae95ebf84d25cba1fdc27db971aa2d9f1ab6bba19a9dd"}, - {file = "numpy-1.26.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:8c6adc33561bd1d46f81131d5352348350fc23df4d742bb246cdfca606ea1208"}, - {file = "numpy-1.26.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:e062aa24638bb5018b7841977c360d2f5917268d125c833a686b7cbabbec496c"}, - {file = "numpy-1.26.0-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:546b7dd7e22f3c6861463bebb000646fa730e55df5ee4a0224408b5694cc6148"}, - {file = "numpy-1.26.0-cp311-cp311-win32.whl", hash = "sha256:c0b45c8b65b79337dee5134d038346d30e109e9e2e9d43464a2970e5c0e93229"}, - {file = "numpy-1.26.0-cp311-cp311-win_amd64.whl", hash = "sha256:eae430ecf5794cb7ae7fa3808740b015aa80747e5266153128ef055975a72b99"}, - {file = "numpy-1.26.0-cp312-cp312-macosx_10_9_x86_64.whl", hash = "sha256:166b36197e9debc4e384e9c652ba60c0bacc216d0fc89e78f973a9760b503388"}, - {file = "numpy-1.26.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:f042f66d0b4ae6d48e70e28d487376204d3cbf43b84c03bac57e28dac6151581"}, - {file = "numpy-1.26.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:e5e18e5b14a7560d8acf1c596688f4dfd19b4f2945b245a71e5af4ddb7422feb"}, - {file = "numpy-1.26.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:7f6bad22a791226d0a5c7c27a80a20e11cfe09ad5ef9084d4d3fc4a299cca505"}, - {file = "numpy-1.26.0-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:4acc65dd65da28060e206c8f27a573455ed724e6179941edb19f97e58161bb69"}, - {file = "numpy-1.26.0-cp312-cp312-win32.whl", hash = "sha256:bb0d9a1aaf5f1cb7967320e80690a1d7ff69f1d47ebc5a9bea013e3a21faec95"}, - {file = "numpy-1.26.0-cp312-cp312-win_amd64.whl", hash = "sha256:ee84ca3c58fe48b8ddafdeb1db87388dce2c3c3f701bf447b05e4cfcc3679112"}, - {file = "numpy-1.26.0-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:4a873a8180479bc829313e8d9798d5234dfacfc2e8a7ac188418189bb8eafbd2"}, - {file = "numpy-1.26.0-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:914b28d3215e0c721dc75db3ad6d62f51f630cb0c277e6b3bcb39519bed10bd8"}, - {file = "numpy-1.26.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:c78a22e95182fb2e7874712433eaa610478a3caf86f28c621708d35fa4fd6e7f"}, - {file = "numpy-1.26.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:86f737708b366c36b76e953c46ba5827d8c27b7a8c9d0f471810728e5a2fe57c"}, - {file = "numpy-1.26.0-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:b44e6a09afc12952a7d2a58ca0a2429ee0d49a4f89d83a0a11052da696440e49"}, - {file = "numpy-1.26.0-cp39-cp39-win32.whl", hash = "sha256:5671338034b820c8d58c81ad1dafc0ed5a00771a82fccc71d6438df00302094b"}, - {file = "numpy-1.26.0-cp39-cp39-win_amd64.whl", hash = "sha256:020cdbee66ed46b671429c7265cf00d8ac91c046901c55684954c3958525dab2"}, - {file = "numpy-1.26.0-pp39-pypy39_pp73-macosx_10_9_x86_64.whl", hash = "sha256:0792824ce2f7ea0c82ed2e4fecc29bb86bee0567a080dacaf2e0a01fe7654369"}, - {file = "numpy-1.26.0-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:7d484292eaeb3e84a51432a94f53578689ffdea3f90e10c8b203a99be5af57d8"}, - {file = "numpy-1.26.0-pp39-pypy39_pp73-win_amd64.whl", hash = "sha256:186ba67fad3c60dbe8a3abff3b67a91351100f2661c8e2a80364ae6279720299"}, - {file = "numpy-1.26.0.tar.gz", hash = "sha256:f93fc78fe8bf15afe2b8d6b6499f1c73953169fad1e9a8dd086cdff3190e7fdf"}, + {file = "numpy-1.26.1-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:82e871307a6331b5f09efda3c22e03c095d957f04bf6bc1804f30048d0e5e7af"}, + {file = "numpy-1.26.1-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:cdd9ec98f0063d93baeb01aad472a1a0840dee302842a2746a7a8e92968f9575"}, + {file = "numpy-1.26.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:d78f269e0c4fd365fc2992c00353e4530d274ba68f15e968d8bc3c69ce5f5244"}, + {file = "numpy-1.26.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:8ab9163ca8aeb7fd32fe93866490654d2f7dda4e61bc6297bf72ce07fdc02f67"}, + {file = "numpy-1.26.1-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:78ca54b2f9daffa5f323f34cdf21e1d9779a54073f0018a3094ab907938331a2"}, + {file = "numpy-1.26.1-cp310-cp310-win32.whl", hash = "sha256:d1cfc92db6af1fd37a7bb58e55c8383b4aa1ba23d012bdbba26b4bcca45ac297"}, + {file = "numpy-1.26.1-cp310-cp310-win_amd64.whl", hash = "sha256:d2984cb6caaf05294b8466966627e80bf6c7afd273279077679cb010acb0e5ab"}, + {file = "numpy-1.26.1-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:cd7837b2b734ca72959a1caf3309457a318c934abef7a43a14bb984e574bbb9a"}, + {file = "numpy-1.26.1-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:1c59c046c31a43310ad0199d6299e59f57a289e22f0f36951ced1c9eac3665b9"}, + {file = "numpy-1.26.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:d58e8c51a7cf43090d124d5073bc29ab2755822181fcad978b12e144e5e5a4b3"}, + {file = "numpy-1.26.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:6081aed64714a18c72b168a9276095ef9155dd7888b9e74b5987808f0dd0a974"}, + {file = "numpy-1.26.1-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:97e5d6a9f0702c2863aaabf19f0d1b6c2628fbe476438ce0b5ce06e83085064c"}, + {file = "numpy-1.26.1-cp311-cp311-win32.whl", hash = "sha256:b9d45d1dbb9de84894cc50efece5b09939752a2d75aab3a8b0cef6f3a35ecd6b"}, + {file = "numpy-1.26.1-cp311-cp311-win_amd64.whl", hash = "sha256:3649d566e2fc067597125428db15d60eb42a4e0897fc48d28cb75dc2e0454e53"}, + {file = "numpy-1.26.1-cp312-cp312-macosx_10_9_x86_64.whl", hash = "sha256:1d1bd82d539607951cac963388534da3b7ea0e18b149a53cf883d8f699178c0f"}, + {file = "numpy-1.26.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:afd5ced4e5a96dac6725daeb5242a35494243f2239244fad10a90ce58b071d24"}, + {file = "numpy-1.26.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:a03fb25610ef560a6201ff06df4f8105292ba56e7cdd196ea350d123fc32e24e"}, + {file = "numpy-1.26.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:dcfaf015b79d1f9f9c9fd0731a907407dc3e45769262d657d754c3a028586124"}, + {file = "numpy-1.26.1-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:e509cbc488c735b43b5ffea175235cec24bbc57b227ef1acc691725beb230d1c"}, + {file = "numpy-1.26.1-cp312-cp312-win32.whl", hash = "sha256:af22f3d8e228d84d1c0c44c1fbdeb80f97a15a0abe4f080960393a00db733b66"}, + {file = "numpy-1.26.1-cp312-cp312-win_amd64.whl", hash = "sha256:9f42284ebf91bdf32fafac29d29d4c07e5e9d1af862ea73686581773ef9e73a7"}, + {file = "numpy-1.26.1-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:bb894accfd16b867d8643fc2ba6c8617c78ba2828051e9a69511644ce86ce83e"}, + {file = "numpy-1.26.1-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:e44ccb93f30c75dfc0c3aa3ce38f33486a75ec9abadabd4e59f114994a9c4617"}, + {file = "numpy-1.26.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:9696aa2e35cc41e398a6d42d147cf326f8f9d81befcb399bc1ed7ffea339b64e"}, + {file = "numpy-1.26.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:a5b411040beead47a228bde3b2241100454a6abde9df139ed087bd73fc0a4908"}, + {file = "numpy-1.26.1-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:1e11668d6f756ca5ef534b5be8653d16c5352cbb210a5c2a79ff288e937010d5"}, + {file = "numpy-1.26.1-cp39-cp39-win32.whl", hash = "sha256:d1d2c6b7dd618c41e202c59c1413ef9b2c8e8a15f5039e344af64195459e3104"}, + {file = "numpy-1.26.1-cp39-cp39-win_amd64.whl", hash = "sha256:59227c981d43425ca5e5c01094d59eb14e8772ce6975d4b2fc1e106a833d5ae2"}, + {file = "numpy-1.26.1-pp39-pypy39_pp73-macosx_10_9_x86_64.whl", hash = "sha256:06934e1a22c54636a059215d6da99e23286424f316fddd979f5071093b648668"}, + {file = "numpy-1.26.1-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:76ff661a867d9272cd2a99eed002470f46dbe0943a5ffd140f49be84f68ffc42"}, + {file = "numpy-1.26.1-pp39-pypy39_pp73-win_amd64.whl", hash = "sha256:6965888d65d2848e8768824ca8288db0a81263c1efccec881cb35a0d805fcd2f"}, + {file = "numpy-1.26.1.tar.gz", hash = "sha256:c8c6c72d4a9f831f328efb1312642a1cafafaa88981d9ab76368d50d07d93cbe"}, ] [[package]] @@ -2078,35 +2087,35 @@ signedtoken = ["cryptography (>=3.0.0)", "pyjwt (>=2.0.0,<3)"] [[package]] name = "onnxruntime" -version = "1.16.0" +version = "1.16.1" description = "ONNX Runtime is a runtime accelerator for Machine Learning models" optional = false python-versions = "*" files = [ - {file = "onnxruntime-1.16.0-cp310-cp310-macosx_10_15_x86_64.whl", hash = "sha256:69c86ba3d90c166944c4a3c8a5b2a24a7bc45e68ae5997d83279af21ffd0f5f3"}, - {file = "onnxruntime-1.16.0-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:604a46aa2ad6a51f2fc4df1a984ea571a43aa02424aea93464c32ce02d23b3bb"}, - {file = "onnxruntime-1.16.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:a40660516b382031279fb690fc3d068ad004173c2bd12bbdc0bd0fe01ef8b7c3"}, - {file = "onnxruntime-1.16.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:349fd9c7875c1a76609d45b079484f8059adfb1fb87a30506934fb667ceab249"}, - {file = "onnxruntime-1.16.0-cp310-cp310-win32.whl", hash = "sha256:22c9e2f1a1f15b41b01195cd2520c013c22228efc4795ae4118048ea4118aad2"}, - {file = "onnxruntime-1.16.0-cp310-cp310-win_amd64.whl", hash = "sha256:b9667a131abfd226a728cc1c1ecf5cc5afa4fff37422f95a84bc22f7c175b57f"}, - {file = "onnxruntime-1.16.0-cp311-cp311-macosx_10_15_x86_64.whl", hash = "sha256:f7b292726a1f3fa4a483d7e902da083a5889a86a860dbc3a6479988cad342578"}, - {file = "onnxruntime-1.16.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:61eaf288a2482c5561f620fb686c80c32709e92724bbb59a5e4a0d349429e205"}, - {file = "onnxruntime-1.16.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:5fe2239d5821d5501eecccfe5c408485591b5d73eb76a61491a8f78179c2e65a"}, - {file = "onnxruntime-1.16.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:5a4924604fcdf1704b7f7e087b4c0b0e181c58367a687da55b1aec2705631943"}, - {file = "onnxruntime-1.16.0-cp311-cp311-win32.whl", hash = "sha256:55d8456f1ab28c32aec9c478b7638ed145102b03bb9b719b79e065ffc5de9c72"}, - {file = "onnxruntime-1.16.0-cp311-cp311-win_amd64.whl", hash = "sha256:c2a53ffd456187028c841ac7ed0d83b4c2b7e48bd2b1cf2a42d253ecf1e97cb3"}, - {file = "onnxruntime-1.16.0-cp38-cp38-macosx_10_15_x86_64.whl", hash = "sha256:bf5769aa4095cfe2503307867fa95b5f73732909ee21b67fe24da443af445925"}, - {file = "onnxruntime-1.16.0-cp38-cp38-macosx_11_0_arm64.whl", hash = "sha256:c0974deadf11ddab201d915a10517be00fa9d6816def56fa374e4c1a0008985a"}, - {file = "onnxruntime-1.16.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:99dccf1d2eba5ecd7b6c0e8e80d92d0030291f3506726c156e018a4d7a187c6f"}, - {file = "onnxruntime-1.16.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:0170ed05d3a8a7c24fe01fc262a6bc603837751f3bb273df7006a2da73f37fff"}, - {file = "onnxruntime-1.16.0-cp38-cp38-win32.whl", hash = "sha256:5ecd38e98ccdcbbaa7e529e96852f4c1c136559802354b76378d9a19532018ee"}, - {file = "onnxruntime-1.16.0-cp38-cp38-win_amd64.whl", hash = "sha256:1c585c60e9541a9bd4fb319ba9a3ef6122a28dcf4f3dbcdf014df44570cad6f8"}, - {file = "onnxruntime-1.16.0-cp39-cp39-macosx_10_15_x86_64.whl", hash = "sha256:efe59c1e51ad647fb18860233f5971e309961d09ca10697170ef9b7d9fa728f4"}, - {file = "onnxruntime-1.16.0-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:e3c9a9cccab8f6512a0c0207b2816dd8864f2f720f6e9df5cf01e30c4f80194f"}, - {file = "onnxruntime-1.16.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:dcf16a252308ec6e0737db7028b63fed0ac28fbad134f86216c0dfb051a31f38"}, - {file = "onnxruntime-1.16.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:f533aa90ee7189e88b6b612d6adae7d290971090598cfd47ce034ab0d106fc9c"}, - {file = "onnxruntime-1.16.0-cp39-cp39-win32.whl", hash = "sha256:306c7f5d8a0c24c65afb34f7deb0bc526defde2249e53538f1dce083945a2d6e"}, - {file = "onnxruntime-1.16.0-cp39-cp39-win_amd64.whl", hash = "sha256:df8a00a7b057ba497e2822175cc68731d84b89a6d50a3a2a3ec51e98e9c91125"}, + {file = "onnxruntime-1.16.1-cp310-cp310-macosx_10_15_x86_64.whl", hash = "sha256:28b2c7f444b4119950b69370801cd66067f403d19cbaf2a444735d7c269cce4a"}, + {file = "onnxruntime-1.16.1-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:c24e04f33e7899f6aebb03ed51e51d346c1f906b05c5569d58ac9a12d38a2f58"}, + {file = "onnxruntime-1.16.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:9fa93b166f2d97063dc9f33c5118c5729a4a5dd5617296b6dbef42f9047b3e81"}, + {file = "onnxruntime-1.16.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:042dd9201b3016ee18f8f8bc4609baf11ff34ca1ff489c0a46bcd30919bf883d"}, + {file = "onnxruntime-1.16.1-cp310-cp310-win32.whl", hash = "sha256:c20aa0591f305012f1b21aad607ed96917c86ae7aede4a4dd95824b3d124ceb7"}, + {file = "onnxruntime-1.16.1-cp310-cp310-win_amd64.whl", hash = "sha256:5581873e578917bea76d6434ee7337e28195d03488dcf72d161d08e9398c6249"}, + {file = "onnxruntime-1.16.1-cp311-cp311-macosx_10_15_x86_64.whl", hash = "sha256:ef8c0c8abf5f309aa1caf35941380839dc5f7a2fa53da533be4a3f254993f120"}, + {file = "onnxruntime-1.16.1-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:e680380bea35a137cbc3efd67a17486e96972901192ad3026ee79c8d8fe264f7"}, + {file = "onnxruntime-1.16.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:5e62cc38ce1a669013d0a596d984762dc9c67c56f60ecfeee0d5ad36da5863f6"}, + {file = "onnxruntime-1.16.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:025c7a4d57bd2e63b8a0f84ad3df53e419e3df1cc72d63184f2aae807b17c13c"}, + {file = "onnxruntime-1.16.1-cp311-cp311-win32.whl", hash = "sha256:9ad074057fa8d028df248b5668514088cb0937b6ac5954073b7fb9b2891ffc8c"}, + {file = "onnxruntime-1.16.1-cp311-cp311-win_amd64.whl", hash = "sha256:d5e43a3478bffc01f817ecf826de7b25a2ca1bca8547d70888594ab80a77ad24"}, + {file = "onnxruntime-1.16.1-cp38-cp38-macosx_10_15_x86_64.whl", hash = "sha256:3aef4d70b0930e29a8943eab248cd1565664458d3a62b2276bd11181f28fd0a3"}, + {file = "onnxruntime-1.16.1-cp38-cp38-macosx_11_0_arm64.whl", hash = "sha256:55a7b843a57c8ca0c8ff169428137958146081d5d76f1a6dd444c4ffcd37c3c2"}, + {file = "onnxruntime-1.16.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:62c631af1941bf3b5f7d063d24c04aacce8cff0794e157c497e315e89ac5ad7b"}, + {file = "onnxruntime-1.16.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:5671f296c3d5c233f601e97a10ab5a1dd8e65ba35c7b7b0c253332aba9dff330"}, + {file = "onnxruntime-1.16.1-cp38-cp38-win32.whl", hash = "sha256:eb3802305023dd05e16848d4e22b41f8147247894309c0c27122aaa08793b3d2"}, + {file = "onnxruntime-1.16.1-cp38-cp38-win_amd64.whl", hash = "sha256:fecfb07443d09d271b1487f401fbdf1ba0c829af6fd4fe8f6af25f71190e7eb9"}, + {file = "onnxruntime-1.16.1-cp39-cp39-macosx_10_15_x86_64.whl", hash = "sha256:de3e12094234db6545c67adbf801874b4eb91e9f299bda34c62967ef0050960f"}, + {file = "onnxruntime-1.16.1-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:ff723c2a5621b5e7103f3be84d5aae1e03a20621e72219dddceae81f65f240af"}, + {file = "onnxruntime-1.16.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:14a7fb3073aaf6b462e3d7fb433320f7700558a8892e5021780522dc4574292a"}, + {file = "onnxruntime-1.16.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:963159f1f699b0454cd72fcef3276c8a1aab9389a7b301bcd8e320fb9d9e8597"}, + {file = "onnxruntime-1.16.1-cp39-cp39-win32.whl", hash = "sha256:85771adb75190db9364b25ddec353ebf07635b83eb94b64ed014f1f6d57a3857"}, + {file = "onnxruntime-1.16.1-cp39-cp39-win_amd64.whl", hash = "sha256:d32d2b30799c1f950123c60ae8390818381fd5f88bdf3627eeca10071c155dc5"}, ] [package.dependencies] @@ -2261,65 +2270,65 @@ files = [ [[package]] name = "pillow" -version = "10.0.1" +version = "10.1.0" description = "Python Imaging Library (Fork)" optional = false python-versions = ">=3.8" files = [ - {file = "Pillow-10.0.1-cp310-cp310-macosx_10_10_x86_64.whl", hash = "sha256:8f06be50669087250f319b706decf69ca71fdecd829091a37cc89398ca4dc17a"}, - {file = "Pillow-10.0.1-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:50bd5f1ebafe9362ad622072a1d2f5850ecfa44303531ff14353a4059113b12d"}, - {file = "Pillow-10.0.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:e6a90167bcca1216606223a05e2cf991bb25b14695c518bc65639463d7db722d"}, - {file = "Pillow-10.0.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:f11c9102c56ffb9ca87134bd025a43d2aba3f1155f508eff88f694b33a9c6d19"}, - {file = "Pillow-10.0.1-cp310-cp310-manylinux_2_28_aarch64.whl", hash = "sha256:186f7e04248103482ea6354af6d5bcedb62941ee08f7f788a1c7707bc720c66f"}, - {file = "Pillow-10.0.1-cp310-cp310-manylinux_2_28_x86_64.whl", hash = "sha256:0462b1496505a3462d0f35dc1c4d7b54069747d65d00ef48e736acda2c8cbdff"}, - {file = "Pillow-10.0.1-cp310-cp310-musllinux_1_1_aarch64.whl", hash = "sha256:d889b53ae2f030f756e61a7bff13684dcd77e9af8b10c6048fb2c559d6ed6eaf"}, - {file = "Pillow-10.0.1-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:552912dbca585b74d75279a7570dd29fa43b6d93594abb494ebb31ac19ace6bd"}, - {file = "Pillow-10.0.1-cp310-cp310-win_amd64.whl", hash = "sha256:787bb0169d2385a798888e1122c980c6eff26bf941a8ea79747d35d8f9210ca0"}, - {file = "Pillow-10.0.1-cp311-cp311-macosx_10_10_x86_64.whl", hash = "sha256:fd2a5403a75b54661182b75ec6132437a181209b901446ee5724b589af8edef1"}, - {file = "Pillow-10.0.1-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:2d7e91b4379f7a76b31c2dda84ab9e20c6220488e50f7822e59dac36b0cd92b1"}, - {file = "Pillow-10.0.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:19e9adb3f22d4c416e7cd79b01375b17159d6990003633ff1d8377e21b7f1b21"}, - {file = "Pillow-10.0.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:93139acd8109edcdeffd85e3af8ae7d88b258b3a1e13a038f542b79b6d255c54"}, - {file = "Pillow-10.0.1-cp311-cp311-manylinux_2_28_aarch64.whl", hash = "sha256:92a23b0431941a33242b1f0ce6c88a952e09feeea9af4e8be48236a68ffe2205"}, - {file = "Pillow-10.0.1-cp311-cp311-manylinux_2_28_x86_64.whl", hash = "sha256:cbe68deb8580462ca0d9eb56a81912f59eb4542e1ef8f987405e35a0179f4ea2"}, - {file = "Pillow-10.0.1-cp311-cp311-musllinux_1_1_aarch64.whl", hash = "sha256:522ff4ac3aaf839242c6f4e5b406634bfea002469656ae8358644fc6c4856a3b"}, - {file = "Pillow-10.0.1-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:84efb46e8d881bb06b35d1d541aa87f574b58e87f781cbba8d200daa835b42e1"}, - {file = "Pillow-10.0.1-cp311-cp311-win_amd64.whl", hash = "sha256:898f1d306298ff40dc1b9ca24824f0488f6f039bc0e25cfb549d3195ffa17088"}, - {file = "Pillow-10.0.1-cp312-cp312-macosx_10_10_x86_64.whl", hash = "sha256:bcf1207e2f2385a576832af02702de104be71301c2696d0012b1b93fe34aaa5b"}, - {file = "Pillow-10.0.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:5d6c9049c6274c1bb565021367431ad04481ebb54872edecfcd6088d27edd6ed"}, - {file = "Pillow-10.0.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:28444cb6ad49726127d6b340217f0627abc8732f1194fd5352dec5e6a0105635"}, - {file = "Pillow-10.0.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:de596695a75496deb3b499c8c4f8e60376e0516e1a774e7bc046f0f48cd620ad"}, - {file = "Pillow-10.0.1-cp312-cp312-manylinux_2_28_aarch64.whl", hash = "sha256:2872f2d7846cf39b3dbff64bc1104cc48c76145854256451d33c5faa55c04d1a"}, - {file = "Pillow-10.0.1-cp312-cp312-manylinux_2_28_x86_64.whl", hash = "sha256:4ce90f8a24e1c15465048959f1e94309dfef93af272633e8f37361b824532e91"}, - {file = "Pillow-10.0.1-cp312-cp312-musllinux_1_1_aarch64.whl", hash = "sha256:ee7810cf7c83fa227ba9125de6084e5e8b08c59038a7b2c9045ef4dde61663b4"}, - {file = "Pillow-10.0.1-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:b1be1c872b9b5fcc229adeadbeb51422a9633abd847c0ff87dc4ef9bb184ae08"}, - {file = "Pillow-10.0.1-cp312-cp312-win_amd64.whl", hash = "sha256:98533fd7fa764e5f85eebe56c8e4094db912ccbe6fbf3a58778d543cadd0db08"}, - {file = "Pillow-10.0.1-cp38-cp38-macosx_10_10_x86_64.whl", hash = "sha256:764d2c0daf9c4d40ad12fbc0abd5da3af7f8aa11daf87e4fa1b834000f4b6b0a"}, - {file = "Pillow-10.0.1-cp38-cp38-macosx_11_0_arm64.whl", hash = "sha256:fcb59711009b0168d6ee0bd8fb5eb259c4ab1717b2f538bbf36bacf207ef7a68"}, - {file = "Pillow-10.0.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:697a06bdcedd473b35e50a7e7506b1d8ceb832dc238a336bd6f4f5aa91a4b500"}, - {file = "Pillow-10.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:9f665d1e6474af9f9da5e86c2a3a2d2d6204e04d5af9c06b9d42afa6ebde3f21"}, - {file = "Pillow-10.0.1-cp38-cp38-manylinux_2_28_aarch64.whl", hash = "sha256:2fa6dd2661838c66f1a5473f3b49ab610c98a128fc08afbe81b91a1f0bf8c51d"}, - {file = "Pillow-10.0.1-cp38-cp38-manylinux_2_28_x86_64.whl", hash = "sha256:3a04359f308ebee571a3127fdb1bd01f88ba6f6fb6d087f8dd2e0d9bff43f2a7"}, - {file = "Pillow-10.0.1-cp38-cp38-musllinux_1_1_aarch64.whl", hash = "sha256:723bd25051454cea9990203405fa6b74e043ea76d4968166dfd2569b0210886a"}, - {file = "Pillow-10.0.1-cp38-cp38-musllinux_1_1_x86_64.whl", hash = "sha256:71671503e3015da1b50bd18951e2f9daf5b6ffe36d16f1eb2c45711a301521a7"}, - {file = "Pillow-10.0.1-cp38-cp38-win_amd64.whl", hash = "sha256:44e7e4587392953e5e251190a964675f61e4dae88d1e6edbe9f36d6243547ff3"}, - {file = "Pillow-10.0.1-cp39-cp39-macosx_10_10_x86_64.whl", hash = "sha256:3855447d98cced8670aaa63683808df905e956f00348732448b5a6df67ee5849"}, - {file = "Pillow-10.0.1-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:ed2d9c0704f2dc4fa980b99d565c0c9a543fe5101c25b3d60488b8ba80f0cce1"}, - {file = "Pillow-10.0.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:f5bb289bb835f9fe1a1e9300d011eef4d69661bb9b34d5e196e5e82c4cb09b37"}, - {file = "Pillow-10.0.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:3a0d3e54ab1df9df51b914b2233cf779a5a10dfd1ce339d0421748232cea9876"}, - {file = "Pillow-10.0.1-cp39-cp39-manylinux_2_28_aarch64.whl", hash = "sha256:2cc6b86ece42a11f16f55fe8903595eff2b25e0358dec635d0a701ac9586588f"}, - {file = "Pillow-10.0.1-cp39-cp39-manylinux_2_28_x86_64.whl", hash = "sha256:ca26ba5767888c84bf5a0c1a32f069e8204ce8c21d00a49c90dabeba00ce0145"}, - {file = "Pillow-10.0.1-cp39-cp39-musllinux_1_1_aarch64.whl", hash = "sha256:f0b4b06da13275bc02adfeb82643c4a6385bd08d26f03068c2796f60d125f6f2"}, - {file = "Pillow-10.0.1-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:bc2e3069569ea9dbe88d6b8ea38f439a6aad8f6e7a6283a38edf61ddefb3a9bf"}, - {file = "Pillow-10.0.1-cp39-cp39-win_amd64.whl", hash = "sha256:8b451d6ead6e3500b6ce5c7916a43d8d8d25ad74b9102a629baccc0808c54971"}, - {file = "Pillow-10.0.1-pp310-pypy310_pp73-macosx_10_10_x86_64.whl", hash = "sha256:32bec7423cdf25c9038fef614a853c9d25c07590e1a870ed471f47fb80b244db"}, - {file = "Pillow-10.0.1-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:b7cf63d2c6928b51d35dfdbda6f2c1fddbe51a6bc4a9d4ee6ea0e11670dd981e"}, - {file = "Pillow-10.0.1-pp310-pypy310_pp73-manylinux_2_28_x86_64.whl", hash = "sha256:f6d3d4c905e26354e8f9d82548475c46d8e0889538cb0657aa9c6f0872a37aa4"}, - {file = "Pillow-10.0.1-pp310-pypy310_pp73-win_amd64.whl", hash = "sha256:847e8d1017c741c735d3cd1883fa7b03ded4f825a6e5fcb9378fd813edee995f"}, - {file = "Pillow-10.0.1-pp39-pypy39_pp73-macosx_10_10_x86_64.whl", hash = "sha256:7f771e7219ff04b79e231d099c0a28ed83aa82af91fd5fa9fdb28f5b8d5addaf"}, - {file = "Pillow-10.0.1-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:459307cacdd4138edee3875bbe22a2492519e060660eaf378ba3b405d1c66317"}, - {file = "Pillow-10.0.1-pp39-pypy39_pp73-manylinux_2_28_x86_64.whl", hash = "sha256:b059ac2c4c7a97daafa7dc850b43b2d3667def858a4f112d1aa082e5c3d6cf7d"}, - {file = "Pillow-10.0.1-pp39-pypy39_pp73-win_amd64.whl", hash = "sha256:d6caf3cd38449ec3cd8a68b375e0c6fe4b6fd04edb6c9766b55ef84a6e8ddf2d"}, - {file = "Pillow-10.0.1.tar.gz", hash = "sha256:d72967b06be9300fed5cfbc8b5bafceec48bf7cdc7dab66b1d2549035287191d"}, + {file = "Pillow-10.1.0-cp310-cp310-macosx_10_10_x86_64.whl", hash = "sha256:1ab05f3db77e98f93964697c8efc49c7954b08dd61cff526b7f2531a22410106"}, + {file = "Pillow-10.1.0-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:6932a7652464746fcb484f7fc3618e6503d2066d853f68a4bd97193a3996e273"}, + {file = "Pillow-10.1.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:a5f63b5a68daedc54c7c3464508d8c12075e56dcfbd42f8c1bf40169061ae666"}, + {file = "Pillow-10.1.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:c0949b55eb607898e28eaccb525ab104b2d86542a85c74baf3a6dc24002edec2"}, + {file = "Pillow-10.1.0-cp310-cp310-manylinux_2_28_aarch64.whl", hash = "sha256:ae88931f93214777c7a3aa0a8f92a683f83ecde27f65a45f95f22d289a69e593"}, + {file = "Pillow-10.1.0-cp310-cp310-manylinux_2_28_x86_64.whl", hash = "sha256:b0eb01ca85b2361b09480784a7931fc648ed8b7836f01fb9241141b968feb1db"}, + {file = "Pillow-10.1.0-cp310-cp310-musllinux_1_1_aarch64.whl", hash = "sha256:d27b5997bdd2eb9fb199982bb7eb6164db0426904020dc38c10203187ae2ff2f"}, + {file = "Pillow-10.1.0-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:7df5608bc38bd37ef585ae9c38c9cd46d7c81498f086915b0f97255ea60c2818"}, + {file = "Pillow-10.1.0-cp310-cp310-win_amd64.whl", hash = "sha256:41f67248d92a5e0a2076d3517d8d4b1e41a97e2df10eb8f93106c89107f38b57"}, + {file = "Pillow-10.1.0-cp311-cp311-macosx_10_10_x86_64.whl", hash = "sha256:1fb29c07478e6c06a46b867e43b0bcdb241b44cc52be9bc25ce5944eed4648e7"}, + {file = "Pillow-10.1.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:2cdc65a46e74514ce742c2013cd4a2d12e8553e3a2563c64879f7c7e4d28bce7"}, + {file = "Pillow-10.1.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:50d08cd0a2ecd2a8657bd3d82c71efd5a58edb04d9308185d66c3a5a5bed9610"}, + {file = "Pillow-10.1.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:062a1610e3bc258bff2328ec43f34244fcec972ee0717200cb1425214fe5b839"}, + {file = "Pillow-10.1.0-cp311-cp311-manylinux_2_28_aarch64.whl", hash = "sha256:61f1a9d247317fa08a308daaa8ee7b3f760ab1809ca2da14ecc88ae4257d6172"}, + {file = "Pillow-10.1.0-cp311-cp311-manylinux_2_28_x86_64.whl", hash = "sha256:a646e48de237d860c36e0db37ecaecaa3619e6f3e9d5319e527ccbc8151df061"}, + {file = "Pillow-10.1.0-cp311-cp311-musllinux_1_1_aarch64.whl", hash = "sha256:47e5bf85b80abc03be7455c95b6d6e4896a62f6541c1f2ce77a7d2bb832af262"}, + {file = "Pillow-10.1.0-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:a92386125e9ee90381c3369f57a2a50fa9e6aa8b1cf1d9c4b200d41a7dd8e992"}, + {file = "Pillow-10.1.0-cp311-cp311-win_amd64.whl", hash = "sha256:0f7c276c05a9767e877a0b4c5050c8bee6a6d960d7f0c11ebda6b99746068c2a"}, + {file = "Pillow-10.1.0-cp312-cp312-macosx_10_10_x86_64.whl", hash = "sha256:a89b8312d51715b510a4fe9fc13686283f376cfd5abca8cd1c65e4c76e21081b"}, + {file = "Pillow-10.1.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:00f438bb841382b15d7deb9a05cc946ee0f2c352653c7aa659e75e592f6fa17d"}, + {file = "Pillow-10.1.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:3d929a19f5469b3f4df33a3df2983db070ebb2088a1e145e18facbc28cae5b27"}, + {file = "Pillow-10.1.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:9a92109192b360634a4489c0c756364c0c3a2992906752165ecb50544c251312"}, + {file = "Pillow-10.1.0-cp312-cp312-manylinux_2_28_aarch64.whl", hash = "sha256:0248f86b3ea061e67817c47ecbe82c23f9dd5d5226200eb9090b3873d3ca32de"}, + {file = "Pillow-10.1.0-cp312-cp312-manylinux_2_28_x86_64.whl", hash = "sha256:9882a7451c680c12f232a422730f986a1fcd808da0fd428f08b671237237d651"}, + {file = "Pillow-10.1.0-cp312-cp312-musllinux_1_1_aarch64.whl", hash = "sha256:1c3ac5423c8c1da5928aa12c6e258921956757d976405e9467c5f39d1d577a4b"}, + {file = "Pillow-10.1.0-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:806abdd8249ba3953c33742506fe414880bad78ac25cc9a9b1c6ae97bedd573f"}, + {file = "Pillow-10.1.0-cp312-cp312-win_amd64.whl", hash = "sha256:eaed6977fa73408b7b8a24e8b14e59e1668cfc0f4c40193ea7ced8e210adf996"}, + {file = "Pillow-10.1.0-cp38-cp38-macosx_10_10_x86_64.whl", hash = "sha256:fe1e26e1ffc38be097f0ba1d0d07fcade2bcfd1d023cda5b29935ae8052bd793"}, + {file = "Pillow-10.1.0-cp38-cp38-macosx_11_0_arm64.whl", hash = "sha256:7a7e3daa202beb61821c06d2517428e8e7c1aab08943e92ec9e5755c2fc9ba5e"}, + {file = "Pillow-10.1.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:24fadc71218ad2b8ffe437b54876c9382b4a29e030a05a9879f615091f42ffc2"}, + {file = "Pillow-10.1.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:fa1d323703cfdac2036af05191b969b910d8f115cf53093125e4058f62012c9a"}, + {file = "Pillow-10.1.0-cp38-cp38-manylinux_2_28_aarch64.whl", hash = "sha256:912e3812a1dbbc834da2b32299b124b5ddcb664ed354916fd1ed6f193f0e2d01"}, + {file = "Pillow-10.1.0-cp38-cp38-manylinux_2_28_x86_64.whl", hash = "sha256:7dbaa3c7de82ef37e7708521be41db5565004258ca76945ad74a8e998c30af8d"}, + {file = "Pillow-10.1.0-cp38-cp38-musllinux_1_1_aarch64.whl", hash = "sha256:9d7bc666bd8c5a4225e7ac71f2f9d12466ec555e89092728ea0f5c0c2422ea80"}, + {file = "Pillow-10.1.0-cp38-cp38-musllinux_1_1_x86_64.whl", hash = "sha256:baada14941c83079bf84c037e2d8b7506ce201e92e3d2fa0d1303507a8538212"}, + {file = "Pillow-10.1.0-cp38-cp38-win_amd64.whl", hash = "sha256:2ef6721c97894a7aa77723740a09547197533146fba8355e86d6d9a4a1056b14"}, + {file = "Pillow-10.1.0-cp39-cp39-macosx_10_10_x86_64.whl", hash = "sha256:0a026c188be3b443916179f5d04548092e253beb0c3e2ee0a4e2cdad72f66099"}, + {file = "Pillow-10.1.0-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:04f6f6149f266a100374ca3cc368b67fb27c4af9f1cc8cb6306d849dcdf12616"}, + {file = "Pillow-10.1.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:bb40c011447712d2e19cc261c82655f75f32cb724788df315ed992a4d65696bb"}, + {file = "Pillow-10.1.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:1a8413794b4ad9719346cd9306118450b7b00d9a15846451549314a58ac42219"}, + {file = "Pillow-10.1.0-cp39-cp39-manylinux_2_28_aarch64.whl", hash = "sha256:c9aeea7b63edb7884b031a35305629a7593272b54f429a9869a4f63a1bf04c34"}, + {file = "Pillow-10.1.0-cp39-cp39-manylinux_2_28_x86_64.whl", hash = "sha256:b4005fee46ed9be0b8fb42be0c20e79411533d1fd58edabebc0dd24626882cfd"}, + {file = "Pillow-10.1.0-cp39-cp39-musllinux_1_1_aarch64.whl", hash = "sha256:4d0152565c6aa6ebbfb1e5d8624140a440f2b99bf7afaafbdbf6430426497f28"}, + {file = "Pillow-10.1.0-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:d921bc90b1defa55c9917ca6b6b71430e4286fc9e44c55ead78ca1a9f9eba5f2"}, + {file = "Pillow-10.1.0-cp39-cp39-win_amd64.whl", hash = "sha256:cfe96560c6ce2f4c07d6647af2d0f3c54cc33289894ebd88cfbb3bcd5391e256"}, + {file = "Pillow-10.1.0-pp310-pypy310_pp73-macosx_10_10_x86_64.whl", hash = "sha256:937bdc5a7f5343d1c97dc98149a0be7eb9704e937fe3dc7140e229ae4fc572a7"}, + {file = "Pillow-10.1.0-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:b1c25762197144e211efb5f4e8ad656f36c8d214d390585d1d21281f46d556ba"}, + {file = "Pillow-10.1.0-pp310-pypy310_pp73-manylinux_2_28_x86_64.whl", hash = "sha256:afc8eef765d948543a4775f00b7b8c079b3321d6b675dde0d02afa2ee23000b4"}, + {file = "Pillow-10.1.0-pp310-pypy310_pp73-win_amd64.whl", hash = "sha256:883f216eac8712b83a63f41b76ddfb7b2afab1b74abbb413c5df6680f071a6b9"}, + {file = "Pillow-10.1.0-pp39-pypy39_pp73-macosx_10_10_x86_64.whl", hash = "sha256:b920e4d028f6442bea9a75b7491c063f0b9a3972520731ed26c83e254302eb1e"}, + {file = "Pillow-10.1.0-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:1c41d960babf951e01a49c9746f92c5a7e0d939d1652d7ba30f6b3090f27e412"}, + {file = "Pillow-10.1.0-pp39-pypy39_pp73-manylinux_2_28_x86_64.whl", hash = "sha256:1fafabe50a6977ac70dfe829b2d5735fd54e190ab55259ec8aea4aaea412fa0b"}, + {file = "Pillow-10.1.0-pp39-pypy39_pp73-win_amd64.whl", hash = "sha256:3b834f4b16173e5b92ab6566f0473bfb09f939ba14b23b8da1f54fa63e4b623f"}, + {file = "Pillow-10.1.0.tar.gz", hash = "sha256:e6bf8de6c36ed96c86ea3b6e1d5273c53f46ef518a062464cd7ef5dd2cf92e38"}, ] [package.extras] @@ -2435,25 +2444,27 @@ files = [ [[package]] name = "psutil" -version = "5.9.5" +version = "5.9.6" description = "Cross-platform lib for process and system monitoring in Python." optional = false -python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*" -files = [ - {file = "psutil-5.9.5-cp27-cp27m-macosx_10_9_x86_64.whl", hash = "sha256:be8929ce4313f9f8146caad4272f6abb8bf99fc6cf59344a3167ecd74f4f203f"}, - {file = "psutil-5.9.5-cp27-cp27m-manylinux2010_i686.whl", hash = "sha256:ab8ed1a1d77c95453db1ae00a3f9c50227ebd955437bcf2a574ba8adbf6a74d5"}, - {file = "psutil-5.9.5-cp27-cp27m-manylinux2010_x86_64.whl", hash = "sha256:4aef137f3345082a3d3232187aeb4ac4ef959ba3d7c10c33dd73763fbc063da4"}, - {file = "psutil-5.9.5-cp27-cp27mu-manylinux2010_i686.whl", hash = "sha256:ea8518d152174e1249c4f2a1c89e3e6065941df2fa13a1ab45327716a23c2b48"}, - {file = "psutil-5.9.5-cp27-cp27mu-manylinux2010_x86_64.whl", hash = "sha256:acf2aef9391710afded549ff602b5887d7a2349831ae4c26be7c807c0a39fac4"}, - {file = "psutil-5.9.5-cp27-none-win32.whl", hash = "sha256:5b9b8cb93f507e8dbaf22af6a2fd0ccbe8244bf30b1baad6b3954e935157ae3f"}, - {file = "psutil-5.9.5-cp27-none-win_amd64.whl", hash = "sha256:8c5f7c5a052d1d567db4ddd231a9d27a74e8e4a9c3f44b1032762bd7b9fdcd42"}, - {file = "psutil-5.9.5-cp36-abi3-macosx_10_9_x86_64.whl", hash = "sha256:3c6f686f4225553615612f6d9bc21f1c0e305f75d7d8454f9b46e901778e7217"}, - {file = "psutil-5.9.5-cp36-abi3-manylinux_2_12_i686.manylinux2010_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:7a7dd9997128a0d928ed4fb2c2d57e5102bb6089027939f3b722f3a210f9a8da"}, - {file = "psutil-5.9.5-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:89518112647f1276b03ca97b65cc7f64ca587b1eb0278383017c2a0dcc26cbe4"}, - {file = "psutil-5.9.5-cp36-abi3-win32.whl", hash = "sha256:104a5cc0e31baa2bcf67900be36acde157756b9c44017b86b2c049f11957887d"}, - {file = "psutil-5.9.5-cp36-abi3-win_amd64.whl", hash = "sha256:b258c0c1c9d145a1d5ceffab1134441c4c5113b2417fafff7315a917a026c3c9"}, - {file = "psutil-5.9.5-cp38-abi3-macosx_11_0_arm64.whl", hash = "sha256:c607bb3b57dc779d55e1554846352b4e358c10fff3abf3514a7a6601beebdb30"}, - {file = "psutil-5.9.5.tar.gz", hash = "sha256:5410638e4df39c54d957fc51ce03048acd8e6d60abc0f5107af51e5fb566eb3c"}, +python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*, !=3.5.*" +files = [ + {file = "psutil-5.9.6-cp27-cp27m-macosx_10_9_x86_64.whl", hash = "sha256:fb8a697f11b0f5994550555fcfe3e69799e5b060c8ecf9e2f75c69302cc35c0d"}, + {file = "psutil-5.9.6-cp27-cp27m-manylinux2010_i686.whl", hash = "sha256:91ecd2d9c00db9817a4b4192107cf6954addb5d9d67a969a4f436dbc9200f88c"}, + {file = "psutil-5.9.6-cp27-cp27m-manylinux2010_x86_64.whl", hash = "sha256:10e8c17b4f898d64b121149afb136c53ea8b68c7531155147867b7b1ac9e7e28"}, + {file = "psutil-5.9.6-cp27-cp27mu-manylinux2010_i686.whl", hash = "sha256:18cd22c5db486f33998f37e2bb054cc62fd06646995285e02a51b1e08da97017"}, + {file = "psutil-5.9.6-cp27-cp27mu-manylinux2010_x86_64.whl", hash = "sha256:ca2780f5e038379e520281e4c032dddd086906ddff9ef0d1b9dcf00710e5071c"}, + {file = "psutil-5.9.6-cp27-none-win32.whl", hash = "sha256:70cb3beb98bc3fd5ac9ac617a327af7e7f826373ee64c80efd4eb2856e5051e9"}, + {file = "psutil-5.9.6-cp27-none-win_amd64.whl", hash = "sha256:51dc3d54607c73148f63732c727856f5febec1c7c336f8f41fcbd6315cce76ac"}, + {file = "psutil-5.9.6-cp36-abi3-macosx_10_9_x86_64.whl", hash = "sha256:c69596f9fc2f8acd574a12d5f8b7b1ba3765a641ea5d60fb4736bf3c08a8214a"}, + {file = "psutil-5.9.6-cp36-abi3-manylinux_2_12_i686.manylinux2010_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:92e0cc43c524834af53e9d3369245e6cc3b130e78e26100d1f63cdb0abeb3d3c"}, + {file = "psutil-5.9.6-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:748c9dd2583ed86347ed65d0035f45fa8c851e8d90354c122ab72319b5f366f4"}, + {file = "psutil-5.9.6-cp36-cp36m-win32.whl", hash = "sha256:3ebf2158c16cc69db777e3c7decb3c0f43a7af94a60d72e87b2823aebac3d602"}, + {file = "psutil-5.9.6-cp36-cp36m-win_amd64.whl", hash = "sha256:ff18b8d1a784b810df0b0fff3bcb50ab941c3b8e2c8de5726f9c71c601c611aa"}, + {file = "psutil-5.9.6-cp37-abi3-win32.whl", hash = "sha256:a6f01f03bf1843280f4ad16f4bde26b817847b4c1a0db59bf6419807bc5ce05c"}, + {file = "psutil-5.9.6-cp37-abi3-win_amd64.whl", hash = "sha256:6e5fb8dc711a514da83098bc5234264e551ad980cec5f85dabf4d38ed6f15e9a"}, + {file = "psutil-5.9.6-cp38-abi3-macosx_11_0_arm64.whl", hash = "sha256:daecbcbd29b289aac14ece28eca6a3e60aa361754cf6da3dfb20d4d32b6c7f57"}, + {file = "psutil-5.9.6.tar.gz", hash = "sha256:e4b92ddcd7dd4cdd3f900180ea1e104932c7bce234fb88976e2a3b296441225a"}, ] [package.extras] @@ -3201,6 +3212,11 @@ files = [ {file = "scikit_learn-1.3.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:f66eddfda9d45dd6cadcd706b65669ce1df84b8549875691b1f403730bdef217"}, {file = "scikit_learn-1.3.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:c6448c37741145b241eeac617028ba6ec2119e1339b1385c9720dae31367f2be"}, {file = "scikit_learn-1.3.1-cp311-cp311-win_amd64.whl", hash = "sha256:c413c2c850241998168bbb3bd1bb59ff03b1195a53864f0b80ab092071af6028"}, + {file = "scikit_learn-1.3.1-cp312-cp312-macosx_10_9_x86_64.whl", hash = "sha256:ef540e09873e31569bc8b02c8a9f745ee04d8e1263255a15c9969f6f5caa627f"}, + {file = "scikit_learn-1.3.1-cp312-cp312-macosx_12_0_arm64.whl", hash = "sha256:9147a3a4df4d401e618713880be023e36109c85d8569b3bf5377e6cd3fecdeac"}, + {file = "scikit_learn-1.3.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:d2cd3634695ad192bf71645702b3df498bd1e246fc2d529effdb45a06ab028b4"}, + {file = "scikit_learn-1.3.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:0c275a06c5190c5ce00af0acbb61c06374087949f643ef32d355ece12c4db043"}, + {file = "scikit_learn-1.3.1-cp312-cp312-win_amd64.whl", hash = "sha256:0e1aa8f206d0de814b81b41d60c1ce31f7f2c7354597af38fae46d9c47c45122"}, {file = "scikit_learn-1.3.1-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:52b77cc08bd555969ec5150788ed50276f5ef83abb72e6f469c5b91a0009bbca"}, {file = "scikit_learn-1.3.1-cp38-cp38-macosx_12_0_arm64.whl", hash = "sha256:a683394bc3f80b7c312c27f9b14ebea7766b1f0a34faf1a2e9158d80e860ec26"}, {file = "scikit_learn-1.3.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:a15d964d9eb181c79c190d3dbc2fff7338786bf017e9039571418a1d53dab236"}, @@ -3707,31 +3723,28 @@ tests = ["absl-py", "pytest", "tensorflow-datasets (>=3.2.0)"] [[package]] name = "tensorstore" -version = "0.1.45" +version = "0.1.46" description = "Read and write large, multi-dimensional arrays" optional = false -python-versions = ">=3.8" +python-versions = ">=3.9" files = [ - {file = "tensorstore-0.1.45-cp310-cp310-macosx_10_14_x86_64.whl", hash = "sha256:2ff6e5177ba2702f348bef3edc37619aa7646e43f33d1a567ba267db455699e4"}, - {file = "tensorstore-0.1.45-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:9bc7cde6318363eb9d35fc6cacb6fcd5d7a03b0ee57bdd69249108c0164692d8"}, - {file = "tensorstore-0.1.45-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:405bf40271eed5632a566cdb935beba87d9896d2f80caf75386febb529ddba45"}, - {file = "tensorstore-0.1.45-cp310-cp310-win_amd64.whl", hash = "sha256:537805adb06fff2ce9a259b81920af4c34a20f752fa28205e722b7e58a60c790"}, - {file = "tensorstore-0.1.45-cp311-cp311-macosx_10_14_x86_64.whl", hash = "sha256:73df4ddafe4da8e0f919ed5a75f48839013da3a99128a719fe730855252051a6"}, - {file = "tensorstore-0.1.45-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:f38bba6fc0668a950b76752c743b66851c4fc7360857e8b37a4f7a4e9786760b"}, - {file = "tensorstore-0.1.45-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:ca212d127fcc4debb9f6b4274d584fe7724b2a349ca9444258a4127878dc3033"}, - {file = "tensorstore-0.1.45-cp311-cp311-win_amd64.whl", hash = "sha256:a8960f0e546ee493ed67b77998859f0cb94772ea31e865bf76b0c79976ac9204"}, - {file = "tensorstore-0.1.45-cp38-cp38-macosx_10_14_x86_64.whl", hash = "sha256:871a1fde0712a153ac44774ddace3ad841609ff5be792734d44cffb520258e92"}, - {file = "tensorstore-0.1.45-cp38-cp38-macosx_11_0_arm64.whl", hash = "sha256:0ce1a3d2bdbdb2c1102100ee23fa99a95b0bcdee9773862622d7da833516c8c9"}, - {file = "tensorstore-0.1.45-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:8659688ec9d89cdd71046c35b3c84cf92cd8c88251e6068f8a99d6991a965028"}, - {file = "tensorstore-0.1.45-cp38-cp38-win_amd64.whl", hash = "sha256:c034fec18b6e3174d26df1cdd91ec67b720fc5de7ef0cc3804017dad8c211622"}, - {file = "tensorstore-0.1.45-cp39-cp39-macosx_10_14_x86_64.whl", hash = "sha256:4915aee8355ee7dbc6f534d77a28c18001e19696f44f78760ec42845ac51edee"}, - {file = "tensorstore-0.1.45-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:4346ab7afa0963dcaa8e64388a2bedab741c790786b577326a0b174d226c9320"}, - {file = "tensorstore-0.1.45-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:05196a0464ce51867f1edd96e992fe01281de283b034d434ca6e81db319368c0"}, - {file = "tensorstore-0.1.45-cp39-cp39-win_amd64.whl", hash = "sha256:6d7b6cccb96b36356d3e61c4e89972b82123d799cc2ca50f743e30ce45d70739"}, - {file = "tensorstore-0.1.45.tar.gz", hash = "sha256:38468c621b2edf09cfdd2df4905890e83f1805c7645ec13e16df5eafabf0e5e5"}, + {file = "tensorstore-0.1.46-cp310-cp310-macosx_10_14_x86_64.whl", hash = "sha256:5a5701fc2da014cb92977d855e34f8401dac15008d760589f115d8c2c775b05f"}, + {file = "tensorstore-0.1.46-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:ec24332e97086b1f3a4654f4e295cd3e53541b68cf65027217a7bd3219d5d484"}, + {file = "tensorstore-0.1.46-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:d10ca11417123ee5e606304dbd5e3001bb2a775abde58a247b96a455442f2e21"}, + {file = "tensorstore-0.1.46-cp310-cp310-win_amd64.whl", hash = "sha256:f754cff97e934a87ecefc6134c1a118576cd6b2ae5e9fdef4017f546f93cd49c"}, + {file = "tensorstore-0.1.46-cp311-cp311-macosx_10_14_x86_64.whl", hash = "sha256:490f867d543f1a640c4df2e55cd3100d720816fec809465ba6e4a94458f0f1a0"}, + {file = "tensorstore-0.1.46-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:f95cb13f7cfc5e6ef8a3c8d33be8a6ec7ddc9eee72303fc864a1307ed99c0194"}, + {file = "tensorstore-0.1.46-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:66450fc8ca2fd4a19edb2fda9aa932055ee90b14ac6617724d0ddd72a17d8d4d"}, + {file = "tensorstore-0.1.46-cp311-cp311-win_amd64.whl", hash = "sha256:278f4991ee852175dfea6a1d48385abb379eb8e02fa6f0a67530c85c8ea70b35"}, + {file = "tensorstore-0.1.46-cp39-cp39-macosx_10_14_x86_64.whl", hash = "sha256:f261e3298bba3be6b0b1ad058d5a5324ed4ebfcff1ca730a478ac11a2ef00c82"}, + {file = "tensorstore-0.1.46-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:462769f0b105773e23479a88f57ad0d3d4df5b788452ed8ad68696f14daa2a95"}, + {file = "tensorstore-0.1.46-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:8a25736710ed521a866484a7c17073def9984fb111748c3fdc3ba83723183443"}, + {file = "tensorstore-0.1.46-cp39-cp39-win_amd64.whl", hash = "sha256:5f9a1b7eb79068719b269ca4ec5368332cfe91dfdf02847228da701de45e4c0a"}, + {file = "tensorstore-0.1.46.tar.gz", hash = "sha256:b5074a67919ebf89da1e794f88f87dc0d19f98f4ce7b484a083f7422c83cf9de"}, ] [package.dependencies] +ml-dtypes = ">=0.3.1" numpy = ">=1.16.0" [[package]] @@ -4098,13 +4111,13 @@ test = ["argcomplete (>=3.0.3)", "mypy (>=1.5.1)", "pre-commit", "pytest (>=7.0, [[package]] name = "transformers" -version = "4.34.0" +version = "4.34.1" description = "State-of-the-art Machine Learning for JAX, PyTorch and TensorFlow" optional = false python-versions = ">=3.8.0" files = [ - {file = "transformers-4.34.0-py3-none-any.whl", hash = "sha256:3f0187183a7f22c51ecbbc9eac5145df666c5b86bec6feed10e11f0363f3a1f9"}, - {file = "transformers-4.34.0.tar.gz", hash = "sha256:cc2ae61bfbfaa45337fd9017326669fc60e4f55125f589d50da47819e3d6f504"}, + {file = "transformers-4.34.1-py3-none-any.whl", hash = "sha256:d06ac09151d7b845e4a4acd6b143a591d946031ee67b4cbb20693b241920ffc0"}, + {file = "transformers-4.34.1.tar.gz", hash = "sha256:1d0258d5a18063b66005bbe1e3276ec5943d9ab4ab47f020db1fd485cc40ea22"}, ] [package.dependencies] @@ -4199,13 +4212,13 @@ files = [ [[package]] name = "urllib3" -version = "2.0.6" +version = "2.0.7" description = "HTTP library with thread-safe connection pooling, file post, and more." optional = false python-versions = ">=3.7" files = [ - {file = "urllib3-2.0.6-py3-none-any.whl", hash = "sha256:7a7c7003b000adf9e7ca2a377c9688bbc54ed41b985789ed576570342a375cd2"}, - {file = "urllib3-2.0.6.tar.gz", hash = "sha256:b19e1a85d206b56d7df1d5e683df4a7725252a964e3993648dd0fb5a1c157564"}, + {file = "urllib3-2.0.7-py3-none-any.whl", hash = "sha256:fdb6d215c776278489906c2f8916e6e7d4f5a9b602ccbcfdf7f016fc8da0596e"}, + {file = "urllib3-2.0.7.tar.gz", hash = "sha256:c97dfde1f7bd43a71c8d2a58e369e9b2bf692d1334ea9f9cae55add7d0dd0f84"}, ] [package.extras] @@ -4252,77 +4265,135 @@ standard = ["colorama (>=0.4)", "httptools (>=0.5.0)", "python-dotenv (>=0.13)", [[package]] name = "uvloop" -version = "0.17.0" +version = "0.18.0" description = "Fast implementation of asyncio event loop on top of libuv" optional = false -python-versions = ">=3.7" +python-versions = ">=3.7.0" files = [ - {file = "uvloop-0.17.0-cp310-cp310-macosx_10_9_universal2.whl", hash = "sha256:ce9f61938d7155f79d3cb2ffa663147d4a76d16e08f65e2c66b77bd41b356718"}, - {file = "uvloop-0.17.0-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:68532f4349fd3900b839f588972b3392ee56042e440dd5873dfbbcd2cc67617c"}, - {file = "uvloop-0.17.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:0949caf774b9fcefc7c5756bacbbbd3fc4c05a6b7eebc7c7ad6f825b23998d6d"}, - {file = "uvloop-0.17.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:ff3d00b70ce95adce264462c930fbaecb29718ba6563db354608f37e49e09024"}, - {file = "uvloop-0.17.0-cp310-cp310-musllinux_1_1_aarch64.whl", hash = "sha256:a5abddb3558d3f0a78949c750644a67be31e47936042d4f6c888dd6f3c95f4aa"}, - {file = "uvloop-0.17.0-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:8efcadc5a0003d3a6e887ccc1fb44dec25594f117a94e3127954c05cf144d811"}, - {file = "uvloop-0.17.0-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:3378eb62c63bf336ae2070599e49089005771cc651c8769aaad72d1bd9385a7c"}, - {file = "uvloop-0.17.0-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:6aafa5a78b9e62493539456f8b646f85abc7093dd997f4976bb105537cf2635e"}, - {file = "uvloop-0.17.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:c686a47d57ca910a2572fddfe9912819880b8765e2f01dc0dd12a9bf8573e539"}, - {file = "uvloop-0.17.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:864e1197139d651a76c81757db5eb199db8866e13acb0dfe96e6fc5d1cf45fc4"}, - {file = "uvloop-0.17.0-cp311-cp311-musllinux_1_1_aarch64.whl", hash = "sha256:2a6149e1defac0faf505406259561bc14b034cdf1d4711a3ddcdfbaa8d825a05"}, - {file = "uvloop-0.17.0-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:6708f30db9117f115eadc4f125c2a10c1a50d711461699a0cbfaa45b9a78e376"}, - {file = "uvloop-0.17.0-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:23609ca361a7fc587031429fa25ad2ed7242941adec948f9d10c045bfecab06b"}, - {file = "uvloop-0.17.0-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:2deae0b0fb00a6af41fe60a675cec079615b01d68beb4cc7b722424406b126a8"}, - {file = "uvloop-0.17.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:45cea33b208971e87a31c17622e4b440cac231766ec11e5d22c76fab3bf9df62"}, - {file = "uvloop-0.17.0-cp37-cp37m-musllinux_1_1_aarch64.whl", hash = "sha256:9b09e0f0ac29eee0451d71798878eae5a4e6a91aa275e114037b27f7db72702d"}, - {file = "uvloop-0.17.0-cp37-cp37m-musllinux_1_1_x86_64.whl", hash = "sha256:dbbaf9da2ee98ee2531e0c780455f2841e4675ff580ecf93fe5c48fe733b5667"}, - {file = "uvloop-0.17.0-cp38-cp38-macosx_10_9_universal2.whl", hash = "sha256:a4aee22ece20958888eedbad20e4dbb03c37533e010fb824161b4f05e641f738"}, - {file = "uvloop-0.17.0-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:307958f9fc5c8bb01fad752d1345168c0abc5d62c1b72a4a8c6c06f042b45b20"}, - {file = "uvloop-0.17.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:3ebeeec6a6641d0adb2ea71dcfb76017602ee2bfd8213e3fcc18d8f699c5104f"}, - {file = "uvloop-0.17.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:1436c8673c1563422213ac6907789ecb2b070f5939b9cbff9ef7113f2b531595"}, - {file = "uvloop-0.17.0-cp38-cp38-musllinux_1_1_aarch64.whl", hash = "sha256:8887d675a64cfc59f4ecd34382e5b4f0ef4ae1da37ed665adba0c2badf0d6578"}, - {file = "uvloop-0.17.0-cp38-cp38-musllinux_1_1_x86_64.whl", hash = "sha256:3db8de10ed684995a7f34a001f15b374c230f7655ae840964d51496e2f8a8474"}, - {file = "uvloop-0.17.0-cp39-cp39-macosx_10_9_universal2.whl", hash = "sha256:7d37dccc7ae63e61f7b96ee2e19c40f153ba6ce730d8ba4d3b4e9738c1dccc1b"}, - {file = "uvloop-0.17.0-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:cbbe908fda687e39afd6ea2a2f14c2c3e43f2ca88e3a11964b297822358d0e6c"}, - {file = "uvloop-0.17.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:3d97672dc709fa4447ab83276f344a165075fd9f366a97b712bdd3fee05efae8"}, - {file = "uvloop-0.17.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:f1e507c9ee39c61bfddd79714e4f85900656db1aec4d40c6de55648e85c2799c"}, - {file = "uvloop-0.17.0-cp39-cp39-musllinux_1_1_aarch64.whl", hash = "sha256:c092a2c1e736086d59ac8e41f9c98f26bbf9b9222a76f21af9dfe949b99b2eb9"}, - {file = "uvloop-0.17.0-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:30babd84706115626ea78ea5dbc7dd8d0d01a2e9f9b306d24ca4ed5796c66ded"}, - {file = "uvloop-0.17.0.tar.gz", hash = "sha256:0ddf6baf9cf11a1a22c71487f39f15b2cf78eb5bde7e5b45fbb99e8a9d91b9e1"}, + {file = "uvloop-0.18.0-cp310-cp310-macosx_10_9_universal2.whl", hash = "sha256:1f354d669586fca96a9a688c585b6257706d216177ac457c92e15709acaece10"}, + {file = "uvloop-0.18.0-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:280904236a5b333a273292b3bcdcbfe173690f69901365b973fa35be302d7781"}, + {file = "uvloop-0.18.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:ad79cd30c7e7484bdf6e315f3296f564b3ee2f453134a23ffc80d00e63b3b59e"}, + {file = "uvloop-0.18.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:99deae0504547d04990cc5acf631d9f490108c3709479d90c1dcd14d6e7af24d"}, + {file = "uvloop-0.18.0-cp310-cp310-musllinux_1_1_aarch64.whl", hash = "sha256:edbb4de38535f42f020da1e3ae7c60f2f65402d027a08a8c60dc8569464873a6"}, + {file = "uvloop-0.18.0-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:54b211c46facb466726b227f350792770fc96593c4ecdfaafe20dc00f3209aef"}, + {file = "uvloop-0.18.0-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:25b714f07c68dcdaad6994414f6ec0f2a3b9565524fba181dcbfd7d9598a3e73"}, + {file = "uvloop-0.18.0-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:1121087dfeb46e9e65920b20d1f46322ba299b8d93f7cb61d76c94b5a1adc20c"}, + {file = "uvloop-0.18.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:74020ef8061678e01a40c49f1716b4f4d1cc71190d40633f08a5ef8a7448a5c6"}, + {file = "uvloop-0.18.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:1f4a549cd747e6f4f8446f4b4c8cb79504a8372d5d3a9b4fc20e25daf8e76c05"}, + {file = "uvloop-0.18.0-cp311-cp311-musllinux_1_1_aarch64.whl", hash = "sha256:6132318e1ab84a626639b252137aa8d031a6c0550250460644c32ed997604088"}, + {file = "uvloop-0.18.0-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:585b7281f9ea25c4a5fa993b1acca4ad3d8bc3f3fe2e393f0ef51b6c1bcd2fe6"}, + {file = "uvloop-0.18.0-cp312-cp312-macosx_10_9_universal2.whl", hash = "sha256:61151cc207cf5fc88863e50de3d04f64ee0fdbb979d0b97caf21cae29130ed78"}, + {file = "uvloop-0.18.0-cp312-cp312-macosx_10_9_x86_64.whl", hash = "sha256:c65585ae03571b73907b8089473419d8c0aff1e3826b3bce153776de56cbc687"}, + {file = "uvloop-0.18.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:e3d301e23984dcbc92d0e42253e0e0571915f0763f1eeaf68631348745f2dccc"}, + {file = "uvloop-0.18.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:680da98f12a7587f76f6f639a8aa7708936a5d17c5e7db0bf9c9d9cbcb616593"}, + {file = "uvloop-0.18.0-cp312-cp312-musllinux_1_1_aarch64.whl", hash = "sha256:75baba0bfdd385c886804970ae03f0172e0d51e51ebd191e4df09b929771b71e"}, + {file = "uvloop-0.18.0-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:ed3c28337d2fefc0bac5705b9c66b2702dc392f2e9a69badb1d606e7e7f773bb"}, + {file = "uvloop-0.18.0-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:8849b8ef861431543c07112ad8436903e243cdfa783290cbee3df4ce86d8dd48"}, + {file = "uvloop-0.18.0-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:211ce38d84118ae282a91408f61b85cf28e2e65a0a8966b9a97e0e9d67c48722"}, + {file = "uvloop-0.18.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:b0a8f706b943c198dcedf1f2fb84899002c195c24745e47eeb8f2fb340f7dfc3"}, + {file = "uvloop-0.18.0-cp37-cp37m-musllinux_1_1_aarch64.whl", hash = "sha256:58e44650cbc8607a218caeece5a689f0a2d10be084a69fc32f7db2e8f364927c"}, + {file = "uvloop-0.18.0-cp37-cp37m-musllinux_1_1_x86_64.whl", hash = "sha256:2b8b7cf7806bdc745917f84d833f2144fabcc38e9cd854e6bc49755e3af2b53e"}, + {file = "uvloop-0.18.0-cp38-cp38-macosx_10_9_universal2.whl", hash = "sha256:56c1026a6b0d12b378425e16250acb7d453abaefe7a2f5977143898db6cfe5bd"}, + {file = "uvloop-0.18.0-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:12af0d2e1b16780051d27c12de7e419b9daeb3516c503ab3e98d364cc55303bb"}, + {file = "uvloop-0.18.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:b028776faf9b7a6d0a325664f899e4c670b2ae430265189eb8d76bd4a57d8a6e"}, + {file = "uvloop-0.18.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:53aca21735eee3859e8c11265445925911ffe410974f13304edb0447f9f58420"}, + {file = "uvloop-0.18.0-cp38-cp38-musllinux_1_1_aarch64.whl", hash = "sha256:847f2ed0887047c63da9ad788d54755579fa23f0784db7e752c7cf14cf2e7506"}, + {file = "uvloop-0.18.0-cp38-cp38-musllinux_1_1_x86_64.whl", hash = "sha256:6e20bb765fcac07879cd6767b6dca58127ba5a456149717e0e3b1f00d8eab51c"}, + {file = "uvloop-0.18.0-cp39-cp39-macosx_10_9_universal2.whl", hash = "sha256:e14de8800765b9916d051707f62e18a304cde661fa2b98a58816ca38d2b94029"}, + {file = "uvloop-0.18.0-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:f3b18663efe0012bc4c315f1b64020e44596f5fabc281f5b0d9bc9465288559c"}, + {file = "uvloop-0.18.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:c6d341bc109fb8ea69025b3ec281fcb155d6824a8ebf5486c989ff7748351a37"}, + {file = "uvloop-0.18.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:895a1e3aca2504638a802d0bec2759acc2f43a0291a1dff886d69f8b7baff399"}, + {file = "uvloop-0.18.0-cp39-cp39-musllinux_1_1_aarch64.whl", hash = "sha256:4d90858f32a852988d33987d608bcfba92a1874eb9f183995def59a34229f30d"}, + {file = "uvloop-0.18.0-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:db1fcbad5deb9551e011ca589c5e7258b5afa78598174ac37a5f15ddcfb4ac7b"}, + {file = "uvloop-0.18.0.tar.gz", hash = "sha256:d5d1135beffe9cd95d0350f19e2716bc38be47d5df296d7cc46e3b7557c0d1ff"}, ] [package.extras] -dev = ["Cython (>=0.29.32,<0.30.0)", "Sphinx (>=4.1.2,<4.2.0)", "aiohttp", "flake8 (>=3.9.2,<3.10.0)", "mypy (>=0.800)", "psutil", "pyOpenSSL (>=22.0.0,<22.1.0)", "pycodestyle (>=2.7.0,<2.8.0)", "pytest (>=3.6.0)", "sphinx-rtd-theme (>=0.5.2,<0.6.0)", "sphinxcontrib-asyncio (>=0.3.0,<0.4.0)"] docs = ["Sphinx (>=4.1.2,<4.2.0)", "sphinx-rtd-theme (>=0.5.2,<0.6.0)", "sphinxcontrib-asyncio (>=0.3.0,<0.4.0)"] -test = ["Cython (>=0.29.32,<0.30.0)", "aiohttp", "flake8 (>=3.9.2,<3.10.0)", "mypy (>=0.800)", "psutil", "pyOpenSSL (>=22.0.0,<22.1.0)", "pycodestyle (>=2.7.0,<2.8.0)"] +test = ["Cython (>=0.29.36,<0.30.0)", "aiohttp (==3.9.0b0)", "aiohttp (>=3.8.1)", "flake8 (>=5.0,<6.0)", "mypy (>=0.800)", "psutil", "pyOpenSSL (>=23.0.0,<23.1.0)", "pycodestyle (>=2.9.0,<2.10.0)"] [[package]] name = "watchfiles" -version = "0.20.0" +version = "0.21.0" description = "Simple, modern and high performance file watching and code reload in python." optional = false -python-versions = ">=3.7" +python-versions = ">=3.8" files = [ - {file = "watchfiles-0.20.0-cp37-abi3-macosx_10_7_x86_64.whl", hash = "sha256:3796312bd3587e14926013612b23066912cf45a14af71cf2b20db1c12dadf4e9"}, - {file = "watchfiles-0.20.0-cp37-abi3-macosx_11_0_arm64.whl", hash = "sha256:d0002d81c89a662b595645fb684a371b98ff90a9c7d8f8630c82f0fde8310458"}, - {file = "watchfiles-0.20.0-cp37-abi3-manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:570848706440373b4cd8017f3e850ae17f76dbdf1e9045fc79023b11e1afe490"}, - {file = "watchfiles-0.20.0-cp37-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:9a0351d20d03c6f7ad6b2e8a226a5efafb924c7755ee1e34f04c77c3682417fa"}, - {file = "watchfiles-0.20.0-cp37-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:007dcc4a401093010b389c044e81172c8a2520dba257c88f8828b3d460c6bb38"}, - {file = "watchfiles-0.20.0-cp37-abi3-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:0d82dbc1832da83e441d112069833eedd4cf583d983fb8dd666fbefbea9d99c0"}, - {file = "watchfiles-0.20.0-cp37-abi3-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:99f4c65fd2fce61a571b2a6fcf747d6868db0bef8a934e8ca235cc8533944d95"}, - {file = "watchfiles-0.20.0-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:5392dd327a05f538c56edb1c6ebba6af91afc81b40822452342f6da54907bbdf"}, - {file = "watchfiles-0.20.0-cp37-abi3-musllinux_1_1_aarch64.whl", hash = "sha256:08dc702529bb06a2b23859110c214db245455532da5eaea602921687cfcd23db"}, - {file = "watchfiles-0.20.0-cp37-abi3-musllinux_1_1_x86_64.whl", hash = "sha256:7d4e66a857621584869cfbad87039e65dadd7119f0d9bb9dbc957e089e32c164"}, - {file = "watchfiles-0.20.0-cp37-abi3-win32.whl", hash = "sha256:a03d1e6feb7966b417f43c3e3783188167fd69c2063e86bad31e62c4ea794cc5"}, - {file = "watchfiles-0.20.0-cp37-abi3-win_amd64.whl", hash = "sha256:eccc8942bcdc7d638a01435d915b913255bbd66f018f1af051cd8afddb339ea3"}, - {file = "watchfiles-0.20.0-cp37-abi3-win_arm64.whl", hash = "sha256:b17d4176c49d207865630da5b59a91779468dd3e08692fe943064da260de2c7c"}, - {file = "watchfiles-0.20.0-pp38-pypy38_pp73-macosx_10_7_x86_64.whl", hash = "sha256:d97db179f7566dcf145c5179ddb2ae2a4450e3a634eb864b09ea04e68c252e8e"}, - {file = "watchfiles-0.20.0-pp38-pypy38_pp73-macosx_11_0_arm64.whl", hash = "sha256:835df2da7a5df5464c4a23b2d963e1a9d35afa422c83bf4ff4380b3114603644"}, - {file = "watchfiles-0.20.0-pp38-pypy38_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:608cd94a8767f49521901aff9ae0c92cc8f5a24d528db7d6b0295290f9d41193"}, - {file = "watchfiles-0.20.0-pp38-pypy38_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:89d1de8218874925bce7bb2ae9657efc504411528930d7a83f98b1749864f2ef"}, - {file = "watchfiles-0.20.0-pp39-pypy39_pp73-macosx_10_7_x86_64.whl", hash = "sha256:13f995d5152a8ba4ed7c2bbbaeee4e11a5944defc7cacd0ccb4dcbdcfd78029a"}, - {file = "watchfiles-0.20.0-pp39-pypy39_pp73-macosx_11_0_arm64.whl", hash = "sha256:9b5c8d3be7b502f8c43a33c63166ada8828dbb0c6d49c8f9ce990a96de2f5a49"}, - {file = "watchfiles-0.20.0-pp39-pypy39_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:e43af4464daa08723c04b43cf978ab86cc55c684c16172622bdac64b34e36af0"}, - {file = "watchfiles-0.20.0-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:87d9e1f75c4f86c93d73b5bd1ebe667558357548f11b4f8af4e0e272f79413ce"}, - {file = "watchfiles-0.20.0.tar.gz", hash = "sha256:728575b6b94c90dd531514677201e8851708e6e4b5fe7028ac506a200b622019"}, + {file = "watchfiles-0.21.0-cp310-cp310-macosx_10_7_x86_64.whl", hash = "sha256:27b4035013f1ea49c6c0b42d983133b136637a527e48c132d368eb19bf1ac6aa"}, + {file = "watchfiles-0.21.0-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:c81818595eff6e92535ff32825f31c116f867f64ff8cdf6562cd1d6b2e1e8f3e"}, + {file = "watchfiles-0.21.0-cp310-cp310-manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:6c107ea3cf2bd07199d66f156e3ea756d1b84dfd43b542b2d870b77868c98c03"}, + {file = "watchfiles-0.21.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:0d9ac347653ebd95839a7c607608703b20bc07e577e870d824fa4801bc1cb124"}, + {file = "watchfiles-0.21.0-cp310-cp310-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:5eb86c6acb498208e7663ca22dbe68ca2cf42ab5bf1c776670a50919a56e64ab"}, + {file = "watchfiles-0.21.0-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:f564bf68404144ea6b87a78a3f910cc8de216c6b12a4cf0b27718bf4ec38d303"}, + {file = "watchfiles-0.21.0-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:3d0f32ebfaa9c6011f8454994f86108c2eb9c79b8b7de00b36d558cadcedaa3d"}, + {file = "watchfiles-0.21.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:b6d45d9b699ecbac6c7bd8e0a2609767491540403610962968d258fd6405c17c"}, + {file = "watchfiles-0.21.0-cp310-cp310-musllinux_1_1_aarch64.whl", hash = "sha256:aff06b2cac3ef4616e26ba17a9c250c1fe9dd8a5d907d0193f84c499b1b6e6a9"}, + {file = "watchfiles-0.21.0-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:d9792dff410f266051025ecfaa927078b94cc7478954b06796a9756ccc7e14a9"}, + {file = "watchfiles-0.21.0-cp310-none-win32.whl", hash = "sha256:214cee7f9e09150d4fb42e24919a1e74d8c9b8a9306ed1474ecaddcd5479c293"}, + {file = "watchfiles-0.21.0-cp310-none-win_amd64.whl", hash = "sha256:1ad7247d79f9f55bb25ab1778fd47f32d70cf36053941f07de0b7c4e96b5d235"}, + {file = "watchfiles-0.21.0-cp311-cp311-macosx_10_7_x86_64.whl", hash = "sha256:668c265d90de8ae914f860d3eeb164534ba2e836811f91fecc7050416ee70aa7"}, + {file = "watchfiles-0.21.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:3a23092a992e61c3a6a70f350a56db7197242f3490da9c87b500f389b2d01eef"}, + {file = "watchfiles-0.21.0-cp311-cp311-manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:e7941bbcfdded9c26b0bf720cb7e6fd803d95a55d2c14b4bd1f6a2772230c586"}, + {file = "watchfiles-0.21.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:11cd0c3100e2233e9c53106265da31d574355c288e15259c0d40a4405cbae317"}, + {file = "watchfiles-0.21.0-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:d78f30cbe8b2ce770160d3c08cff01b2ae9306fe66ce899b73f0409dc1846c1b"}, + {file = "watchfiles-0.21.0-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:6674b00b9756b0af620aa2a3346b01f8e2a3dc729d25617e1b89cf6af4a54eb1"}, + {file = "watchfiles-0.21.0-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:fd7ac678b92b29ba630d8c842d8ad6c555abda1b9ef044d6cc092dacbfc9719d"}, + {file = "watchfiles-0.21.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:9c873345680c1b87f1e09e0eaf8cf6c891b9851d8b4d3645e7efe2ec20a20cc7"}, + {file = "watchfiles-0.21.0-cp311-cp311-musllinux_1_1_aarch64.whl", hash = "sha256:49f56e6ecc2503e7dbe233fa328b2be1a7797d31548e7a193237dcdf1ad0eee0"}, + {file = "watchfiles-0.21.0-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:02d91cbac553a3ad141db016e3350b03184deaafeba09b9d6439826ee594b365"}, + {file = "watchfiles-0.21.0-cp311-none-win32.whl", hash = "sha256:ebe684d7d26239e23d102a2bad2a358dedf18e462e8808778703427d1f584400"}, + {file = "watchfiles-0.21.0-cp311-none-win_amd64.whl", hash = "sha256:4566006aa44cb0d21b8ab53baf4b9c667a0ed23efe4aaad8c227bfba0bf15cbe"}, + {file = "watchfiles-0.21.0-cp311-none-win_arm64.whl", hash = "sha256:c550a56bf209a3d987d5a975cdf2063b3389a5d16caf29db4bdddeae49f22078"}, + {file = "watchfiles-0.21.0-cp312-cp312-macosx_10_7_x86_64.whl", hash = "sha256:51ddac60b96a42c15d24fbdc7a4bfcd02b5a29c047b7f8bf63d3f6f5a860949a"}, + {file = "watchfiles-0.21.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:511f0b034120cd1989932bf1e9081aa9fb00f1f949fbd2d9cab6264916ae89b1"}, + {file = "watchfiles-0.21.0-cp312-cp312-manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:cfb92d49dbb95ec7a07511bc9efb0faff8fe24ef3805662b8d6808ba8409a71a"}, + {file = "watchfiles-0.21.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:3f92944efc564867bbf841c823c8b71bb0be75e06b8ce45c084b46411475a915"}, + {file = "watchfiles-0.21.0-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:642d66b75eda909fd1112d35c53816d59789a4b38c141a96d62f50a3ef9b3360"}, + {file = "watchfiles-0.21.0-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:d23bcd6c8eaa6324fe109d8cac01b41fe9a54b8c498af9ce464c1aeeb99903d6"}, + {file = "watchfiles-0.21.0-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:18d5b4da8cf3e41895b34e8c37d13c9ed294954907929aacd95153508d5d89d7"}, + {file = "watchfiles-0.21.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:1b8d1eae0f65441963d805f766c7e9cd092f91e0c600c820c764a4ff71a0764c"}, + {file = "watchfiles-0.21.0-cp312-cp312-musllinux_1_1_aarch64.whl", hash = "sha256:1fd9a5205139f3c6bb60d11f6072e0552f0a20b712c85f43d42342d162be1235"}, + {file = "watchfiles-0.21.0-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:a1e3014a625bcf107fbf38eece0e47fa0190e52e45dc6eee5a8265ddc6dc5ea7"}, + {file = "watchfiles-0.21.0-cp312-none-win32.whl", hash = "sha256:9d09869f2c5a6f2d9df50ce3064b3391d3ecb6dced708ad64467b9e4f2c9bef3"}, + {file = "watchfiles-0.21.0-cp312-none-win_amd64.whl", hash = "sha256:18722b50783b5e30a18a8a5db3006bab146d2b705c92eb9a94f78c72beb94094"}, + {file = "watchfiles-0.21.0-cp312-none-win_arm64.whl", hash = "sha256:a3b9bec9579a15fb3ca2d9878deae789df72f2b0fdaf90ad49ee389cad5edab6"}, + {file = "watchfiles-0.21.0-cp38-cp38-macosx_10_7_x86_64.whl", hash = "sha256:4ea10a29aa5de67de02256a28d1bf53d21322295cb00bd2d57fcd19b850ebd99"}, + {file = "watchfiles-0.21.0-cp38-cp38-macosx_11_0_arm64.whl", hash = "sha256:40bca549fdc929b470dd1dbfcb47b3295cb46a6d2c90e50588b0a1b3bd98f429"}, + {file = "watchfiles-0.21.0-cp38-cp38-manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:9b37a7ba223b2f26122c148bb8d09a9ff312afca998c48c725ff5a0a632145f7"}, + {file = "watchfiles-0.21.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:ec8c8900dc5c83650a63dd48c4d1d245343f904c4b64b48798c67a3767d7e165"}, + {file = "watchfiles-0.21.0-cp38-cp38-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:8ad3fe0a3567c2f0f629d800409cd528cb6251da12e81a1f765e5c5345fd0137"}, + {file = "watchfiles-0.21.0-cp38-cp38-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:9d353c4cfda586db2a176ce42c88f2fc31ec25e50212650c89fdd0f560ee507b"}, + {file = "watchfiles-0.21.0-cp38-cp38-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:83a696da8922314ff2aec02987eefb03784f473281d740bf9170181829133765"}, + {file = "watchfiles-0.21.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:5a03651352fc20975ee2a707cd2d74a386cd303cc688f407296064ad1e6d1562"}, + {file = "watchfiles-0.21.0-cp38-cp38-musllinux_1_1_aarch64.whl", hash = "sha256:3ad692bc7792be8c32918c699638b660c0de078a6cbe464c46e1340dadb94c19"}, + {file = "watchfiles-0.21.0-cp38-cp38-musllinux_1_1_x86_64.whl", hash = "sha256:06247538e8253975bdb328e7683f8515ff5ff041f43be6c40bff62d989b7d0b0"}, + {file = "watchfiles-0.21.0-cp38-none-win32.whl", hash = "sha256:9a0aa47f94ea9a0b39dd30850b0adf2e1cd32a8b4f9c7aa443d852aacf9ca214"}, + {file = "watchfiles-0.21.0-cp38-none-win_amd64.whl", hash = "sha256:8d5f400326840934e3507701f9f7269247f7c026d1b6cfd49477d2be0933cfca"}, + {file = "watchfiles-0.21.0-cp39-cp39-macosx_10_7_x86_64.whl", hash = "sha256:7f762a1a85a12cc3484f77eee7be87b10f8c50b0b787bb02f4e357403cad0c0e"}, + {file = "watchfiles-0.21.0-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:6e9be3ef84e2bb9710f3f777accce25556f4a71e15d2b73223788d528fcc2052"}, + {file = "watchfiles-0.21.0-cp39-cp39-manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:4c48a10d17571d1275701e14a601e36959ffada3add8cdbc9e5061a6e3579a5d"}, + {file = "watchfiles-0.21.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:6c889025f59884423428c261f212e04d438de865beda0b1e1babab85ef4c0f01"}, + {file = "watchfiles-0.21.0-cp39-cp39-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:66fac0c238ab9a2e72d026b5fb91cb902c146202bbd29a9a1a44e8db7b710b6f"}, + {file = "watchfiles-0.21.0-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:b4a21f71885aa2744719459951819e7bf5a906a6448a6b2bbce8e9cc9f2c8128"}, + {file = "watchfiles-0.21.0-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:1c9198c989f47898b2c22201756f73249de3748e0fc9de44adaf54a8b259cc0c"}, + {file = "watchfiles-0.21.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:d8f57c4461cd24fda22493109c45b3980863c58a25b8bec885ca8bea6b8d4b28"}, + {file = "watchfiles-0.21.0-cp39-cp39-musllinux_1_1_aarch64.whl", hash = "sha256:853853cbf7bf9408b404754b92512ebe3e3a83587503d766d23e6bf83d092ee6"}, + {file = "watchfiles-0.21.0-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:d5b1dc0e708fad9f92c296ab2f948af403bf201db8fb2eb4c8179db143732e49"}, + {file = "watchfiles-0.21.0-cp39-none-win32.whl", hash = "sha256:59137c0c6826bd56c710d1d2bda81553b5e6b7c84d5a676747d80caf0409ad94"}, + {file = "watchfiles-0.21.0-cp39-none-win_amd64.whl", hash = "sha256:6cb8fdc044909e2078c248986f2fc76f911f72b51ea4a4fbbf472e01d14faa58"}, + {file = "watchfiles-0.21.0-pp310-pypy310_pp73-macosx_10_7_x86_64.whl", hash = "sha256:ab03a90b305d2588e8352168e8c5a1520b721d2d367f31e9332c4235b30b8994"}, + {file = "watchfiles-0.21.0-pp310-pypy310_pp73-macosx_11_0_arm64.whl", hash = "sha256:927c589500f9f41e370b0125c12ac9e7d3a2fd166b89e9ee2828b3dda20bfe6f"}, + {file = "watchfiles-0.21.0-pp310-pypy310_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:1bd467213195e76f838caf2c28cd65e58302d0254e636e7c0fca81efa4a2e62c"}, + {file = "watchfiles-0.21.0-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:02b73130687bc3f6bb79d8a170959042eb56eb3a42df3671c79b428cd73f17cc"}, + {file = "watchfiles-0.21.0-pp38-pypy38_pp73-macosx_10_7_x86_64.whl", hash = "sha256:08dca260e85ffae975448e344834d765983237ad6dc308231aa16e7933db763e"}, + {file = "watchfiles-0.21.0-pp38-pypy38_pp73-macosx_11_0_arm64.whl", hash = "sha256:3ccceb50c611c433145502735e0370877cced72a6c70fd2410238bcbc7fe51d8"}, + {file = "watchfiles-0.21.0-pp38-pypy38_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:57d430f5fb63fea141ab71ca9c064e80de3a20b427ca2febcbfcef70ff0ce895"}, + {file = "watchfiles-0.21.0-pp38-pypy38_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:0dd5fad9b9c0dd89904bbdea978ce89a2b692a7ee8a0ce19b940e538c88a809c"}, + {file = "watchfiles-0.21.0-pp39-pypy39_pp73-macosx_10_7_x86_64.whl", hash = "sha256:be6dd5d52b73018b21adc1c5d28ac0c68184a64769052dfeb0c5d9998e7f56a2"}, + {file = "watchfiles-0.21.0-pp39-pypy39_pp73-macosx_11_0_arm64.whl", hash = "sha256:b3cab0e06143768499384a8a5efb9c4dc53e19382952859e4802f294214f36ec"}, + {file = "watchfiles-0.21.0-pp39-pypy39_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:8c6ed10c2497e5fedadf61e465b3ca12a19f96004c15dcffe4bd442ebadc2d85"}, + {file = "watchfiles-0.21.0-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:43babacef21c519bc6631c5fce2a61eccdfc011b4bcb9047255e9620732c8097"}, + {file = "watchfiles-0.21.0.tar.gz", hash = "sha256:c76c635fabf542bb78524905718c39f736a98e5ab25b23ec6d4abede1a85a6a3"}, ] [package.dependencies] diff --git a/demos/palm/python/docs-agent/pyproject.toml b/demos/palm/python/docs-agent/pyproject.toml index 9f9c86d23..8d3f74600 100644 --- a/demos/palm/python/docs-agent/pyproject.toml +++ b/demos/palm/python/docs-agent/pyproject.toml @@ -1,6 +1,6 @@ [tool.poetry] name = "docs-agent" -version = "0.1.2" +version = "0.1.3" description = "" authors = ["Docs Agent contributors"] readme = "README.md" diff --git a/demos/palm/python/docs-agent/scripts/README.md b/demos/palm/python/docs-agent/scripts/README.md new file mode 100644 index 000000000..18481be25 --- /dev/null +++ b/demos/palm/python/docs-agent/scripts/README.md @@ -0,0 +1,76 @@ +# Processing source documents in Docs Agent + +This `README` file provides information on the steps involved in the following two Python scripts, +which are used to process source documents into plain text chunks and store them next to their +generated embeddings in the vector database: + +- [`markdown_to_plain_text.py`][markdown-to-plain-text] +- [`populate_vector_database.py`][populate-vector-database] + +## Steps in the markdown_to_plain_text.py script + +When processing Markdown files to plain text using the +[`markdown_to_plain_text.py`][markdown-to-plain-text] script, the following events take place: + +1. Read the configuration file ([`config.yaml`][config-yaml]) to identify input and output + directories. +1. Construct an array of input sources. +1. **For** each input source, do the following: + 1. Extract all input fields (path, URL, and excluded path). + 1. Call the `process_markdown_files_from_source()` method using these input fields. + 1. **For** each sub-path in the input directory and **for** each file in these directories: + 1. Check if the file extension is `.md` (that is, a Markdown file). + 1. Construct an output directory that preserves the path structure. + 1. Read the content of the Markdown file. + 1. Call the `process_page_and_section_titles()` method to reformat the page and section + titles. + 1. Process Front Matter in Markdown. + 1. Detect (or construct) the title of the page. + 1. Detect Markdown headings (#, ##, and ###). + 1. Convert Markdown headings into plain English (to preserve context when generating + embeddings). + 1. Call the `process_document_into_sections()` method to split the content into small + text chunks. + 1. Create a new empty array. + 1. Divide the content using Markdown headings (#, ##, and ###). + 1. Insert each chunk into the array and simplify the heading to # (title). + 1. Return the array. + 1. **For** each text chunk, do the following: + 1. Call the `markdown_to_text()` method to clean up Markdown and HTML syntax. + 1. Remove `` lines in Markdown. + 1. Convert Markdown to HTML (which makes the plan text extraction easy). + 1. Use `BeautifulSoup` to extract plain text from the HTML. + 1. Remove `[][]` in Markdown. + 1. Remove `{: }` in Markdown. + 1. Remove `{. }` in Markdown. + 1. Remove a single line `sh` in Markdown. + 1. Remove code text and blocks. + 1. Return the plain text. + 1. Construct the text chunk’s metadata (including URL) for the `file_index.json` file. + 1. Write the text chunk into a file in the output directory. + +## Steps in the populate_vector_database.py script + +When processing plain text chunks to embeddings using the +[`populate_vector_database.py`][populate-vector-database] script, the following events take place: + +1. Read the configuration file ([`config.yaml`][config-yaml]) to identify the plain text directory + and Chroma settings. +1. Set up the PaLM API environment. +1. Select the embeddings model. +1. Configure the embedding function (including the API call limit). +1. **For** each sub-path in the plain text directory and **for** each file in these directories: + 1. Check if the file extension is `.md` (that is, a Markdown file). + 1. Read the content of the Markdown file. + 1. Construct the URL of the text chunk’s source. + 1. Read the metadata associated with the text chunk file. + 1. Store the text chunk and metabase to the vector database, which also generates an embedding + for the text chunk at the time of insertion. + 1. Skip if the file size is larger than 10000 bytes (due to the API limit). + 1. Skip if the text chunk is already in the vector database and the checksum hasn’t changed. + + + +[markdown-to-plain-text]: markdown_to_plain_text.py +[populate-vector-database]: populate_vector_database.py +[config-yaml]: ../config.yaml diff --git a/demos/palm/python/docs-agent/scripts/markdown_to_plain_text.py b/demos/palm/python/docs-agent/scripts/markdown_to_plain_text.py index 8f1585dd7..7b4f3721f 100644 --- a/demos/palm/python/docs-agent/scripts/markdown_to_plain_text.py +++ b/demos/palm/python/docs-agent/scripts/markdown_to_plain_text.py @@ -37,7 +37,7 @@ #### Input variables for the script #### # # Note: The hardcoded values below are overwritten -# if the `input-values.yaml` file is found. +# if the config.yaml file is found. # # MY_INPUT_PATH: An array of directories that contain source Markdown files. # URL_PREFIX: An array of prefixes to be used to create URLs for source Markdown files. @@ -50,7 +50,7 @@ ] MY_OUTPUT_PATH = "data/plain_docs" -#### Read the `input-values-yaml` file #### +#### Read the config.yaml file #### # At a minimum, INPUT_YAML must configure the following values: # output_path: The target directory where processed plain text files will be stored. # input: @@ -97,16 +97,16 @@ def markdown_to_text(markdown_string): # Remove [][] in Markdown text = re.sub(r"\[(.*?)\]\[(.*?)\]", "\\1", text) - # Remove {: } in devsite Markdown + # Remove {: } in Markdown text = re.sub(r"\{:(.*?)\}", "", text) - # Remove {. } in g3doc Markdown + # Remove {. } in Markdown text = re.sub(r"\{.(.*?)\}", "", text) - # Remove a single line `sh` in g3doc Markdown + # Remove a single line `sh` in Markdown text = re.sub(r"(?m)^sh$", "", text) - # Remove a single line ````sh` in g3doc Markdown + # Remove a single line ````sh` in Markdown # text = re.sub(r'(?m)^```sh$', '', text) # Remove code snippets @@ -377,7 +377,7 @@ def save_file_index_json(src_file_index): full_file_metadata = {} file_index = [] exclude = [] - # Process `input-values.yaml` into input variables. + # Process config.yaml into input variables. if IS_CONFIG_FILE: # Reads all the input values defined in the configuration file config_values = config.returnConfigValue("input") diff --git a/demos/palm/python/docs-agent/scripts/populate_vector_database.py b/demos/palm/python/docs-agent/scripts/populate_vector_database.py index 51eeb0ce8..e6a0c5090 100644 --- a/demos/palm/python/docs-agent/scripts/populate_vector_database.py +++ b/demos/palm/python/docs-agent/scripts/populate_vector_database.py @@ -37,7 +37,7 @@ # (see `markdown_to_plain_text.py`) # # Do the following: -# 1. If you are not using a `input-values.yaml` file, +# 1. If you are not using the config.yaml file, # edit PLAIN_TEXT_DIR in this script (see below). # 2. Run: # $ python3 ./scripts/populate-vector-database.py @@ -48,7 +48,7 @@ BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) ### Select the input directory of plain text files, this will be overridden by -### `input-values.yaml` +### the config.yaml file. ### Set up the path to the local LLM ### LOCAL_VECTOR_DB_DIR = os.path.join(BASE_DIR, "vector_stores/chroma") COLLECTION_NAME = "docs_collection" diff --git a/demos/palm/python/docs-agent/third_party/g2docsmd-html/LICENSE b/demos/palm/python/docs-agent/third_party/g2docsmd-html/LICENSE new file mode 100644 index 000000000..7a4a3ea24 --- /dev/null +++ b/demos/palm/python/docs-agent/third_party/g2docsmd-html/LICENSE @@ -0,0 +1,202 @@ + + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. \ No newline at end of file diff --git a/demos/palm/python/docs-agent/third_party/g2docsmd-html/exportmd.gs b/demos/palm/python/docs-agent/third_party/g2docsmd-html/exportmd.gs new file mode 100644 index 000000000..aa8afd00f --- /dev/null +++ b/demos/palm/python/docs-agent/third_party/g2docsmd-html/exportmd.gs @@ -0,0 +1,1244 @@ +/* +Parsing from mangini/gdocs2md. +Modified by clearf to add files to the google directory structure. +Modified by lmmx to write Markdown, going back to HTML-incorporation. + +Usage: + NB: don't use on top-level doc (in root Drive folder) See comment in setupScript function. + Adding this script to your doc: + - Tools > Script Manager > New + - Select "Blank Project", then paste this code in and save. + Running the script: + - Tools > Script Manager + - Select "convertDocumentToMarkdown" function. + - Click Run button. + - Converted doc will be added to a "Markdown" folder in the source document's directories. + - Images will be added to a subfolder of the "Markdown" folder. +*/ + +function onInstall(e) { + onOpen(e); +} + +function onOpen() { + // Add a menu with some items, some separators, and a sub-menu. + setupScript(); +// In future: +// DocumentApp.getUi().createAddonMenu(); + DocumentApp.getUi().createMenu('Markdown') + .addItem('View as markdown', 'markdownPopup') + .addSubMenu(DocumentApp.getUi().createMenu('Export \u2192 markdown') + .addItem('Export to local file', 'convertSingleDoc') + .addItem('Export entire folder to local file', 'convertFolder') + .addItem('Customise markdown conversion', 'changeDefaults')) + .addSeparator() + .addSubMenu(DocumentApp.getUi().createMenu('Toggle comment visibility') + .addItem('Image source URLs', 'toggleImageSourceStatus') + .addItem('All comments', 'toggleCommentStatus')) + .addItem("Add comment", 'addCommentDummy') + .addToUi(); +} + +function changeDefaults() { + var ui = DocumentApp.getUi(); + var default_settings = '{ use your imagination... }'; + var greeting = ui.alert('This should be set up to display defaults from variables passed to getDocComments etc., e.g. something like:\n\nDefault settings are:' + + '\ncomments - not checking deleted comments.\nDocument - this document (alternatively specify a document ID).' + + '\n\nClick OK to edit these, or cancel.', + ui.ButtonSet.OK_CANCEL); + ui.alert("There's not really need for this yet, so this won't proceed, regardless of what you just pressed."); + return; + + // Future: + if (greeting == ui.Button.CANCEL) { + ui.alert("Alright, never mind!"); + return; + } + // otherwise user clicked OK + // user clicked OK, to proceed with editing these defaults. Ask case by case whether to edit + + var response = ui.prompt('What is x (default y)?', ui.ButtonSet.YES_NO_CANCEL); + + // Example code from docs at https://developers.google.com/apps-script/reference/base/button-set + // Process the user's response. + if (response.getSelectedButton() == ui.Button.YES) { + Logger.log('The user\'s name is %s.', response.getResponseText()); + } else if (response.getSelectedButton() == ui.Button.NO) { + Logger.log('The user didn\'t want to provide a name.'); + } else { + Logger.log('The user clicked the close button in the dialog\'s title bar.'); + } +} + +function setupScript() { + var script_properties = PropertiesService.getScriptProperties(); + script_properties.setProperty("user_email", Drive.About.get().user.emailAddress); + + // manual way to do the following: + // script_properties.setProperty("folder_id", "INSERT_FOLDER_ID_HERE"); + // script_properties.setProperty("document_id", "INSERT_FILE_ID_HERE"); + + var doc_id = DocumentApp.getActiveDocument().getId(); + script_properties.setProperty("document_id", doc_id); + var doc_parents = DriveApp.getFileById(doc_id).getParents(); + var folders = doc_parents; + while (folders.hasNext()) { + var folder = folders.next(); + var folder_id = folder.getId(); + } + script_properties.setProperty("folder_id", folder_id); + script_properties.setProperty("image_folder_prefix", ""); // add if modifying image location +} + +function addCommentDummy() { + // Dummy function to be switched during development for addComment + DocumentApp.getUi() + .alert('Cancelling comment entry', + "There's not currently a readable anchor for Google Docs - you need to write your own!" + + + "\n\nThe infrastructure for using such an anchoring schema is sketched out in" + + " the exportmd.gs script's addComment function, for an anchor defined in anchor_props" + + + "\n\nSee github.com/lmmx/devnotes/wiki/Custom-Google-Docs-comment-anchoring-schema", + DocumentApp.getUi().ButtonSet.OK + ); + return; +} + +function addComment() { + + var doc_id = PropertiesService.getScriptProperties().getProperty('document_id'); + var user_email = PropertiesService.getScriptProperties().getProperty('email'); +/* Drive.Comments.insert({content: "hello world", + context: { + type: 'text/html', + value: 'hinges' + } + }, document_id); */ + var revision_list = Drive.Revisions.list(doc_id).items; + var recent_revision_id = revision_list[revision_list.length - 1].id; + var anchor_props = { + revision_id: recent_revision_id, + starting_offset: '', + offset_length: '', + total_chars: '' + } + insertComment(doc_id, 'hinges', 'Hello world!', my_email, anchor_props); +} + +function insertComment(fileId, selected_text, content, user_email, anchor_props) { + + // NB Deal with handling missing args + + /* + anchor_props is an object with 4 properties: + - revision_id, + - starting_offset, + - offset_length, + - total_chars + */ + + var context = Drive.newCommentContext(); + context.value = selected_text; + context.type = 'text/html'; + var comment = Drive.newComment(); + comment.kind = 'drive#comment'; + var author = Drive.newUser(); + author.kind = 'drive#user'; + author.displayName = user_email; + author.isAuthenticatedUser = true; + comment.author = author; + comment.content = type; + comment.context = context; + comment.status = 'open'; + comment.anchor = "{'r':" + + anchor_props.revision_id + + ",'a':[{'txt':{'o':" + + anchor_props.starting_offset + + ",'l':" + + anchor_props.offset_length + + ",'ml':" + + anchor_props.total_chars + + "}}]}"; + comment.fileId = fileId; + Drive.Comments.insert(comment, fileId); +} + +function decodeScriptSwitches(optional_storage_name) { + var property_name = (typeof(optional_storage_name) == 'string') ? optional_storage_name : 'switch_settings'; + var script_properties = PropertiesService.getScriptProperties(); + return script_properties + .getProperty(property_name) + .replace(/{|}/g,'') // Get the statements out of brackets... + .replace(',', ';'); // ...swap the separator for a semi-colon... + // ...evaluate the stored object string as statements upon string return and voila, switches interpreted +} + + +function getDocComments(comment_list_settings) { + var possible_settings = ['images', 'include_deleted']; + + // switches are processed and set on a script-wide property called "comment_switches" + var property_name = 'comment_switches'; + switchHandler(comment_list_settings, possible_settings, property_name); + + var script_properties = PropertiesService.getScriptProperties(); + var comment_switches = decodeScriptSwitches(property_name); + eval(comment_switches); + + var document_id = script_properties.getProperty("document_id"); + var comments_list = Drive.Comments.list(document_id, + {includeDeleted: include_deleted, + maxResults: 100 }); // 0 to 100, default 20 + // See https://developers.google.com/drive/v2/reference/comments/list for all options + var comment_array = []; + var image_sources = []; + // To collect all comments' image URLs to match against inlineImage class elements LINK_URL attribute + + for (var i = 0; i < comments_list.items.length; i++) { + var comment = comments_list.items[i]; + var comment_text = comment.content; + var comment_status = comment.status; + /* + images is a generic parameter passed in as a switch to + return image URL-containing comments only. + + If the parameter is provided, it's no longer undefined. + */ + var img_url_regex = /(https?:\/\/.+?\.(png|gif|jpe?g))/; + var has_img_url = img_url_regex.test(comment_text); + + if (images && !has_img_url) continue; // no image URL, don't store comment + if (has_img_url) image_sources.push(RegExp.$1); + comment_array.push(comment); + } + script_properties.setProperty('image_source_URLs', image_sources) + return comment_array; +} + +function isValidAttrib(attribute) { // Sanity check function, called per element in array + + // Possible list of attributes to check against (leaving out unchanging ones like kind) + possible_attrs = [ + 'selfLink', + 'commentId', + 'createdDate', + 'modifiedDate', + 'author', + 'htmlContent', + 'content', + 'deleted', + 'status', + 'context', + 'anchor', + 'fileId', + 'fileTitle', + 'replies', + 'author' + ]; + + // Check if attribute(s) provided can be used to match/filter comments: + + if (typeof(attribute) == 'string' || typeof(attribute) == 'object') { + // Either a string/object (1-tuple) + + // Generated with Javascript, gist: https://gist.github.com/lmmx/451b301e1d78ed2c10b4 + + // Return false from the function if any of the attributes specified are not in the above list + + // If an object, the name is the key, otherwise it's just the string + if (attribute.constructor === Object) { + var att_keys = []; + for (var att_key in attribute) { + if (attribute.hasOwnProperty(att_key)) { + att_keys.push(att_key); + } + } + for (var n=0; n < att_keys.length; n++) { + var attribute_name = att_keys[n]; + var is_valid_attrib = (possible_attrs.indexOf(attribute_name) > -1); + + // The attribute needs to be one of the possible attributes listed above, match its given value(s), + // else returning false will throw an error from onAttribError when within getCommentAttributes + return is_valid_attrib; + } + } else if (typeof(attribute) == 'string') { + var attribute_name = attribute; + var is_valid_attrib = (possible_attrs.indexOf(attribute_name) > -1); + return is_valid_attrib; + // Otherwise is a valid (string) attribute + } else if (attribute.constructor === Array) { + return false; // Again, if within getCommentAttributes this will cause an error - shouldn't pass an array + } else { + // Wouldn't expect this to happen, so give a custom error message + Logger.log('Unknown type (assumed impossible) passed to isValidAttrib: ', attribute, attribute.constructor); + throw new TypeError('Unknown passed to isValidAttrib - this should be receiving 1-tuples only, see logs for details.'); + } + } else return false; // Neither string/object / array of strings &/or objects - not a valid attribute +} + +function getCommentAttributes(attributes, comment_list_settings) { + + // A filter function built on Comments.list, for a given list of attributes + // Objects' values are ignored here, only their property titles are used to filter comments. + + + /* + - attributes: array of attributes to filter/match on + - comment_list_settings: (optional) object with properties corresponding to switches in getDocComments + + This function outputs an array of the same length as the comment list, containing + values for all fields matched/filtered on. + */ + + + /* + * All possible comment attributes are listed at: + * https://developers.google.com/drive/v2/reference/comments#properties + */ + + // Firstly, describe the type in a message to be thrown in case of TypeError: + + var attrib_def_message = "'attributes' should be a string (the attribute to get for each comment), " + + "an object (a key-value pair for attribute and desired value), " + + "or an array of objects (each with key-value pairs)"; + + function onAttribError(message) { + Logger.log(message); + throw new TypeError(message); + } + + // If (optional) comment_list_settings isn't set, make a getDocComments call with switches left blank. + if (typeof(comment_list_settings) == 'undefined') var comment_list_settings = {}; + if (typeof(attributes) == 'undefined') onAttribError(attrib_def_message); // no variables specified + + if (isValidAttrib(attributes)) { // This will be true if there's only one attribute, not provided in an array + + /* + Make a 1-tuple (array of 1) from either an object or a string, + i.e. a single attribute, with or without a defined value respectively. + */ + + var attributes = Array(attributes); + + } else if (attributes.constructor === Array) { + + // Check each item in the array is a valid attribute specification + for (var l = 0; l < attributes.length; l++) { + if (! isValidAttrib(attributes[l]) ) { + onAttribError('Error in attribute ' + + (l+1) + ' of ' + attributes.length + + '\n\n' + + attrib_def_message); + } + } + + } else { // Neither attribute nor array of attributes + throw new TypeError(attrib_def_message); + } + + // Attributes now holds an array of string and/or objects specifying a comment match and/or filter query + + var comment_list = getDocComments(comment_list_settings); + var comment_attrib_lists = []; + for (var i in comment_list) { + var comment = comment_list[i]; + var comment_attrib_list = []; + for (var j in attributes) { + var comment_attribute = comment_list[i][attributes[j]]; + comment_attrib_list.push(comment_attribute); + } + comment_attrib_lists.push(comment_attrib_list); + } + // The array comment_attrib_lists is now full of the requested attributes, + // of length equal to that of attributes + return comment_attrib_lists; +} + +// Example function to use getCommentAttributes: + +function filterComments(attributes, comment_list_settings) { + var comment_attributes = getCommentAttributes(attributes, comment_list_settings); + var m = attribs.indexOf('commentId') // no need to keep track of commentID array position + comm_attribs.map(function(attrib_pair) { + if (attrib_pair[1]); + }) +} + +function toggleCommentStatus(comment_switches){ + // Technically just image URL-containing comments, not sources just yet + var attribs = ['commentId', 'status']; + var comm_attribs = getCommentAttributes(attribs, comment_switches); + var rearrangement = []; + comm_attribs.map( + function(attrib_pair) { // for every comment return with the images_only / images: true comments.list setting, + switch (attrib_pair[1]){ // check the status of each + case 'open': + rearrangement.push([attrib_pair[0],'resolved']); + break; + case 'resolved': + rearrangement.push([attrib_pair[0],'open']); + break; + } + } + ); + var script_properties = PropertiesService.getScriptProperties(); + var doc_id = script_properties.getProperty("document_id"); + rearrangement.map( + function(new_attrib_pair) { // for every comment ID with flipped status + Drive.Comments.patch('{"status": "' + + new_attrib_pair[1] + + '"}', doc_id, new_attrib_pair[0]) + } + ); + return; +} + +function toggleImageSourceStatus(){ + toggleCommentStatus({images: true}); +} + +function flipResolved() { + // Flip the status of resolved comments to open, and open comments to resolved (respectful = true) + // I.e. make resolved URL-containing comments visible, without losing track of normal comments' status + + // To force all comments' statuses to switch between resolved and open en masse set respectful to false + + var switch_settings = {}; + switch_settings.respectful = true; + switch_settings.images_only = false; // If true, only switch status of comments with an image URL + switch_settings.switch_deleted_comments = false; // If true, also switch status of deleted comments + + var comments_list = getDocComments( + { images: switch_settings.images_only, + include_deleted: switch_settings.switch_deleted_comments }); + + // Note: these parameters are unnecessary if both false (in their absence assumed false) + // but included for ease of later reuse + + if (switch_settings.respectful) { + // flip between + } else { + // flip all based on status of first in list + } +} + +function markdownPopup() { + var css_style = ''; + + // The above was written with js since doesn't work: + // https://gist.github.com/lmmx/ec084fc351528395f2bb + + var mdstring = stringMiddleMan(); + + var htmlstring = + '' + + css_style + + '
    '; + + var html5 = HtmlService.createHtmlOutput(htmlstring) + .setSandboxMode(HtmlService.SandboxMode.IFRAME) + .setWidth(800) + .setHeight(500); + + DocumentApp.getUi() + .showModalDialog(html5, 'Markdown output'); +} + +function stringMiddleMan() { + var returned_string; + convertSingleDoc({"return_string": true}); // for some reason needs the scope to be already set... + // could probably rework to use mdstring rather than returned_string, cut out middle man function + return this.returned_string; +} + +function convertSingleDoc(optional_switches) { + var script_properties = PropertiesService.getScriptProperties(); + // renew comments list on every export + var doc_comments = getDocComments(); + var image_urls = getDocComments({images: true}); // NB assumed false - any value will do + script_properties.setProperty("comments", doc_comments); + script_properties.setProperty("image_srcs", image_urls); + var folder_id = script_properties.getProperty("folder_id"); + var document_id = script_properties.getProperty("document_id"); + var source_folder = DriveApp.getFolderById(folder_id); + var markdown_folders = source_folder.getFoldersByName("Markdown"); + + var markdown_folder; + if (markdown_folders.hasNext()) { + markdown_folder = markdown_folders.next(); + } else { + // Create a Markdown folder if it doesn't exist. + markdown_folder = source_folder.createFolder("Markdown") + } + + convertDocumentToMarkdown(DocumentApp.openById(document_id), markdown_folder, optional_switches); +} + +function convertFolder() { + var script_properties = PropertiesService.getScriptProperties(); + var folder_id = script_properties.getProperty("folder_id"); + var source_folder = DriveApp.getFolderById(folder_id); + var markdown_folders = source_folder.getFoldersByName("Markdown"); + + + var markdown_folder; + if (markdown_folders.hasNext()) { + markdown_folder = markdown_folders.next(); + } else { + // Create a Markdown folder if it doesn't exist. + markdown_folder = source_folder.createFolder("Markdown"); + } + + // Only try to convert google docs files. + var gdoc_files = source_folder.getFilesByType("application/vnd.google-apps.document"); + + // For every file in this directory + while(gdoc_files.hasNext()) { + var gdoc_file = gdoc_files.next() + + var filename = gdoc_file.getName(); + var md_files = markdown_folder.getFilesByName(filename + ".md"); + var update_file = false; + + if (md_files.hasNext()) { + var md_file = md_files.next(); + + if (md_files.hasNext()){ // There are multiple markdown files; delete and rerun + update_file = true; + } else if (md_file.getLastUpdated() < gdoc_file.getLastUpdated()) { + update_file = true; + } + } else { + // There is no folder and the conversion needs to be rerun + update_file = true; + } + + if (update_file) { + convertDocumentToMarkdown(DocumentApp.openById(gdoc_file.getId()), markdown_folder); + } + } +} + +function switchHandler(input_switches, potential_switches, optional_storage_name) { + + // Firstly, if no input switches were set, make an empty input object + if (typeof(input_switches) == 'undefined') input_switches = {}; + + // Use optional storage name if it's defined (must be a string), else use default variable name "switch_settings" + var property_name = (typeof(optional_storage_name) == 'string') ? optional_storage_name : 'switch_settings'; + + // Make a blank object to be populated and stored as the script-wide property named after property_name + var switch_settings = {}; + + for (var i in potential_switches) { + var potential_switch = potential_switches[i]; + + // If each switch has been set (in input_switches), evaluate it, else assume it's switched off (false): + + if (input_switches.propertyIsEnumerable(potential_switch)) { + + // Evaluates a string representing a statement which sets switch_settings properties from input_switches + // e.g. "switch_settings.images = true" when input_switches = {images: true} + + eval('switch_settings.' + potential_switch + " = " + input_switches[potential_switch]); + + } else { + + // Alternatively, the evaluated statement sets anything absent from the input_switches object as false + // e.g. "switch_settings.images = false" when input_switches = {} and potential_switches = ['images'] + + eval('switch_settings.' + potential_switch + " = false"); + } + } + + PropertiesService.getScriptProperties().setProperty(property_name, switch_settings); + + /* + Looks bad but more sensible than repeatedly checking if arg undefined. + + Sets every variable named in the potential_switches array to false if + it wasn't passed into the input_switches object, otherwise evaluates. + + Any arguments not passed in are false, but so are any explicitly passed in as false: + all parameters are therefore Boolean until otherwise specified. + */ + +} + +function convertDocumentToMarkdown(document, destination_folder, optional_switches) { + // if returning a string, force_save_images will make the script continue - experimental + var possible_switches = ['return_string', 'force_save_images']; + var property_name = 'conversion_switches'; + switchHandler(optional_switches, possible_switches, property_name); + + // TODO switch off image storage if force_save_images is true - not necessary for normal behaviour + var script_properties = PropertiesService.getScriptProperties(); + var comment_switches = decodeScriptSwitches(property_name); + eval(comment_switches); + + var image_prefix = script_properties.getProperty("image_folder_prefix"); + var numChildren = document.getActiveSection().getNumChildren(); + var text = ""; + var md_filename = document.getName()+".md"; + var image_foldername = document.getName()+"_images"; + var inSrc = false; + var inClass = false; + var globalImageCounter = 0; + var globalListCounters = {}; + // edbacher: added a variable for indent in src
     block. Let style sheet do margin.
    +  var srcIndent = "";
    +
    +  var postHasImages = false;
    +
    +  var files = [];
    +
    +  // Walk through all the child elements of the doc.
    +  for (var i = 0; i < numChildren; i++) {
    +    var child = document.getActiveSection().getChild(i);
    +    var result = processParagraph(i, child, inSrc, globalImageCounter, globalListCounters, image_prefix + image_foldername);
    +    globalImageCounter += (result && result.images) ? result.images.length : 0;
    +    if (result!==null) {
    +      if (result.sourceGlossary==="start" && !inSrc) {
    +        inSrc=true;
    +        text+="
    \n";
    +      } else if (result.sourceGlossary==="end" && inSrc) {
    +        inSrc=false;
    +        text+="
    \n\n"; + } else if (result.sourceFigCap==="start" && !inSrc) { + inSrc=true; + text+="
    \n";
    +      } else if (result.sourceFigCap==="end" && inSrc) {
    +        inSrc=false;
    +        text+="
    \n\n"; + } else if (result.source==="start" && !inSrc) { + inSrc=true; + text+="
    \n";
    +      } else if (result.source==="end" && inSrc) {
    +        inSrc=false;
    +        text+="
    \n\n"; + } else if (result.inClass==="start" && !inClass) { + inClass=true; + text+="
    \n";
    +      } else if (result.inClass==="end" && inClass) {
    +        inClass=false;
    +        text+="
    \n\n"; + } else if (inClass) { + text+=result.text+"\n\n"; + } else if (inSrc) { + text+=(srcIndent+escapeHTML(result.text)+"\n"); + } else if (result.text && result.text.length>0) { + text+=result.text+"\n\n"; + } + + if (result.images && result.images.length>0) { + for (var j=0; j/g, '>'); +} + +function standardQMarks(text) { + return text.replace(/\u2018|\u8216|\u2019|\u8217/g,"'").replace(/\u201c|\u8220|\u201d|\u8221/g, '"') +} + +// Process each child element (not just paragraphs). +function processParagraph(index, element, inSrc, imageCounter, listCounters, image_path) { + // First, check for things that require no processing. + if (element.getNumChildren()==0) { + return null; + } + // Skip on TOC. + if (element.getType() === DocumentApp.ElementType.TABLE_OF_CONTENTS) { + return {"text": "[[TOC]]"}; + } + + // Set up for real results. + var result = {}; + var pOut = ""; + var textElements = []; + var imagePrefix = "image_"; + + // Handle Table elements. Pretty simple-minded now, but works for simple tables. + // Note that Markdown does not process within block-level HTML, so it probably + // doesn't make sense to add markup within tables. + if (element.getType() === DocumentApp.ElementType.TABLE) { + textElements.push("\n"); + var nCols = element.getChild(0).getNumCells(); + for (var i = 0; i < element.getNumChildren(); i++) { + textElements.push(" \n"); + // process this row + for (var j = 0; j < nCols; j++) { + textElements.push(" \n"); + } + textElements.push(" \n"); + } + textElements.push("
    " + element.getChild(i).getChild(j).getText() + "
    \n"); + } + + // Process various types (ElementType). + for (var i = 0; i < element.getNumChildren(); i++) { + var t = element.getChild(i).getType(); + + if (t === DocumentApp.ElementType.TABLE_ROW) { + // do nothing: already handled TABLE_ROW + } else if (t === DocumentApp.ElementType.TEXT) { + var txt = element.getChild(i); + pOut += txt.getText(); + textElements.push(txt); + } else if (t === DocumentApp.ElementType.INLINE_IMAGE) { + var imglink = element.getChild(i).getLinkUrl(); + result.images = result.images || []; + var blob = element.getChild(i).getBlob() + var contentType = blob.getContentType(); + var extension = ""; + if (/\/png$/.test(contentType)) { + extension = ".png"; + } else if (/\/gif$/.test(contentType)) { + extension = ".gif"; + } else if (/\/jpe?g$/.test(contentType)) { + extension = ".jpg"; + } else { + throw "Unsupported image type: "+contentType; + } + + var name = imagePrefix + imageCounter + extension; + blob.setName(name); + + imageCounter++; + if (!return_string || force_save_images) { + textElements.push('![](' + image_path + '/' + name + ')'); + } else { + textElements.push('![](' + imglink + ')'); + } + //result.images.push( { + // "bytes": blob.getBytes(), + // "type": contentType, + // "name": name}); + + result.images.push({ "blob" : blob } ) + + } else if (t === DocumentApp.ElementType.PAGE_BREAK) { + // ignore + } else if (t === DocumentApp.ElementType.HORIZONTAL_RULE) { + textElements.push('* * *\n'); + } else if (t === DocumentApp.ElementType.FOOTNOTE) { + textElements.push(' ('+element.getChild(i).getFootnoteContents().getText()+')'); + } else { + throw "Paragraph "+index+" of type "+element.getType()+" has an unsupported child: " + +t+" "+(element.getChild(i)["getText"] ? element.getChild(i).getText():'')+" index="+index; + } + } + + if (textElements.length==0) { + // Isn't result empty now? + return result; + } + + var ind_f = element.getIndentFirstLine(); + var ind_s = element.getIndentStart(); + var ind_e = element.getIndentEnd(); + var i_fse = ['ind_f','ind_s','ind_e']; + var indents = {}; + for (indt=0;indt 0) indents[indname] = eval(indname); + // lazy test, null (no indent) is not greater than zero, but becomes set if indent 'undone' + } + var inIndent = (Object.keys(indents).length > 0); + + // evb: Add glossary and figure caption too. (And abbreviations: gloss and fig-cap.) + // process source code block: + if (/^\s*---\s+gloss\s*$/.test(pOut) || /^\s*---\s+source glossary\s*$/.test(pOut)) { + result.sourceGlossary = "start"; + } else if (/^\s*---\s+fig-cap\s*$/.test(pOut) || /^\s*---\s+source fig-cap\s*$/.test(pOut)) { + result.sourceFigCap = "start"; + } else if (/^\s*---\s+src\s*$/.test(pOut) || /^\s*---\s+source code\s*$/.test(pOut)) { + result.source = "start"; + } else if (/^\s*---\s+class\s+([^ ]+)\s*$/.test(pOut)) { + result.inClass = "start"; + result.className = RegExp.$1.replace(/\./g,' '); + } else if (/^\s*---\s*$/.test(pOut)) { + result.source = "end"; + result.sourceGlossary = "end"; + result.sourceFigCap = "end"; + result.inClass = "end"; + } else if (/^\s*---\s+jsperf\s*([^ ]+)\s*$/.test(pOut)) { + result.text = ''; + } else { + + prefix = findPrefix(inSrc, element, listCounters); + + var pOut = ""; + for (var i=0; i): + if (gt === DocumentApp.GlyphType.BULLET + || gt === DocumentApp.GlyphType.HOLLOW_BULLET + || gt === DocumentApp.GlyphType.SQUARE_BULLET) { + prefix += "* "; + } else { + // Ordered list (
      ): + var key = listItem.getListId() + '.' + listItem.getNestingLevel(); + var counter = listCounters[key] || 0; + counter++; + listCounters[key] = counter; + prefix += counter+". "; + } + } + } + return prefix; +} + +function processTextElement(inSrc, txt) { + if (typeof(txt) === 'string') { + return txt; + } + + var pOut = txt.getText(); + if (! txt.getTextAttributeIndices) { + return pOut; + } + +// Logger.log("Initial String: " + pOut) + + // CRC introducing reformatted_txt to let us apply rational formatting that we can actually parse + var reformatted_txt = txt.copy(); + reformatted_txt.deleteText(0,pOut.length-1); + reformatted_txt = reformatted_txt.setText(pOut); + + var attrs = txt.getTextAttributeIndices(); + var lastOff = pOut.length; + // We will run through this loop multiple times for the things we care about. + // Font + // URL + // Then for alignment + // Then for bold + // Then for italic. + + // FONTs + var lastOff = pOut.length; // loop goes backwards, so this holds + for (var i=attrs.length-1; i>=0; i--) { + var off=attrs[i]; + var font=txt.getFontFamily(off) + if (font) { + while (i>=1 && txt.getFontFamily(attrs[i-1])==font) { + // detect fonts that are in multiple pieces because of errors on formatting: + i-=1; + off=attrs[i]; + } + reformatted_txt.setFontFamily(off, lastOff-1, font); + } + lastOff=off; + } + + // URL + // XXX TODO actually convert to URL text here. + var lastOff=pOut.length; + for (var i=attrs.length-1; i>=0; i--) { + var off=attrs[i]; + var url=txt.getLinkUrl(off); + if (url) { + while (i>=1 && txt.getLinkUrl(attrs[i-1]) == url) { + // detect urls that are in multiple pieces because of errors on formatting: + i-=1; + off=attrs[i]; + } + reformatted_txt.setLinkUrl(off, lastOff-1, url); + } + lastOff=off; + } + + // alignment + var lastOff=pOut.length; + for (var i=attrs.length-1; i>=0; i--) { + var off=attrs[i]; + var alignment=txt.getTextAlignment(off); + if (alignment) { // + while (i>=1 && txt.getTextAlignment(attrs[i-1]) == alignment) { + i-=1; + off=attrs[i]; + } + reformatted_txt.setTextAlignment(off, lastOff-1, alignment); + } + lastOff=off; + } + + // strike + var lastOff=pOut.length; + for (var i=attrs.length-1; i>=0; i--) { + var off=attrs[i]; + var strike=txt.isStrikethrough(off); + if (strike) { + while (i>=1 && txt.isStrikethrough(attrs[i-1])) { + i-=1; + off=attrs[i]; + } + reformatted_txt.setStrikethrough(off, lastOff-1, strike); + } + lastOff=off; + } + + // bold + var lastOff=pOut.length; + for (var i=attrs.length-1; i>=0; i--) { + var off=attrs[i]; + var bold=txt.isBold(off); + if (bold) { + while (i>=1 && txt.isBold(attrs[i-1])) { + i-=1; + off=attrs[i]; + } + reformatted_txt.setBold(off, lastOff-1, bold); + } + lastOff=off; + } + + // italics + var lastOff=pOut.length; + for (var i=attrs.length-1; i>=0; i--) { + var off=attrs[i]; + var italic=txt.isItalic(off); + if (italic) { + while (i>=1 && txt.isItalic(attrs[i-1])) { + i-=1; + off=attrs[i]; + } + reformatted_txt.setItalic(off, lastOff-1, italic); + } + lastOff=off; + } + + + var mOut=""; // Modified out string + var harmonized_attrs = reformatted_txt.getTextAttributeIndices(); + reformatted_txt.getTextAttributeIndices(); // @lmmx: is this a typo...? + pOut = reformatted_txt.getText(); + + + // Markdown is farily picky about how it will let you intersperse spaces around words and strong/italics chars. This regex (hopefully) clears this up + // Match any number of \*, followed by spaces/word boundaries against anything that is not the \*, followed by boundaries, spaces and * again. + // Test case at http://jsfiddle.net/ovqLv0s9/2/ + + var reAlignStars = /(\*+)(\s*\b)([^\*]+)(\b\s*)(\*+)/g; + + var lastOff=pOut.length; + for (var i=harmonized_attrs.length-1; i>=0; i--) { + var off=harmonized_attrs[i]; + + var raw_text = pOut.substring(off, lastOff) + + var d1 = ""; // @lmmx: build up a modifier prefix + var d2 = ""; // @lmmx: ...and suffix + + var end_font; + + var mark_bold = false; + var mark_italic = false; + var mark_code = false; + var mark_sup = false; + var mark_sub = false; + var mark_strike = false; + + // The end of the text block is a special case. + if (lastOff == pOut.length) { + end_font = reformatted_txt.getFontFamily(lastOff - 1) + if (end_font) { + if (!inSrc && end_font===end_font.COURIER_NEW) { + mark_code = true; + } + } + if (reformatted_txt.isBold(lastOff -1)) { + mark_bold = true; + } + if (reformatted_txt.isItalic(lastOff - 1)) { + // edbacher: changed this to handle bold italic properly. + mark_italic = true; + } + if (reformatted_txt.isStrikethrough(lastOff - 1)) { + mark_strike = true; + } + if (reformatted_txt.getTextAlignment(lastOff - 1)===DocumentApp.TextAlignment.SUPERSCRIPT) { + mark_sup = true; + } + if (reformatted_txt.getTextAlignment(lastOff - 1)===DocumentApp.TextAlignment.SUBSCRIPT) { + mark_sub = true; + } + } else { + end_font = reformatted_txt.getFontFamily(lastOff -1 ) + if (end_font) { + if (!inSrc && end_font===end_font.COURIER_NEW && reformatted_txt.getFontFamily(lastOff) != end_font) { + mark_code=true; + } + } + if (reformatted_txt.isBold(lastOff - 1) && !reformatted_txt.isBold(lastOff) ) { + mark_bold=true; + } + if (reformatted_txt.isStrikethrough(lastOff - 1) && !reformatted_txt.isStrikethrough(lastOff)) { + mark_strike=true; + } + if (reformatted_txt.isItalic(lastOff - 1) && !reformatted_txt.isItalic(lastOff)) { + mark_italic=true; + } + if (reformatted_txt.getTextAlignment(lastOff - 1)===DocumentApp.TextAlignment.SUPERSCRIPT) { + if (reformatted_txt.getTextAlignment(lastOff)!==DocumentApp.TextAlignment.SUPERSCRIPT) { + mark_sup = true; + } + } + if (reformatted_txt.getTextAlignment(lastOff - 1)===DocumentApp.TextAlignment.SUBSCRIPT) { + if (reformatted_txt.getTextAlignment(lastOff)!==DocumentApp.TextAlignment.SUBSCRIPT) { + mark_sub = true; + } + } + } + + if (mark_code) { + d2 = '`'; // shouldn't these go last? or will it interfere w/ reAlignStars? + } + if (mark_bold) { + d2 = "**" + d2; + } + if (mark_italic) { + d2 = "*" + d2; + } + if (mark_strike) { + d2 = "" + d2; + } + if (mark_sup) { + d2 = '' + d2; + } + if (mark_sub) { + d2 = '' + d2; + } + + mark_bold = mark_italic = mark_code = mark_sup = mark_sub = mark_strike = false; + + var font=reformatted_txt.getFontFamily(off); + if (off == 0) { + if (font) { + if (!inSrc && font===font.COURIER_NEW) { + mark_code = true; + } + } + if (reformatted_txt.isBold(off)) { + mark_bold = true; + } + if (reformatted_txt.isItalic(off)) { + mark_italic = true; + } + if (reformatted_txt.isStrikethrough(off)) { + mark_strike = true; + } + if (reformatted_txt.getTextAlignment(off)===DocumentApp.TextAlignment.SUPERSCRIPT) { + mark_sup = true; + } + if (reformatted_txt.getTextAlignment(off)===DocumentApp.TextAlignment.SUBSCRIPT) { + mark_sub = true; + } + } else { + if (font) { + if (!inSrc && font===font.COURIER_NEW && reformatted_txt.getFontFamily(off - 1) != font) { + mark_code=true; + } + } + if (reformatted_txt.isBold(off) && !reformatted_txt.isBold(off -1) ) { + mark_bold=true; + } + if (reformatted_txt.isItalic(off) && !reformatted_txt.isItalic(off - 1)) { + mark_italic=true; + } + if (reformatted_txt.isStrikethrough(off) && !reformatted_txt.isStrikethrough(off - 1)) { + mark_strike=true; + } + if (reformatted_txt.getTextAlignment(off)===DocumentApp.TextAlignment.SUPERSCRIPT) { + if (reformatted_txt.getTextAlignment(off - 1)!==DocumentApp.TextAlignment.SUPERSCRIPT) { + mark_sup = true; + } + } + if (reformatted_txt.getTextAlignment(off)===DocumentApp.TextAlignment.SUBSCRIPT) { + if (reformatted_txt.getTextAlignment(off - 1)!==DocumentApp.TextAlignment.SUBSCRIPT) { + mark_sub = true; + } + } + } + + + if (mark_code) { + d1 = '`'; + } + + if (mark_bold) { + d1 = d1 + "**"; + } + + if (mark_italic) { + d1 = d1 + "*"; + } + + if (mark_sup) { + d1 = d1 + ''; + } + + if (mark_sub) { + d1 = d1 + ''; + } + + if (mark_strike) { + d1 = d1 + ''; + } + + var url=reformatted_txt.getLinkUrl(off); + if (url) { + mOut = d1 + '['+ raw_text +']('+url+')' + d2 + mOut; + } else { + var new_text = d1 + raw_text + d2; + new_text = new_text.replace(reAlignStars, "$2$1$3$5$4"); + mOut = new_text + mOut; + } + + lastOff=off; +// Logger.log("Modified String: " + mOut) + } + + mOut = pOut.substring(0, off) + mOut; + return mOut; +} \ No newline at end of file diff --git a/demos/palm/python/docs-agent/third_party/g2docsmd-html/patches/001-initial-changes-for-docs-agent.patch b/demos/palm/python/docs-agent/third_party/g2docsmd-html/patches/001-initial-changes-for-docs-agent.patch new file mode 100644 index 000000000..af6d3aa93 --- /dev/null +++ b/demos/palm/python/docs-agent/third_party/g2docsmd-html/patches/001-initial-changes-for-docs-agent.patch @@ -0,0 +1,153 @@ +--- third_party/g2docsmd-html/exportmd.gs 2023-10-20 22:03:56.577441177 +0000 ++++ apps_script/exportmd.gs 2023-10-20 22:45:27.268431292 +0000 +@@ -1,4 +1,22 @@ +-/* ++/** ++ * Copyright 2023 Google LLC ++ * ++ * Licensed under the Apache License, Version 2.0 (the "License"); ++ * you may not use this file except in compliance with the License. ++ * You may obtain a copy of the License at ++ * ++ * http://www.apache.org/licenses/LICENSE-2.0 ++ * ++ * Unless required by applicable law or agreed to in writing, software ++ * distributed under the License is distributed on an "AS IS" BASIS, ++ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. ++ * See the License for the specific language governing permissions and ++ * limitations under the License. ++ */ ++ ++/* Original script is from: ++https://github.com/lmmx/gdocs2md-html/blob/master/exportmd.gs ++and commit: 0d86cfa + Parsing from mangini/gdocs2md. + Modified by clearf to add files to the google directory structure. + Modified by lmmx to write Markdown, going back to HTML-incorporation. +@@ -601,7 +619,7 @@ + + } + +-function convertDocumentToMarkdown(document, destination_folder, optional_switches) { ++function convertDocumentToMarkdown(document, destination_folder, frontmatter_input, optional_switches) { + // if returning a string, force_save_images will make the script continue - experimental + var possible_switches = ['return_string', 'force_save_images']; + var property_name = 'conversion_switches'; +@@ -614,8 +632,13 @@ + + var image_prefix = script_properties.getProperty("image_folder_prefix"); + var numChildren = document.getActiveSection().getNumChildren(); +- var text = ""; +- var md_filename = document.getName()+".md"; ++ if (frontmatter_input != "") { ++ var text = frontmatter_input; ++ } ++ else { ++ var text = "" ++ } ++ var md_filename = sanitizeFileName(document.getName()) + ".md"; + var image_foldername = document.getName()+"_images"; + var inSrc = false; + var inClass = false; +@@ -724,7 +747,7 @@ + } + DriveApp.removeFile(saved_file) // Removes from google drive root. + } +- ++return saved_file; + } + + function escapeHTML(text) { +@@ -738,6 +761,9 @@ + // Process each child element (not just paragraphs). + function processParagraph(index, element, inSrc, imageCounter, listCounters, image_path) { + // First, check for things that require no processing. ++ if (element.getType() === DocumentApp.ElementType.UNSUPPORTED) { ++ return null; ++ } + if (element.getNumChildren()==0) { + return null; + } +@@ -769,6 +795,11 @@ + textElements.push("\n"); + } + ++ // Need to handle this element type, return null for now ++ if (element.getType() === DocumentApp.ElementType.CODE_SNIPPET) { ++ return null ++ } ++ + // Process various types (ElementType). + for (var i = 0; i < element.getNumChildren(); i++) { + var t = element.getChild(i).getType(); +@@ -811,12 +842,38 @@ + + result.images.push({ "blob" : blob } ) + +- } else if (t === DocumentApp.ElementType.PAGE_BREAK) { ++ // Need to fix this case TODO ++ } else if (t === DocumentApp.ElementType.INLINE_DRAWING) { ++ ++ imageCounter++; ++ if (!return_string || force_save_images) { ++ textElements.push('![](' + "drawing" + '/' + " name" + ')'); ++ } else { ++ textElements.push('![](' + "drawing" + ')'); ++ } ++ //result.images.push( { ++ // "bytes": blob.getBytes(), ++ // "type": contentType, ++ // "name": name}); ++ ++ // result.images.push({ "blob" : blob } ) ++ ++ } ++ else if (t === DocumentApp.ElementType.PAGE_BREAK) { + // ignore + } else if (t === DocumentApp.ElementType.HORIZONTAL_RULE) { + textElements.push('* * *\n'); + } else if (t === DocumentApp.ElementType.FOOTNOTE) { + textElements.push(' ('+element.getChild(i).getFootnoteContents().getText()+')'); ++ // Fixes for new elements ++ } else if (t === DocumentApp.ElementType.DATE) { ++ textElements.push(' ('+element.getChild(i)+')'); ++ } else if (t === DocumentApp.ElementType.RICH_LINK) { ++ textElements.push(' ('+element.getChild(i).getUrl()+')'); ++ } else if (t === DocumentApp.ElementType.PERSON) { ++ textElements.push(element.getChild(i).getName() + ', '); ++ } else if (t === DocumentApp.ElementType.UNSUPPORTED) { ++ textElements.push(' '); + } else { + throw "Paragraph "+index+" of type "+element.getType()+" has an unsupported child: " + +t+" "+(element.getChild(i)["getText"] ? element.getChild(i).getText():'')+" index="+index; +@@ -828,10 +885,17 @@ + return result; + } + +- var ind_f = element.getIndentFirstLine(); +- var ind_s = element.getIndentStart(); +- var ind_e = element.getIndentEnd(); +- var i_fse = ['ind_f','ind_s','ind_e']; ++// Fix for unrecognized command getIndentFirstLine ++ var ind_f = 0; ++ var ind_s = 0; ++ var ind_e = 0; ++ if (t === DocumentApp.ElementType.PARAGRAPH) { ++ ++ var ind_f = element.getIndentFirstLine(); ++ var ind_s = element.getIndentStart(); ++ var ind_e = element.getIndentEnd(); ++ } ++ var i_fse = [ind_f,ind_s,ind_e]; + var indents = {}; + for (indt=0;indt