Lithops-3.0.0

New

• [Azure Virtual Machines] Added new 'Azure Virtual Machines' standalone backend, by @JosepSampe

Added

• [Serverless] Added support for python 3.10 and 3.11 runtimes, by @JosepSampe
• [Executor] Allow to set all the compute backend params programatically in the FunctionExecutor(), by @JosepSampe
• [AWS EC2] Allow to automatically create the VPC and all the necessary resources, by @JosepSampe
• [IBM VPC & AWS EC2] General fixes and Improvements, by @JosepSampe
• [Executor] Allow to pass the config file location in the FunctionExecutor(), by @JosepSampe
• [Storage] Automatically create the storage bucket if not provided in the config, by @JosepSampe
• [IBM] Allow to set "region" under "ibm" section, by @JosepSampe
• [AWS] Allow to set "region" under "aws" section, by @JosepSampe
• [Cli] Added new 'lithops image build' command for standalone backends, by @JosepSampe
• [Cli] Added new 'lithops image list' command for standalone backends, by @JosepSampe
• [IBM VPC] Added build_image() method for automatically building VM images, by @JosepSampe
• [IBM VPC] Added list_image() method for listing the availabe VM images, by @JosepSampe
• [AWS EC2] Added build_image() method for automatically building VM images, by @JosepSampe
• [AWS EC2] Added list_image() method for listing the availabe VM images, by @JosepSampe
• [Azure VMS] Added list_image() method for listing the availabe VM images, by @JosepSampe
• [IBM CF] Automatically create a CF namespace if not provided in config, by @JosepSampe
• [IBM VPC] Added Madrid (Spain) region, by @JosepSampe
• [Code Engine] Automatically create a new project if not provided in config, by @JosepSampe

Changed

• [Azure] Changed 'location' config parameter to 'region' for compatibility, by @JosepSampe
• [Aliyun] Changed 'endpoint' config parameter to 'region' for compatibility, by @JosepSampe
• [AWS EC2] Reduced number of mandatory parameters, by @JosepSampe
• [AWS] Allow 'region' config parameter instead of 'region_name' for compatibility, by @JosepSampe
• [IBM CF] Cloud-foundry namespaces have been deprecated in IBM Cloud. From now all the users must use an IAM-based namespace, by @JosepSampe
• [IBM COS] Changed 'access_key' config parameter to 'access_key_id' for compatibility, by @JosepSampe
• [IBM COS] Changed 'secret_key' config parameter to 'secret_access_key' for compatibility, by @JosepSampe
• [IBM] Improved token manager, by @JosepSampe
• [Core] Job creation now checks that each element in 'iterdata' is smaller than 8 KB, by @JosepSampe
• [MapReduce] Make 'results' variable name not mandatory in the reduce function signature, by @JosepSampe
• [CLI] Renamed 'lithops test' command to 'lithops hello', by @JosepSampe
• [CLI] Renamed 'lithops verify' command to 'lithops test', by @JosepSampe

Fixed

• [IBM VPC & AWS EC2] Make sure only VMs from the given VPC are removed, by @JosepSampe
• [IBM VPC] Reuse floating IPs for public gateways, by @JosepSampe
• [Serializer] Prevent analyzing the same module multiple times, by @JosepSampe
• [SSH Cli] Fix issue with RSA keys, by @JosepSampe
• [Tests] Fix broken links of testing files, by @JosepSampe
• [Azure Container APPs] Make sure the lithops worker app is deployed, by @JosepSampe
• [AWS Lambda] Fixed error with urllib3. Pin urllib3 to <2 as for now botocore does not support urllib3 2.0, by @JosepSampe
• [Multiprocessing] Check redis connection before starting to submit jobs, by @JosepSampe
• [Redis] Fixed redis backend exception regarding storage_bucket, by @JosepSampe

---

⚠ BREAKING CHANGES

• [Azure] Changed 'location' config parameter to 'region' for compatibility. Please follow the new instructions to configure the backend
• [Aliyun] Changed 'endpoint' config parameter to 'region' for compatibility. Please follow the new instructions to configure the backend
• [IBM CF] Cloud-foundry namespaces have been deprecated in IBM Cloud. From now all the users must use an IAM-based namespace

Lithops-2.9.0

Added

• [AWS S3] Allow to use a public bucket without credentials, by @JosepSampe
• [IBM] Automatically login to the container registry if the credentials are present in config, by @JosepSampe
• [IBM] Force --platform=linux/amd64 in the "lithops runtime build" command, by @JosepSampe
• [k8s] Added boto3 as a dependency of the default runtime, by @JosepSampe
• [IBM VPC] Automatically get the ubuntu image ID, by @JosepSampe
• [IBM VPC] Allowed to reuse floating IPs, by @JosepSampe
• [IBM VPC] Automatically create resources if not provided in config, by @JosepSampe
• [IBM VPC] Added 'region' config parameter, by @JosepSampe
• [Partitioner] Allow wildcards in the object reference, by @gilv

Changed

• [IBM VPC] Reduced the number of mandatory config parameters, by @JosepSampe
• [IBM VPC] Renamed profile_name config param to worker_profile_name, by @JosepSampe
• [IBM VPC] Renamed ip_address config param to floating_ip, by @JosepSampe

Fixed

• [AWS EC2] Fix spot instance price, by @JosepSampe
• [Cli] Fix wrong config in the "lithops runtime update" command, by @JosepSampe
• [Standalone] Fix missing private IP address, by @JosepSampe
• [VM] Fix VM standalone backend, by @JosepSampe

Lithops-2.8.0

Added

• [Config] Allow to store the config file in "/etc/lithops/config", by @JosepSampe
• [CLI] Allow to specify 'memory' and 'version' in the 'lithops runtime delete' command, by @JosepSampe
• [GCP Cloudrun] Allow setting min_workers to the autoscaler, by @JosepSampe
• [GCP Functions] Added https trigger, by @JosepSampe
• [Function Executor] Added additional arguments to pass to the reduce function in map_reduce(), by @JosepSampe
• [AWS] Added session token as optional, by @aitorarjona

Changed

• [Core] Include function result in the status.json file if it is < 8KB, by @JosepSampe
• [Core] Include python version in the lithops worker name, by @JosepSampe

Fixed

• [Serverless] Allow to delete runtimes from different lithops versions, by @JosepSampe
• [AWS Batch] Fixed list_runtimes() method, by @JosepSampe
• [localhost] Fixed localhost storage backend head method, by @LachlanStuart

Lithops-2.7.1

Added

• [Google Cloud Functions] Added Python 3.10 runtime compatibility, by @JosepSampe
• [Core] Allow to automatically transfer .so (cythonized .py) files if referenced in the code, by @JosepSampe

Changed

• [Core] Improved cython coverage, by @JosepSampe
• [IBM VPC] Make 'image_id' mandatory in config, by @JosepSampe
• [IBM VPC] Infer zone_name from subnet, by @JosepSampe
• [Knative] Reduced service name length, by @JosepSampe
• [AWS EC2] Updated ec2 default ubuntu image to 22.04, by @JosepSampe

Fixed

• [IBM VPC] Create floating IP using the specified resource group, by @JosepSampe
• [IBM VPC] Attach floating IP to the subnet, by @JosepSampe
• [Multiprocessing] Fix 'cpu_count' function from multiprocessing API, by @gfinol
• [Code Engine] Add CE conflict exception to retriables, by @kpavel
• [Core] Show logs from module_dependency.py, by @JosepSampe
• [GCP Functions] Fix runtime_build command, by @JosepSampe
• [Infinispan] Fix infinispan storage backend, by @JosepSampe
• [Core] Detect a Class if passed as a lithops input function, by @JosepSampe

Lithops-2.7.0

New

• [Azure Container APPs] Added new 'Azure Container APPs' serverless backend, by @JosepSampe

Added

• [Knative] Added Kourier as the networking layer, by @JosepSampe
• [AWS Lambda] Added "env_vars" and "ephemeral_storage" parameters for lambda runtime config, by @aitorarjona
• [Azure Functions] Allow to build a runtime from a custom requirements.txt file, by @JosepSampe
• [K8s] Append 'docker_server' as a prefix of the runtime, by @JosepSampe
• [Code Engine] Append 'docker_server' as a prefix of the runtime, by @JosepSampe
• [Knative] Append 'docker_server' as a prefix of the runtime, by @JosepSampe
• [Google Cloud Storage] Add missing logic in gcp_storage, by @JosepSampe
• [Google Cloud] project_name and service_account are no longer required in config, by @JosepSampe
• [Google Cloud] Allow to use GOOGLE_APPLICATION_CREDENTIALS for service_credentials, by @JosepSampe
• [Google Cloud Run] Allow CPU values <1, 6 and 8, by @JosepSampe
• [Alibaba Cloud Functions] Added Python 3.9 runtime compatibility, by @JosepSampe
• [Alibaba Cloud Functions] Allow to build a runtime from a custom requirements.txt file, by @JosepSampe
• [Core] Add support for building container runtimes with podman, by @JosepSampe
• [Core] Delete only runtimes from the specified backend on 'lithops clean', by @JosepSampe
• [Executor] Added obj_newline parameter in map() and map_reduce() methods, by @JosepSampe
• [Infinispan] Support DIGEST authentication by default with the Infinispan REST backend, by @tristantarrant

Changed

• [Core] Load docker command only when needed instead of loading it always, by @JosepSampe
• [Core] Load runtime data only on invocation, by @JosepSampe
• [Google Cloud] project_name and service_account config parameters are no longer required, by @JosepSampe
• [Multiprocessing] Improved remote logging, by @aitorarjona
• [Monitor] Reduce debug log verbosity, status is printed every 30s or when a future changes state, by @aitorarjona
• [AWS Batch] Increased resource limits, by @aitorarjona
• [Executor] Changed 'reducer_one_per_object' parameter to 'obj_reduce_by_key', by @JosepSampe

Fixed

• [Google Cloud Functions] Fixed errors when deploying a function, by @JosepSampe
• [Core] Make sure all temp files generated during build_runtime() are cleaned, by @JosepSampe
• [Core] Fix internal partitioner, by @JosepSampe
• [knative] AttributeError: 'KnativeServingBackend' object has no attribute 'coreV1Api', by @JosepSampe
• [knative] Fixed service deployment, by @JosepSampe
• [Alibaba Cloud Functions] Fixed errors when deploying a function, by @JosepSampe
• [Azure Functions] Fixed errors when deploying a function, by @JosepSampe
• [Azure Functions] Fixed issue that was preventing building runtimes from a non-Linux system, by @JosepSampe
• [Code Engine] Fix runtime_timeout not being applied, by @JosepSampe

Lithops-2.6.0

Added

• [Code Engine] Improved except-retry logic, by @gilv
• [IBM COS] Enables separate IAM authorization for COS and Compute backends, by @gilv

Lithops-2.5.9

Added

• [Core] Add support for Python 3.10, by @JosepSampe
• [Storage] Added download_file() and upload_file() methods to Storage API to enable multipart upload/download, by @JosepSampe
• [AWS Lambda] Added 'architecture' option in config to allow deploy arm64 runtimes, by @JosepSampe
• [AWS Batch] Added 'service_role' config parameter, by @aitorarjona
• [IBM VPC] add gpu support for ibm_vpc backend, by @JosepSampe
• [Standalone] Added docker login to standalone setup script, by @kpavel
• [AWS EC2] Automatically get the default Ubuntu 20.04 AMI when not present in config, by @JosepSampe
• [Code Engine] Added retry logic on connection exception, by @kpavel

Changed

• [CLI] Renamed "lithops runtime create" command to "lithops runtime deploy", by @JosepSampe

Fixed

• [AWS Lambda] Fixed "layer builder" function creation necessary to deploy the default runtime, by @aitorarjona
• [AWS Lambda] Allow to create container runtimes whose names contain '.', '/' and '_'., by @aitorarjona
• [AWS Lambda] List only the runtimes deployed by the current user, by @JosepSampe
• [AWS Lambda] Do not raise an exception if the runtime is already deployed, by @JosepSampe
• [Standalone] Fix cloudinit initialization script, by @JosepSampe
• [Future] Fix host_status_query_count stat, by @JosepSampe
• [Google Cloud Run] Fixed wrong variable name 'runtime_cpus', by @MPNeuber

Lithops-2.5.8

Added

• [Standalone] Raise an exception when the ssh key is not found, by @JosepSampe
• [Standalone] Raise an exception when the ssh key is not valid for login to the mater VM, by @kpavel

Fixed

• [IBM VPC] Fixed get_ssh_client() method that in certain circumstances was not working properly, by @kpavel

Lithops-2.5.7

Added

• [AWS Batch] Added AWS Batch backend, by @aitorarjona
• [Standalone] Allow to start workers using a public key instead of using a password, by @JosepSampe
• [Standalone] Added diferent levels of worker verification, by @kpavel
• [Infinispan] Added new Infinispan Hot Rod storage backend, by @rigazilla

Fixed

• [Core] Fixed map_reduce jobs, by @JosepSampe
• [Standalone] Fixed internal executions in standalone, by @JosepSampe
• [IBM VPC] Fixed issue that prevented creating workers on create|reuse modes, by @kpavel
• [IBM VPC] Fixed issue with ip_address in consume mode, by @JosepSampe
• [AWS Lambda] Allow to delete functions from other lithops versions through 'lithops runtime delete', by @JosepSampe
• [Core] Fixed FunctionExecutor.plot() when a list of futures is passed to the method, by @aitorarjona
• [Infinispan] Fixes in Infinispan storage backend, by @rigazilla

Lithops-2.5.6

Added

• [AWS_EC2] Added AWS EC2 Standalone backend, by @JosepSampe
• [AWS_EC2] Allow to start workers using Spot instances in AWS EC2 Standalone backend, by @JosepSampe
• [Standalone] Added the logic to create the missing delta of workers in reuse mode, by @kpavel
• [Standalone] Cancel running job tasks on ctrl-c, by @JosepSampe
• [Standalone] New logic to verify that the master VM is correctly setup, by @kpavel
• [Standalone] Added new command "lithops attach" that allows to create live ssh connections to the master VM, by @JosepSampe
• [Config] Allow to set monitoring_interval in config, by @JosepSampe

Changed

• [Standalone] Improved the performance of the master VM when getting the free available workers (reuse mode), by @JosepSampe

Fixed

• [Standalone] Fixed VM initial installation script, by @kpavel
• [Standalone] Fixed get_workers method on master, by @kpavel
• [Standalone] Deleted unnecessary extra worker, by @JosepSampe
• [Standalone] Ensure all workers are properly started on reuse mode, by @JosepSampe
• [Localhost] Fixed storage delete_objects method that was deleting the entire folder of a file, by @JosepSampe
• [IBM VPC] General fixes in IBM VPC backend, by @JosepSampe