diff --git a/.github/styles/HouseStyle/tech-terms/phrases.txt b/.github/styles/HouseStyle/tech-terms/phrases.txt index 842fb916b..d58c73270 100644 --- a/.github/styles/HouseStyle/tech-terms/phrases.txt +++ b/.github/styles/HouseStyle/tech-terms/phrases.txt @@ -61,4 +61,7 @@ dsn bom ko dpkg -apk \ No newline at end of file +apk + +uv +cmd \ No newline at end of file diff --git a/blog/_posts/2021-07-27-continuous-testing-in-devops.md b/blog/_posts/2021-07-27-continuous-testing-in-devops.md index 8c35af9de..ce5009fae 100644 --- a/blog/_posts/2021-07-27-continuous-testing-in-devops.md +++ b/blog/_posts/2021-07-27-continuous-testing-in-devops.md @@ -26,8 +26,8 @@ Extending the CI process by adding automated tests is referred to as continuous A good CI/CT process always contains at least the following steps: -- build -- deployment +- Build +- Deployment - Integration tests - End-to-end tests @@ -81,7 +81,7 @@ A GitHub Actions workflow contains three elements: It's straightforward to extend a workflow once you understand those three concepts. Here's a sample workflow for a Python application: -```yaml +~~~ name: Python application on: @@ -113,7 +113,7 @@ jobs: - name: Test with pytest run: | pytest -``` +~~~ This what a basic workflow looks like: @@ -132,7 +132,7 @@ But let's not rely on a third party yet. Instead, generate a badge to display in - Generate the badge and add it to your README. Follow the setup step in the documentation of [schneegans/dynamic-badges-action@v1.1.0](https://github.com/Schneegans/dynamic-badges-action). {% raw %} -```yml +~~~ # This workflow will install Python dependencies, run tests, and lint with a single version of Python # For more information see: https://help.github.com/actions/language-and-framework-guides/using-python-with-github-actions @@ -202,7 +202,7 @@ jobs: label: coverage message: ${{ env.COVERAGE }} color: ${{ env.COLOR }} -``` +~~~ {% endraw %} @@ -238,7 +238,7 @@ Now edit your workflow configuration: - Run Newman using the Action you just found in the Marketplace. - Move the gating job at the end of the workflow by changing the `needs` property. -```yaml +~~~ name: Python application on: @@ -262,7 +262,7 @@ jobs: gating: needs: tests_api [...] # we move the gating at the end of the workflow -``` +~~~ Now your workflow should contain four sequential jobs: @@ -288,7 +288,7 @@ Have a look at the result: Your configuration should look along those lines to achieve this workflow: -```yaml +~~~ name: Python application on: @@ -328,7 +328,7 @@ jobs: gating: needs: [tests_api, test_e2e] [...] # move the gating at the end of the workflow -``` +~~~ ### Add Performance Testing @@ -350,7 +350,7 @@ As before, create a new job (called `test_performance`). This time I did not fin Here is the workflow I came up with for my Python application: -```yml +~~~ name: Python application on: @@ -396,7 +396,7 @@ jobs: comment-on-alert: true fail-on-alert: true alert-comment-cc-users: '@xNok' -``` +~~~ Your final workflow must look like this: diff --git a/blog/_posts/2021-11-24-spinnaker-kubernetes.md b/blog/_posts/2021-11-24-spinnaker-kubernetes.md index b50269ad2..1ea34aa29 100644 --- a/blog/_posts/2021-11-24-spinnaker-kubernetes.md +++ b/blog/_posts/2021-11-24-spinnaker-kubernetes.md @@ -187,7 +187,7 @@ Spinnaker allows you to define providers that are integrated into the cloud plat - Google App Engine - AWS - Azure -- cloud Foundry +- Cloud Foundry - DC/OS - Google Compute Engine - Kubernetes diff --git a/blog/_posts/2022-03-09-golang-monorepo.md b/blog/_posts/2022-03-09-golang-monorepo.md index 411c57fbc..97f9987bb 100644 --- a/blog/_posts/2022-03-09-golang-monorepo.md +++ b/blog/_posts/2022-03-09-golang-monorepo.md @@ -54,7 +54,7 @@ monorepo. **Cons:** -- build tooling can be more complicated in a monorepo +- Build tooling can be more complicated in a monorepo - It can be easy to accidentally tightly-couple components that should be decoupled - Components may be less autonomous, and developers may have less freedom to do things "their own way" diff --git a/blog/_posts/2022-04-8-docker-vagrant.md b/blog/_posts/2022-04-8-docker-vagrant.md index 46d4f1a9a..c719dc730 100644 --- a/blog/_posts/2022-04-8-docker-vagrant.md +++ b/blog/_posts/2022-04-8-docker-vagrant.md @@ -65,7 +65,7 @@ However, this involves configuring inter-container networking, service access, p There are some great benefits of using Dockerized application containers. For example: - Dockerfiles, their associated images and the containers are lightweight and portable. -- containers use fewer resources than VMs. +- Containers use fewer resources than VMs. - Building and deploying containers takes less time than deploying VMs. - A physical host can run many more containers than virtual machines. diff --git a/blog/_posts/2022-07-25-pongo.md b/blog/_posts/2022-07-25-pongo.md index 852599e4b..594b97d49 100644 --- a/blog/_posts/2022-07-25-pongo.md +++ b/blog/_posts/2022-07-25-pongo.md @@ -33,7 +33,7 @@ In this **first article** we will learn how to: - Start working with the tcell package - Write text to the terminal -- make text move +- Make text move - Create a ball that "bounces" when it reaches the edge of the screen [The complete code for part one](https://github.com/jalletto/pongo). diff --git a/blog/_posts/2023-05-10-logging-in-python.md b/blog/_posts/2023-05-10-logging-in-python.md index 00a6605d4..5cd6938e7 100644 --- a/blog/_posts/2023-05-10-logging-in-python.md +++ b/blog/_posts/2023-05-10-logging-in-python.md @@ -851,7 +851,7 @@ In order to effectively use logging in Python, you should consider the following - Log at the appropriate level to ensure that your logs contain the necessary information without being cluttered with unnecessary details. - Consider using custom loggers to help organize and categorize your logs based on different parts of your code or different components of your application. - Rotate your logs to save disk space and ensure that you can easily find and analyze relevant logs from different points in time. -- make sure to handle errors and exceptions appropriately in your logging code to avoid unexpected behavior and ensure that you capture all relevant information. +- Make sure to handle errors and exceptions appropriately in your logging code to avoid unexpected behavior and ensure that you capture all relevant information. - Consider using structured logging formats like [JSON](/blog/convert-to-from-json) or XML to make it easier to parse and analyze your logs with automated tools. ## Conclusion diff --git a/blog/_posts/2023-05-11-oauth-2-in-non-web-clients.md b/blog/_posts/2023-05-11-oauth-2-in-non-web-clients.md index 798373b83..a1b134256 100644 --- a/blog/_posts/2023-05-11-oauth-2-in-non-web-clients.md +++ b/blog/_posts/2023-05-11-oauth-2-in-non-web-clients.md @@ -608,7 +608,7 @@ Recall that the browser flow does not occur on the non-web client. So, the non-w The `get_access_token_from_login_code` function involves the following steps: -- makes an API call to the Facebook access token endpoint with the device code and access token passed as query params. +- Makes an API call to the Facebook access token endpoint with the device code and access token passed as query params. - Calls the `_handle_error_from_login_code` function if there is an error. If the error is not a `PendingActionError` it raises an exception that terminates the function execution and if otherwise, delays the code by the 5s using `await asyncio.sleep()`. - The polling is achieved by using [recursion](https://users.cs.utah.edu/~germain/PPS/Topics/recursion.html). You can also use a loop to achieve the polling. diff --git a/blog/_posts/2023-06-12-using-bazel-with-typescript.md b/blog/_posts/2023-06-12-using-bazel-with-typescript.md index 6a6f02d08..3c6a4e67e 100644 --- a/blog/_posts/2023-06-12-using-bazel-with-typescript.md +++ b/blog/_posts/2023-06-12-using-bazel-with-typescript.md @@ -193,7 +193,7 @@ The previous rules in the `BUILD` file tell Bazel the following: - The rule is named `transpile`. - The input files that need to be compiled are to be found in the path `src/index.ts`. -- bazel should fetch the compiler configuration from the `tsconfig.json` file in the current directory, as denoted by `//`. +- Bazel should fetch the compiler configuration from the `tsconfig.json` file in the current directory, as denoted by `//`. - `allow_js` is set as `True`, which means that TypeScript should allow JavaScript files to be imported within the `.ts` project files. - `resolve_json_module` is set as `True`, which means that Bazel should allow the import of `.json` extension files within the `.ts` project files. diff --git a/blog/_posts/2023-09-05-image-upload-api-cloudinary-golang.md b/blog/_posts/2023-09-05-image-upload-api-cloudinary-golang.md index d186763df..b10840461 100644 --- a/blog/_posts/2023-09-05-image-upload-api-cloudinary-golang.md +++ b/blog/_posts/2023-09-05-image-upload-api-cloudinary-golang.md @@ -31,7 +31,7 @@ Storing media in Cloudinary brings advantages like improved performance, scalabi To implement an image upload system in Golang, you need the following prerequisites: - Basic understanding of REST APIs in Golang -- cloudinary account +- Cloudinary account - [Go installation](https://go.dev/doc/install) in a local environment. The code used in this tutorial can be found on [my GitHub repository](https://github.com/Tee-Stark/go-image-uploader), in case you'd like to use it as a reference as you follow along. Let's get started! diff --git a/blog/_posts/2023-09-12-shutting-down-earthly-ci.md b/blog/_posts/2023-09-12-shutting-down-earthly-ci.md index 409609b14..26b9bb21f 100644 --- a/blog/_posts/2023-09-12-shutting-down-earthly-ci.md +++ b/blog/_posts/2023-09-12-shutting-down-earthly-ci.md @@ -207,7 +207,7 @@ It seems that Earthly Satellites are taking off, not just because we are deliver - Satellite metrics – including CPU, memory, disk, and network I/O usage. -- build history – for both local and Satellites builds – in the web UI. +- Build history – for both local and Satellites builds – in the web UI. - Auto-skip – the ability to skip a build instantly if the changed files don't impact it. diff --git a/blog/_posts/2024-06-04-earthly-cloud-ui-updates.md b/blog/_posts/2024-06-04-earthly-cloud-ui-updates.md index 725e3811f..05b3afe25 100644 --- a/blog/_posts/2024-06-04-earthly-cloud-ui-updates.md +++ b/blog/_posts/2024-06-04-earthly-cloud-ui-updates.md @@ -26,7 +26,7 @@ The Earthly Cloud home dashboard isn't new, but it's a great entry point to Eart ## New Build Details Screen -The new Build Details screen aims to give you more information about your builds, including what commands were executed, how long different parts of the build took to execute, the build graph, and build logs. It has 4 tabs: Overview, Timings, Graph, and Logs. +The new Build Details screen aims to give you more information about your builds, including what commands were executed, how long different parts of the build took to execute, the build graph, and build logs. It has 4 tabs: Overview, Timings, Graph, and Logs. ### Overview diff --git a/blog/_posts/2024-06-13-python-uv.md b/blog/_posts/2024-06-13-python-uv.md new file mode 100644 index 000000000..f9da0d616 --- /dev/null +++ b/blog/_posts/2024-06-13-python-uv.md @@ -0,0 +1,289 @@ +--- +title: "How to Create a Python Virtual Environment with uv" +toc: true +author: Furqan Butt + +internal-links: + - python virtual environment with uv + - python virtual environment + - create python virtual environment + - create python virtual environment with uv +categories: + - python-tooling + - python +--- + +Imagine you have a Python 3.10 backend application with packages a2.1, b2.2, and c2.3 installed system-wide. Everything works fine until you start a new project that also uses Python 3.10 but needs a1.2, b2.2, and c2.1. Installing these new packages causes dependency issues with your first project because a2.1 and c2.3 are no longer available. If you start a third project requiring Python 3.8, you would need to downgrade Python, causing further conflicts. + +Virtual environments solve these problems by isolating dependencies for each project, allowing you to maintain different packages and Python versions without conflict. A virtual environment is an independent environment created on top of an existing Python installation. It has its own independent set of Python packages installed in its site directories and only contains packages from its base environment (the system-wide Python installation) if explicitly specified. The environment is disposable and can be easily deleted and recreated as required. + +In this tutorial, you'll learn how to set up and use virtual environments using [uv](https://astral.sh/blog/uv), a package installer that's easy to use and performs [10 to 100 times](https://github.com/astral-sh/uv/blob/main/BENCHMARKS.md) better than pip. + +## Installing `uv` + +To create a virtual environment with uv, you need to start by installing it. There are multiple ways to do so, depending on your device. If you're using Linux or Mac, you can run the following command to install uv: + +~~~{.bash caption=">_"} +curl -LsSf https://astral.sh/uv/install.sh | sh +~~~ + +On Mac, you also have the option to install uv via [Homebrew](https://brew.sh/): + +~~~{.bash caption=">_"} +brew install uv +~~~ + +On Windows, you can run the following: + +~~~{.bash caption=">_"} +powershell -c "irm https://astral.sh/uv/install.ps1 | iex" +~~~ + +Or you can install uv with pip: + +~~~{.bash caption=">_"} +pip install uv +~~~ + +After installation, verify uv on your device by running the following command: + +~~~{.bash caption=">_"} +uv --version +~~~ + +You will get an output like this: + +~~~{ caption="Output"} +uv 0.1.44 +~~~ + +The rest of this tutorial assumes you're operating on a Linux or Mac operating system. + +## Creating a Project and a Virtual Environment Using uv + +Once uv is installed, you need to create a project and install the necessary dependencies: + +~~~{.bash caption=">_"} +mkdir my_uv_project +~~~ + +Then, change into this directory via `cd my_uv_project`. + +Creating a virtual environment for your project is fairly simple. Run the following command: + +~~~{.bash caption=">_"} +uv venv my_env +~~~ + +You should get a similar output to this: + +~~~{ caption="Output"} + +user@guest-MacBook-Air my_uv_project % uv venv my_env +Using Python 3.9.6 interpreter at: /Library/Developer/CommandLineTools/usr/bin/python3 +Creating virtualenv at: my_env +Activate with: source my_env/bin/activate +~~~ + +You can see that a Python 3.9 environment has been created. + +## Activating Your Environment + +Once you've created your virtual environment, you need to activate it by running the following command: + +~~~{.bash caption=">_"} +source my_env/bin/activate +~~~ + +To verify that the environment you created doesn't contain any packages, run the following command: + +~~~{.bash caption=">_"} +uv pip list +~~~ + +Your output will not list anything, and any packages or dependencies you install in this environment will be isolated from the rest of the system. + +**Note:** If you're using a different shell than Bash, such as cmd, you'll have different activate scripts, such as `my_envScriptsactivate.bat`. + +## Installing Packages in Your Environment + +Once you've activated your environment, it's time to install some packages. First, install [pandas](https://pandas.pydata.org/) with the following command: + +~~~{.bash caption=">_"} +uv pip install pandas +~~~ + +You'll get an output similar to this: + +~~~{ caption="Output"} + +(my_env) user@guest-MacBook-Air my_uv_project % uv pip install pandas +Resolved 6 packages in 143ms +Installed 6 packages in 76ms + + numpy==1.26.4 + + pandas==2.2.2 + + python-dateutil==2.9.0.post0 + + pytz==2024.1 + + six==1.16.0 +~~~ + +As you can see, pandas version 2.2.2 has been installed. When you install pandas, other supporting packages are also installed. + +Verify your package installation by running the following command: + +~~~{.bash caption=">_"} +uv pip list +~~~ + +You'll get an output similar to this: + +~~~{ caption="Output"} + +(my_env) user@guest-MacBook-Air my_uv_project % uv pip list +Package Version +--------------- ----------- +numpy 1.26.4 +pandas 2.2.2 +python-dateutil 2.9.0.post0 +pytz 2024.1 +six 1.16.0 +tzdata 2024.1 +~~~ + +## Creating a Second Virtual Environment + +To test the isolation of environments, you'll create a second environment and install a different version of pandas with another package. To do so, perform the same steps as above to create a virtual environment named `my_second_env`: + +~~~{.bash caption=">_"} + +user@guest-MacBook-Air my_uv_project % uv venv my_second_env +Using Python 3.9.6 interpreter at: /Library/Developer/CommandLineTools/usr/bin/python3 +Creating virtualenv at: my_second_env +Activate with: source my_second_env/bin/activate +~~~ + +Next, install [Flask](https://flask.palletsprojects.com/en/3.0.x/) and pandas version 2.1.0 by running this command: + +~~~{.bash caption=">_"} +my_uv_project % uv pip install pandas==2.1.0 flask +~~~ + +Upon installation, you'll get an output similar to this: + +~~~{ caption="Output"} + +(my_second_env) user@guest-MacBook-Air my_uv_project % uv pip install pandas==2.1.0 flask +Resolved 15 packages in 195ms +Downloaded 2 packages in 4.48s +Installed 15 packages in 93ms + + blinker==1.8.2 + + click==8.1.7 + + flask==3.0.3 + + importlib-metadata==7.1.0 + + itsdangerous==2.2.0 + + jinja2==3.1.4 + + markupsafe==2.1.5 + + numpy==1.26.4 + + pandas==2.1.0 + + python-dateutil==2.9.0.post0 + + pytz==2024.1 + + six==1.16.0 + + tzdata==2024.1 + + werkzeug==3.0.3 + + zipp==3.19.1 +~~~ + +As you can see, it does not contain the pandas package version 2.2.2 that you installed in the `my_env` virtual environment. + +## Creating a Virtual Environment with a Different Python Version + +You can also create a virtual environment with a different Python version. The earlier environments you created had Python 3.9. Let's now create one that has Python 3.11. To do so, execute the following command: + +~~~{.bash caption=">_"} +uv venv 3rd_env --python 3.11 +~~~ + +You'll get a similar output to this: + +~~~{ caption="Output"} + +Using Python 3.11.5 interpreter at: /Library/Developer/CommandLineTools/usr/bin/python3.11 +Creating virtualenv at: 3rd_env +Activate with: source 3rd_env/bin/activate +~~~ + +You can once again install pandas here: + +~~~{.bash caption=">_"} + +user@guest-MacBook-Air my_uv_project % source 3rd_env/bin/activate +(3rd_env) user@guest-MacBook-Air my_uv_project % uv pip install pandas +Resolved 6 packages in 215ms +Downloaded 2 packages in 2.59s +Installed 6 packages in 72ms + + numpy==1.26.4 + + pandas==2.2.2 + + python-dateutil==2.9.0.post0 + + pytz==2024.1 + + six==1.16.0 + + tzdata==2024.1 +~~~ + +## Switching between Virtual Environments and Removing a Virtual Environment + +To switch between different virtual environments, you can deactivate the first one by running this command: + +~~~{.bash caption=">_"} +deactivate +~~~ + +Follow it by running this command: + +~~~{.bash caption=">_"} +source my_second_env/bin/activate +~~~ + +If you now run the `ls` command in your project directory, you'll see all the environments listed: + +~~~{.bash caption=">_"} +user@guest-MacBook-Air my_uv_project % ls +3rd_env my_env my_second_env +~~~ + +As you can see, these environments are created like regular directories under your project. If you want to remove one, you can delete the environment folder like this: + +~~~{.bash caption=">_"} +rm -rf my_second_env +~~~ + +This will delete the second environment you created. + +## Overcoming the Limitations of `uv` + +![Limitations]({{site.images}}{{page.slug}}/limitations.png)\ + +While uv can act as a great replacement for pip, it only manages dependencies related to Python. + +Let's say your projects involve installing packages system-wide, meaning they affect all users and applications on the system. For instance, you may want to install the [libpq-dev](https://www.postgresql.org/docs/devel/libpq.html) package for your [PostgreSQL](https://www.postgresql.org/) backend server. uv is not designed to manage such system-wide package installations. + +Another limitation of uv is that it lacks [containerization](https://en.wikipedia.org/wiki/Containerization_(computing)) capabilities. It has no caching mechanism to help speed up the build process by reusing identical dependencies in different virtual environments. If you use multiple virtual environments for a single project (for example, to test the project in different Python versions), uv will install every dependency in each virtual environment, even if some of them are identical. This can be inefficient and take up a lot of space and time for a large project. + +These limitations make it cumbersome for larger Python projects or those with many dependencies. + +[Earthly](https://earthly.dev/) helps address these issues by simplifying the dependency management process, ensuring consistency across different environments, and streamlining the build and deployment process. + +For instance, you can use Earthly to install your Python-related dependencies (like pandas) as usual. If you also need to install system-wide dependencies like libpq-dev, you can do so within the same environment. + +Earthly is designed to work like a [Docker container](https://www.docker.com/resources/what-container/). It uses an `Earthfile`, which is very similar to a `Dockerfile`. You can add and install all your Python and system-wide dependencies in a single `Earthfile`, and your project is good to go. + +It also offers fast build speed and supports caching, which reuses intermediate build steps when dependencies or code haven't changed. Other notable features include [dependency management](https://earthly.dev/blog/python-earthly/), maintaining virtual environments, managing language-agnostic builds, and multiplatform deployments and CI/CD integrations. + +## Conclusion + +In this article, you saw how to use virtual environments to help isolate development environments to avoid package-related issues and conflicts in Python with uv. + +Even though uv works well for this use case, it has limitations. It only works for Python environments, so you can't use it to install system-wide dependencies. It also doesn't offer containerization capabilities. + +If you need these features, consider [Earthly](https://earthly.dev/), a simple build tool that helps you manage dependencies and maintain virtual environments. Similar to Docker containers, it offers many familiar features like containerization, caching and reusing builds, and installing dependencies that can span system-wide. It can easily manage large projects involving many dependencies and provides a great development experience. + +{% include_html cta/bottom-cta.html %} diff --git a/blog/assets/images/python-uv/header.jpg b/blog/assets/images/python-uv/header.jpg new file mode 100644 index 000000000..3d9be65c0 Binary files /dev/null and b/blog/assets/images/python-uv/header.jpg differ diff --git a/blog/assets/images/python-uv/limitations.png b/blog/assets/images/python-uv/limitations.png new file mode 100644 index 000000000..689db1ee7 Binary files /dev/null and b/blog/assets/images/python-uv/limitations.png differ