Merge pull request #51 from bcgov/AE/azure-documentation-updates

TechDocs (Azure) Updates

docs/azure/best-practices/azure-ai.md

@@ -0,0 +1,41 @@
+# Azure AI Services
+
+Last updated: **November 26, 2024**
+
+Many of the ministry teams are using Azure AI services to build intelligent applications. Artificial Intelligence and Machine Learning are rapidly changing technologies. The following are some recommendations and guidance based on observations and experiences from the ministry teams.
+
+## Region Availability
+
+Although the [Azure AI Foundry (formerly Azure AI Studio)](https://learn.microsoft.com/en-us/azure/ai-studio/what-is-ai-studio) is available in the Canada Azure regions, not all [models](https://azure.microsoft.com/en-us/products/ai-model-catalog?msockid=2274ddfe4fb768de0595c8be4e1d6918#tabs-pill-bar-oc92d8_tab0) or services may be available in the Canada regions. It is recommended to check the availability of the services in the Canada region before starting the development.
+
+The most common Azure AI Services that are used by the ministry teams are:
+
+- Azure OpenAI
+- AI Search
+- Document Intelligence
+
+## Deploying Models
+
+When working with Azure AI services, due to security guardrails that have been put in place (to protect government data from the Internet), you may need to deploy a Virtual Machine within the Azure network to be able to successfully deploy models.
+
+The simplest method to do this, is to deploy an [Azure Bastion](https://learn.microsoft.com/en-us/azure/bastion/quickstart-host-portal) within your virtual network.
+
+> Note: The minimum Bastion SKU required is **Basic**, as the **Developer** tier is not currently available in the Canada regions.
+
+This does require a specific Subnet to be created within the VNet. The subnet name must be **AzureBastionSubnet**. The subnet address range that you specify must be **/26 or larger** (for example, /25 or /24). After adding this subnet to your virtual network, you can deploy Bastion.
+
+Additionally, you will need to create the appropriate ingress and egress Network Security Group (NSG) rules to allow traffic to and from the Azure Bastion service. Please refer to the [Working with NSG access and Azure Bastion](https://learn.microsoft.com/en-us/azure/bastion/bastion-nsg#apply) documentation for specific details.
+
+> Note: The rule priority does not need to match the example below, but the rule configuration should be similar.
+
+[![Azure Bastion - Ingress Rules](../images/azure-bastion-inbound-nsg-rules.png "Azure Bastion - Ingress Rules")](https://learn.microsoft.com/en-us/azure/bastion/media/bastion-nsg/inbound.png#lightbox)
+
+[![Azure Bastion - Egress Rules](../images/azure-bastion-outbound-nsg-rules.png "Azure Bastion - Egress Rules")](https://learn.microsoft.com/en-us/azure/bastion/media/bastion-nsg/outbound.png#lightbox)
+
+## Azure OpenAI and Private DNS
+
+When working with Azure OpenAI, you may need to create a Private Endpoint to resolve the Azure OpenAI service endpoints.
+
+It has been observed in several cases, where the DNS `A-Record` for the Azure OpenAI service is not being created properly in the Private DNS Zone. This can cause issues with the service not being able to resolve the endpoint.
+
+If you encounter this issue, please open a [support ticket](../../welcome/support.md) with the Public Cloud Platform support team to investigate and resolve the issue.

docs/azure/best-practices/be-mindful.md

@@ -1,5 +1,7 @@
 # Be mindful
 
+Last updated: **November 20, 2024**
+
 The following are some things to be aware of when working within the Azure Landing Zone.
 
 ## Virtual Network (VNet) integration
@@ -10,17 +12,25 @@ As a best practice for using Azure App Services with VNet integration, if you pl
 
 ## Private Endpoints and DNS
 
-As a security requirement, some Azure services (ie. Databases, Key Vaults, etc.) have been restricted to private-only connectivity. This means during deployment, you will need to include the creation of a [Private Endpoint](https://learn.microsoft.com/en-us/azure/private-link/private-endpoint-overview) for this service.
+As a security requirement, some Azure PaaS services (ie. Databases, Key Vaults, etc.) have been restricted to private-only connectivity. This means during deployment, you will need to include the creation of a [Private Endpoint](https://learn.microsoft.com/en-us/azure/private-link/private-endpoint-overview) for this service.
 
-As part of creating the Private Endpoint, you will be asked about **Private DNS Integration**. The Azure portal defaults the "**Integrate with private DNS zone**" option to "**Yes**". However, we have the Azure Landing Zones already configured with custom Private DNS Zones, so you should select "**No**" for this option.
+As part of creating the Private Endpoint, you will be asked about **Private DNS Integration**. The Azure portal defaults the "**Integrate with private DNS zone**" option to "**Yes**". However, we have the Azure Landing Zones already configured with centralized custom Private DNS Zones, so you should select "**No**" for this option.
 
 ![Private Endpoint - Private DNS Integration](../images/private-endpoints-dns.png "Private Endpoint - Private DNS Integration")
 
 Once your resource is deployed, a DNS `A-record` will be automatically created in the custom Private DNS Zone in approximately **10 minutes**, pointing to the private IP address of the resource. This will allow you to access the resource using the custom DNS name within the private network.
 
-However, since the endpoint is private-only, you will not be able to access the resource from outside the VNet. To access and work with these specific resources, you need to use either [Azure Bastion](https://learn.microsoft.com/en-us/azure/bastion/bastion-overview) or [Azure Virtual Desktop (AVD)](https://learn.microsoft.com/en-us/azure/virtual-desktop/overview) from within the VNet.
+However, since the endpoint is private-only, you will not be able to access the resource from outside the VNet. To access and work with these specific resources, you need to use either [Azure Bastion](https://learn.microsoft.com/en-us/azure/bastion/bastion-overview) or an [Azure Virtual Desktop (AVD)](https://learn.microsoft.com/en-us/azure/virtual-desktop/overview) from within the VNet.
+
+In the future, once [Express Route](../upcoming-features/express-route.md) is available, you will also be able to access these resources from the on-premises network or through a VPN.
+
+## Custom DNS Zones
 
-In the future, once [Express Route](../upcoming-features/express-route.md) is available, you will also be able to access these resources from the on-premises network.
+In some scenarios, you may have a need to create a custom DNS Zone. Generally, this is not recommended, as the Azure Landing Zones are already configured with centralized custom Private DNS Zones for the Azure services. However, when working with third-party services (ie. Confluent Cloud), we might not have a Private DNS Zone for the specific service.
+
+If this is your scenario, please submit a [Public Cloud Support request](https://citz-do.atlassian.net/servicedesk/customer/portal/3), so that the Public Cloud team can work with you to create and attach the custom DNS Zone to the central Private DNS Resolver.
+
+> Note: Attaching your custom Private DNS Zone to your Virtual Network (VNet) will not work, as all DNS queries are routed through the central Private DNS Resolver.
 
 ## Using Terraform to create Subnets
 
@@ -30,6 +40,8 @@ The Azure Landing Zones have an Azure Policy implemented that requires every sub
 
 Therefore, instead of using the `azurerm_subnet` resource to create subnets, you must use the `azapi_update_resource` resource from the [AzAPI Terraform Provider](https://registry.terraform.io/providers/Azure/azapi/latest/docs). This resource allows you to create subnets with an associated NSG in a single step.
 
+> Note: You need to use the `azapi_update_resource` resource, because you are updating an existing Virtual Network (VNet) with a new subnet (and associated Network Security Group).
+
 **Example code:**
 
 ```hcl
@@ -81,8 +93,18 @@ If you encounter issues when trying to delete a resource you've created (such as
 
 3. **Perform your operation**: Once the lock is removed, you should be able to delete your resource as needed.
 
-4. **Be aware of automation**: Our automation systems will periodically reapply these locks to ensure ongoing protection. If you need the lock to remain off for an extended period, please contact the Cloud Pathfinder team.
+4. **Be aware of automation**: Our automation systems will periodically reapply these locks to ensure ongoing protection. If you need the lock to remain off for an extended period, please contact the Public Cloud team.
 
 5. **Best practice**: After completing your task, if the automation hasn't yet reapplied the lock, consider manually reapplying it to maintain security.
 
 Remember, these locks are in place for good reason. Always double-check that you're deleting the correct resources and understand the implications before removing any locks.
+
+## Azure Control-Plane vs Data-Plane access differences
+
+When working with Azure services, it's important to understand the differences between [control-plane](https://learn.microsoft.com/en-us/azure/azure-resource-manager/management/control-plane-and-data-plane#control-plane) and [data-plane](https://learn.microsoft.com/en-us/azure/azure-resource-manager/management/control-plane-and-data-plane#data-plane) access. The control plane is used to manage resources, while the data plane is used to interact with the resources themselves.
+
+Even though someone may have **Owner-level** permissions on a resource, they may not have the necessary permissions to interact with the **data plane**. This is because the data plane permissions are separate from the control plane permissions.
+
+For example, some Azure services and solution patterns may require additional data-level permissions, such as [Storage Blob Data Reader](https://learn.microsoft.com/en-us/azure/role-based-access-control/built-in-roles/storage#storage-blob-data-reader), [Search Index Data Contributor](https://learn.microsoft.com/en-us/azure/role-based-access-control/built-in-roles/ai-machine-learning#search-index-data-contributor), etc.
+
+For a list of built-in roles and their permissions, refer to the [Azure built-in roles](https://learn.microsoft.com/en-us/azure/role-based-access-control/built-in-roles) documentation.

docs/azure/best-practices/ci-cd.md

@@ -1,5 +1,7 @@
 # CI/CD Best practices
 
+Last updated: **November 20, 2024**
+
 ## GitHub Actions
 
 If you are using GitHub Actions for your CI/CD pipeline, consider the following best practices:
@@ -8,7 +10,7 @@ If you are using GitHub Actions for your CI/CD pipeline, consider the following
 
 * [Self-hosted runners](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/about-self-hosted-runners) on Azure are required to access data storage and database services from GitHub Actions. Public access to these services is not supported.
 
-* If using [Terraform](https://www.terraform.io/), be aware of the limitations when [creating Subnets](../best-practices/be-mindful.md#using-terraform-to-create-subnets), and the use of the [AzAPI Terraform Provider](be-mindful.md#azapi-terraform-provider-using-azapi_update_resource)
+* If using [Terraform](https://www.terraform.io/), be aware of the limitations when [creating Subnets](../best-practices/be-mindful.md#using-terraform-to-create-subnets), and the use of the [AzAPI Terraform Provider](be-mindful.md#azapi-terraform-provider-using-azapi_update_resource).
 
 ### Configuring GitHub Action OIDC Authentication to Azure
 
@@ -20,10 +22,13 @@ Here's a quick summary on how to set it up:
 
 1. The GitHub Identity Provider has already been configured in the Azure subscriptions in your Project Set
 2. In your Azure subscription:
+
   - Create an Entra ID application and a service principal
   - Add federated credentials for the Entra ID application
   - Create GitHub secrets for storing Azure configuration
+
 3. In your GitHub workflows:
+
   - Add permissions settings for the token
   - Use the [azure/login](https://github.com/Azure/login) action to exchange the OIDC token (JWT) for a cloud access token
 

docs/azure/best-practices/governance.md

@@ -0,0 +1,30 @@
+# Governance
+
+Last updated: **November 26, 2024**
+
+The following describes how to manage and govern your Azure environment. While the Public Cloud team provides the initial guardrails and standards for the environment, **each ministry team is responsible** for managing their own resources and ensuring compliance with the standards.
+
+## Security
+
+Within the Azure portal, navigate to [Microsoft Defender for Cloud](https://portal.azure.com/#view/Microsoft_Azure_Security/SecurityMenuBlade/~/0). This service provides a centralized view of the security posture and regulatory compliance of your Azure environment. It also provides recommendations on how to improve your security.
+
+![Defender for Cloud Security Posture](../images/defender-for-cloud-security-posture.png "Defender for Cloud Security Posture")
+
+![Defender for Cloud Regulatory Compliance](../images/defender-for-cloud-regulatory-compliance.png "Defender for Cloud Regulatory Compliance")
+
+## Compliance
+
+Within the Azure portal, navigate to [Azure Policy](https://portal.azure.com/#view/Microsoft_Azure_Policy/PolicyMenuBlade/~/Compliance). This service provides a centralized view of the compliance of your Azure environment. It also provides recommendations on how to improve your compliance.
+
+![Azure Policy Overview](../images/azure-policy-overview.png "Azure Policy Overview")
+
+## Cost
+
+Within the Azure portal, navigate to the [Cost Management - Cost Analysis](https://portal.azure.com/#blade/Microsoft_Azure_CostManagement/Menu/costanalysis) view. This service provides a centralized view of the costs of your Azure environments.
+
+> **IMPORTANT**
+>
+> Please make sure that you select the appropriate **Scope** to view the costs of your environment.
+> ![Azure Cost Management - Scope](../images/azure-cost-management-scope.png "Azure Cost Management - Scope")
+
+For more detail about managing costs, see the **Understand your billing** > [Azure Cost Management](../understanding-your-bill/azure-billing-and-cost-management.md) section.

docs/azure/design-build-deploy/deploy-to-the-azure-landing-zone.md

@@ -1,6 +1,6 @@
 # Deploy to the Azure Landing Zone
 
-Last updated: **September 24, 2024**
+Last updated: **November 20, 2024**
 
 There are multiple ways to deploy your application to the Azure Landing Zone. This document outlines a few examples at a high-level, but it is up to each team to determine the best approach for their application.
 
@@ -29,21 +29,22 @@ The Azure CLI is a command-line tool that allows you to manage your Azure resour
 ```azurecli
 az login
 az account set --subscription "<your-Azure-subscription-ID>"
+az <AZURE RESOURCE TYPE> create ...
 ```
 
-**Note:** There are other command-line tools like [Azure PowerShell](https://learn.microsoft.com/en-us/powershell/azure/), that you can use to manage your Azure resources. You can choose the tool that you are most comfortable with.
+**Note:** There are other command-line tools like [Azure PowerShell](https://learn.microsoft.com/en-us/powershell/azure/), that you can use to create and manage your Azure resources. You can choose the tool that you are most comfortable with.
 
 ## GitHub Actions / Infrastructure-as-Code (IaC)
 
-GitHub Actions is a feature of GitHub that allows you to automate your workflow. Microsoft developed GitHub Actions for Azure and designed them be used with Azure. You can see all of the GitHub Actions for Azure in the [GitHub Marketplace](https://github.com/marketplace?query=Azure&type=actions).
+GitHub Actions is a feature of GitHub that allows you to automate your workflow. Microsoft developed **GitHub Actions for Azure** and designed them be used with Azure. You can see all of the GitHub Actions for Azure in the [GitHub Marketplace](https://github.com/marketplace?query=Azure&type=actions).
 
 ![GitHub Actions for Azure](../images/azure-github-actions-marketplace.png "GitHub Actions for Azure")
 
 **Note:** Be sure to review the [CI/CD Best Practices](../best-practices/ci-cd.md) before implementing GitHub Actions.
 
 ### Infrastructure-as-Code (IaC)
 
-There are multiple ways to deploy your application to the Azure Landing Zone using Infrastructure-as-Code (IaC). You can use tools like [Terraform](https://www.terraform.io/), [Azure Resource Manager (ARM) templates](https://learn.microsoft.com/en-us/azure/azure-resource-manager/templates/), [Bicep](https://learn.microsoft.com/en-us/azure/azure-resource-manager/bicep/overview?tabs=bicep), [Ansible](https://learn.microsoft.com/en-us/azure/developer/ansible/overview), [Chef](https://learn.microsoft.com/en-us/azure/developer/chef/overview), or [Pulumi](https://devblogs.microsoft.com/devops/infrastructure-as-code-azure-python-wpulumi/) to define your infrastructure as code and deploy it to Azure.
+There are multiple ways to deploy your application to the Azure Landing Zone using Infrastructure-as-Code (IaC). You can use tools like [Terraform](https://www.terraform.io/), [Azure Resource Manager (ARM) templates](https://learn.microsoft.com/en-us/azure/azure-resource-manager/templates/), [Bicep](https://learn.microsoft.com/en-us/azure/azure-resource-manager/bicep/overview?tabs=bicep), [Ansible](https://learn.microsoft.com/en-us/azure/developer/ansible/overview), [Chef](https://learn.microsoft.com/en-us/azure/developer/chef/overview), or [Pulumi](https://devblogs.microsoft.com/devops/infrastructure-as-code-azure-python-wpulumi/) to define your Infrastructure-as-Code and deploy it to Azure.
 
 While we don't have a specific example for each tool, we recommend that you choose the tool that you are most comfortable with and that best fits your needs.