Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add dataOps pester tests #2391

Merged
merged 29 commits into from
Feb 11, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
9501adc
Add Azure CLI command to set subscription ID
sebassem Dec 19, 2023
e90c4f5
Add subscription ID retrieval and set active subscription
sebassem Dec 19, 2023
726f3ce
Update Arc version to 1.18.0
sebassem Dec 19, 2023
cb2eb4e
Merge branch 'microsoft:arcbox_3.0' into arcbox_3.0
sebassem Dec 25, 2023
47a0fe1
Update Arc version to 1.25.0
sebassem Dec 25, 2023
2de1765
Merge pull request #2 from microsoft/arcbox_3.0
sebassem Jan 1, 2024
f8a3f20
Update PowerShell version to latest release
sebassem Jan 8, 2024
197dba3
Merge pull request #6 from microsoft/arcbox_3.0
sebassem Jan 14, 2024
986130b
Add tests to verify infrastructure and display results on wallpaper u…
sebassem Jan 15, 2024
6ed039a
Remove unnecessary 'az account set' command
sebassem Jan 15, 2024
58de4c8
Add custom location and data controller variables
sebassem Jan 17, 2024
f71737d
Update DataOpsLogonScript.ps1 to use variables for cluster name and d…
sebassem Jan 17, 2024
3de5ccb
Refactor test script invocation in DataOpsLogonScript.ps1
sebassem Jan 18, 2024
ec62467
Add OpenSSL Light package using WinGet
sebassem Jan 18, 2024
39840da
Update Azure CLI commands and remove unused code
sebassem Jan 23, 2024
7bf04d3
Remove unnecessary blank line and print Azure CLI version
sebassem Jan 23, 2024
5c84555
Merge pull request #7 from microsoft/arcbox_3.0
sebassem Jan 23, 2024
bf841b0
Update Azure CLI versions and set subscription ID
sebassem Jan 23, 2024
b1a4c78
Add OpenSSL to PATH environment variable
sebassem Jan 23, 2024
fdc7f54
Fix Azure subscription issue
sebassem Jan 23, 2024
6016801
Merge branch 'microsoft:arcbox_3.0' into arcbox_3.0
sebassem Jan 24, 2024
ca5579c
Refactor test execution in DataOpsLogonScript.ps1
sebassem Jan 28, 2024
ed502e4
Merge pull request #8 from microsoft/arcbox_3.0
sebassem Jan 28, 2024
3f5a552
Merge branch 'arcbox_3.0' of https://github.com/sebassem/azure_arc in…
sebassem Jan 28, 2024
05fb254
Add Az.CustomLocation module and update OpenSSL path
sebassem Jan 29, 2024
d93f676
Updated SQL instance names and added tests for SQL Managed Instances
sebassem Jan 30, 2024
c0a0574
Add OpenSSL to path environment variable and update tests
sebassem Jan 30, 2024
c20be12
Update OpenSSL path in DataOpsAppScript.ps1
sebassem Jan 30, 2024
c63454d
Add tests for VM and Azure Arc Connected Machine
sebassem Jan 31, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions azure_jumpstart_arcbox/artifacts/Bootstrap.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,7 @@ Resize-Partition -DriveLetter C -Size $(Get-PartitionSupportedSize -DriveLetter
Install-PackageProvider -Name NuGet -MinimumVersion 2.8.5.201 -Force

Install-Module -Name Microsoft.PowerShell.PSResourceGet -Force
$modules = @("Az", "Az.ConnectedMachine", "Azure.Arc.Jumpstart.Common", "Posh-SSH", "Pester")
$modules = @("Az", "Az.ConnectedMachine", "Az.ConnectedKubernetes", "Az.CustomLocation", "Azure.Arc.Jumpstart.Common", "Posh-SSH", "Pester")

foreach ($module in $modules) {
Install-PSResource -Name $module -Scope AllUsers -Quiet -AcceptLicense -TrustRepository
Expand Down Expand Up @@ -154,7 +154,6 @@ $url = "https://github.com/PowerShell/PowerShell/releases/latest"
$latestVersion = (Invoke-WebRequest -UseBasicParsing -Uri $url).Content | Select-String -Pattern "v[0-9]+\.[0-9]+\.[0-9]+" | Select-Object -ExpandProperty Matches | Select-Object -ExpandProperty Value
$downloadUrl = "https://github.com/PowerShell/PowerShell/releases/download/$latestVersion/PowerShell-$($latestVersion.Substring(1,5))-win-x64.msi"
Invoke-WebRequest -UseBasicParsing -Uri $downloadUrl -OutFile .\PowerShell7.msi
msiexec.exe /package PowerShell7.msi /quiet ADD_EXPLORER_CONTEXT_MENU_OPENPOWERSHELL=1 ADD_FILE_CONTEXT_MENU_RUNPOWERSHELL=1 ENABLE_PSREMOTING=1 REGISTER_MANIFEST=1 USE_MU=1 ENABLE_MU=1 ADD_PATH=1
Start-Process msiexec.exe -Wait -ArgumentList '/I PowerShell7.msi /quiet ADD_EXPLORER_CONTEXT_MENU_OPENPOWERSHELL=1 ADD_FILE_CONTEXT_MENU_RUNPOWERSHELL=1 ENABLE_PSREMOTING=1 REGISTER_MANIFEST=1 USE_MU=1 ENABLE_MU=1 ADD_PATH=1'
Remove-Item .\PowerShell7.msi

Expand Down
8 changes: 7 additions & 1 deletion azure_jumpstart_arcbox/artifacts/DataOpsAppScript.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -10,14 +10,20 @@ $sqlInstance = "capi"

Start-Transcript -Path $Env:ArcBoxLogsDir\DataOpsAppScript.log

# Add OpenSSL to path environment variable
$openSSL = "C:\Program Files\FireDaemon OpenSSL 3\bin"
$currentPathVariable = [Environment]::GetEnvironmentVariable("PATH", [EnvironmentVariableTarget]::Process)
$newPathVariable = $currentPathVariable + ";" + $openSSL
[Environment]::SetEnvironmentVariable("PATH", $newPathVariable, [EnvironmentVariableTarget]::Process)

Write-Host "Generating a TLS Certificate"
$cert = New-SelfSignedCertificate -DnsName $certdns -KeyAlgorithm RSA -KeyLength 2048 -NotAfter (Get-Date).AddYears(1) -CertStoreLocation "Cert:\CurrentUser\My"
$certPassword = ConvertTo-SecureString -String $password -Force -AsPlainText
Export-PfxCertificate -Cert "cert:\CurrentUser\My\$($cert.Thumbprint)" -FilePath "$Env:TempDir\$CName.pfx" -Password $certPassword
Import-PfxCertificate -FilePath "$Env:TempDir\$CName.pfx" -CertStoreLocation Cert:\LocalMachine\Root -Password $certPassword

openssl pkcs12 -in "$Env:TempDir\$CName.pfx" -nocerts -out "$Env:TempDir\$CName.key" -password pass:$password -passout pass:$password
openssl pkcs12 -in "$Env:TempDir\$CName.pfx" -clcerts -nokeys -out "$Env:TempDir\$CName.crt" -password pass:$password
openssl pkcs12 -in "$Env:TempDir\$CName.pfx" -clcerts -nokeys -out "$Env:TempDir\$CName.crt" -password pass:$password
openssl rsa -in "$Env:TempDir\$CName.key" -out "$Env:TempDir\$CName-dec.key" -passin pass:$password

Write-Header "Creating Ingress Controller"
Expand Down
32 changes: 22 additions & 10 deletions azure_jumpstart_arcbox/artifacts/DataOpsLogonScript.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ $Env:ArcBoxDir = "C:\ArcBox"
$Env:ArcBoxLogsDir = "C:\ArcBox\Logs"
$Env:ArcBoxVMDir = "$Env:ArcBoxDir\Virtual Machines"
$Env:ArcBoxIconDir = "C:\ArcBox\Icons"
$Env:ArcBoxTestsDir = "$Env:ArcBoxDir\Tests"

$clusters = @(
[pscustomobject]@{clusterName = $Env:capiArcDataClusterName; dataController = "$Env:capiArcDataClusterName-dc" ; customLocation = "$Env:capiArcDataClusterName-cl" ; storageClassName = 'managed-premium' ; licenseType = 'LicenseIncluded' ; context = 'capi' ; kubeConfig = "C:\Users\$Env:adminUsername\.kube\config-capi" }
Expand Down Expand Up @@ -32,7 +33,8 @@ Connect-AzAccount -Credential $psCred -TenantId $Env:spnTenantId -ServicePrincip

# Required for CLI commands
Write-Header "Az CLI Login"
az login --service-principal --username $Env:spnClientID --password=$Env:spnClientSecret --tenant $Env:spnTenantId
az login --service-principal --username $Env:spnClientID --password $Env:spnClientSecret --tenant $Env:spnTenantId
az account set -s $Env:subscriptionId

# Register Azure providers
Write-Header "Registering Providers"
Expand Down Expand Up @@ -180,6 +182,9 @@ $clusters | Foreach-Object -ThrottleLimit 5 -Parallel {
$cluster = $_
$context = $cluster.context
$clusterName = $cluster.clusterName
$customLocation = $cluster.customLocation
$dataController = $cluster.dataController

Start-Transcript -Path "$Env:ArcBoxLogsDir\DataController-$context.log"
Write-Host "Deploying arc data services on $clusterName"
Write-Host "`n"
Expand All @@ -191,7 +196,7 @@ $clusters | Foreach-Object -ThrottleLimit 5 -Parallel {
--auto-upgrade false `
--scope cluster `
--release-namespace arc `
--version 1.26.0 `
--version 1.25.0 `
--config Microsoft.CustomLocation.ServiceAccount=sa-bootstrapper

Write-Host "`n"
Expand All @@ -203,17 +208,17 @@ $clusters | Foreach-Object -ThrottleLimit 5 -Parallel {
} while ($podStatus -eq "Nope")
Write-Host "Bootstrapper pod is ready!"

$connectedClusterId = az connectedk8s show --name $cluster.clusterName --resource-group $Env:resourceGroup --query id -o tsv
$extensionId = az k8s-extension show --name arc-data-services --cluster-type connectedClusters --cluster-name $cluster.clusterName --resource-group $Env:resourceGroup --query id -o tsv
$connectedClusterId = az connectedk8s show --name $clusterName --resource-group $Env:resourceGroup --query id -o tsv
$extensionId = az k8s-extension show --name arc-data-services --cluster-type connectedClusters --cluster-name $clusterName --resource-group $Env:resourceGroup --query id -o tsv
Start-Sleep -Seconds 10
az customlocation create --name $cluster.customLocation --resource-group $Env:resourceGroup --namespace arc --host-resource-id $connectedClusterId --cluster-extension-ids $extensionId --kubeconfig $cluster.kubeConfig --only-show-errors
az customlocation create --name $customLocation --resource-group $Env:resourceGroup --namespace arc --host-resource-id $connectedClusterId --cluster-extension-ids $extensionId --kubeconfig $cluster.kubeConfig --only-show-errors

Start-Sleep -Seconds 20

# Deploying the Azure Arc Data Controller

$context = $cluster.context
$customLocationId = $(az customlocation show --name $cluster.customLocation --resource-group $Env:resourceGroup --query id -o tsv)
$customLocationId = $(az customlocation show --name $customLocation --resource-group $Env:resourceGroup --query id -o tsv)
$workspaceId = $(az resource show --resource-group $Env:resourceGroup --name $Env:workspaceName --resource-type "Microsoft.OperationalInsights/workspaces" --query properties.customerId -o tsv)
$workspaceKey = $(az monitor log-analytics workspace get-shared-keys --resource-group $Env:resourceGroup --workspace-name $Env:workspaceName --query primarySharedKey -o tsv)
Copy-Item "$Env:ArcBoxDir\dataController.parameters.json" -Destination "$Env:ArcBoxDir\dataController-$context-stage.parameters.json"
Expand All @@ -234,7 +239,7 @@ $clusters | Foreach-Object -ThrottleLimit 5 -Parallel {

Write-Host "Deploying arc data controller on $clusterName"
Write-Host "`n"
az deployment group create --resource-group $Env:resourceGroup --name $cluster.dataController --template-file "$Env:ArcBoxDir\dataController.json" --parameters "$Env:ArcBoxDir\dataController-$context-stage.parameters.json"
az deployment group create --resource-group $Env:resourceGroup --name $dataController --template-file "$Env:ArcBoxDir\dataController.json" --parameters "$Env:ArcBoxDir\dataController-$context-stage.parameters.json"
Write-Host "`n"

Do {
Expand All @@ -260,13 +265,16 @@ $Env:WORKSPACE_ID = $(az resource show --resource-group $Env:resourceGroup --nam
$Env:WORKSPACE_SHARED_KEY = $(az monitor log-analytics workspace get-shared-keys --resource-group $Env:resourceGroup --workspace-name $Env:workspaceName --query primarySharedKey -o tsv)

foreach($cluster in $clusters){
$Env:MSI_OBJECT_ID = (az k8s-extension show --resource-group $Env:resourceGroup --cluster-name $cluster.clusterName --cluster-type connectedClusters --name arc-data-services | convertFrom-json).identity.principalId
$clusterName = $cluster.clusterName
$dataController = $cluster.dataController
$Env:MSI_OBJECT_ID = (az k8s-extension show --resource-group $Env:resourceGroup --cluster-name $clusterName --cluster-type connectedClusters --name arc-data-services | convertFrom-json).identity.principalId
az role assignment create --assignee $Env:MSI_OBJECT_ID --role 'Monitoring Metrics Publisher' --scope "/subscriptions/$Env:subscriptionId/resourceGroups/$Env:resourceGroup"
az arcdata dc update --name $cluster.dataController --resource-group $Env:resourceGroup --auto-upload-metrics true
az arcdata dc update --name $cluster.dataController --resource-group $Env:resourceGroup --auto-upload-logs true
az arcdata dc update --name $dataController --resource-group $Env:resourceGroup --auto-upload-metrics true
az arcdata dc update --name $dataController --resource-group $Env:resourceGroup --auto-upload-logs true
}

Write-Header "Deploying App"

# Deploy App
& "$Env:ArcBoxDir\DataOpsAppScript.ps1"

Expand Down Expand Up @@ -342,6 +350,10 @@ if ($null -ne (Get-ScheduledTask -TaskName "DataOpsLogonScript" -ErrorAction Sil

Start-Sleep -Seconds 5

Write-Header "Running tests to verify infrastructure"

& "$Env:ArcBoxTestsDir\Invoke-Test.ps1"

# Executing the deployment logs bundle PowerShell script in a new window
Write-Header "Uploading Log Bundle"
Invoke-Expression 'cmd /c start Powershell -Command {
Expand Down
8 changes: 8 additions & 0 deletions azure_jumpstart_arcbox/artifacts/dsc/dataops.dsc.yml
Original file line number Diff line number Diff line change
Expand Up @@ -44,4 +44,12 @@ properties:
settings:
id: Microsoft.AzureDataStudio
source: winget
- resource: Microsoft.WinGet.DSC/WinGetPackage
id: FireDaemon.OpenSSL
directives:
description: Install OpenSSL
settings:
id: FireDaemon.OpenSSL
source: winget

configurationVersion: 0.2.0
1 change: 1 addition & 0 deletions azure_jumpstart_arcbox/artifacts/installCAPI.sh
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@ sudo -u $adminUsername az extension add --name k8s-extension
echo "Log in to Azure"
sudo -u $adminUsername az login --service-principal --username $SPN_CLIENT_ID --password=$SPN_CLIENT_SECRET --tenant $SPN_TENANT_ID
subscriptionId=$(sudo -u $adminUsername az account show --query id --output tsv)
sudo -u $adminUsername az account set -s $subscriptionId
export AZURE_RESOURCE_GROUP=$(sudo -u $adminUsername az resource list --query "[?name=='$stagingStorageAccountName']".[resourceGroup] --resource-type "Microsoft.Storage/storageAccounts" -o tsv)
az -v
echo ""
Expand Down
4 changes: 3 additions & 1 deletion azure_jumpstart_arcbox/artifacts/installK3s.sh
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,9 @@ sudo -u $adminUsername az extension add --name k8s-extension

echo ""
echo "Log in to Azure"
sudo -u $adminUsername az login --service-principal --username $SPN_CLIENT_ID --password=$SPN_CLIENT_SECRET --tenant $SPN_TENANT_ID
sudo -u $adminUsername az login --service-principal --username $SPN_CLIENT_ID --password $SPN_CLIENT_SECRET --tenant $SPN_TENANT_ID
subscriptionId=$(sudo -u $adminUsername az account show --query id --output tsv)
sudo -u $adminUsername az account set -s $subscriptionId
az -v
echo ""

Expand Down
2 changes: 1 addition & 1 deletion azure_jumpstart_arcbox/artifacts/tests/Invoke-Test.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ switch ($env:flavor) {
$tests_failed = $tests_failed + $tests_devops.Failed.Count
}
'DataOps' {
Invoke-Pester -Path "$Env:ArcBoxTestsDir\dataops.tests.ps1" -Output Detailed -Output Detailed -PassThru -OutVariable tests_dataops
Invoke-Pester -Path "$Env:ArcBoxTestsDir\dataops.tests.ps1" -Output Detailed -PassThru -OutVariable tests_dataops
$tests_passed = $tests_passed + $tests_dataops.Passed.Count
$tests_failed = $tests_failed + $tests_dataops.Failed.Count
}
Expand Down
112 changes: 112 additions & 0 deletions azure_jumpstart_arcbox/artifacts/tests/dataops.tests.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@

BeforeDiscovery {

$capiArcDataClusterName = $env:capiArcDataClusterName
$aksArcClusterName = $env:aksArcClusterName
$aksdrArcClusterName = $env:aksdrArcClusterName

$clusters = @($capiArcDataClusterName, $aksArcClusterName, $aksdrArcClusterName)
$customLocations = @("${capiArcDataClusterName}-cl", "${aksArcClusterName}-cl", "${aksdrArcClusterName}-cl")
$dataControllers = @("${capiArcDataClusterName}-dc", "${aksArcClusterName}-dc", "${aksdrArcClusterName}-dc")
$sqlMiInstances = @("capi-sql", "aks-sql", "aks-dr-sql")
$drPartners = @("capi-sql", "aks-dr-sql")
$VMs = @("ArcBox-SQL")

$spnpassword = ConvertTo-SecureString $env:spnClientSecret -AsPlainText -Force
$spncredential = New-Object System.Management.Automation.PSCredential ($env:spnClientId, $spnpassword)

$null = Connect-AzAccount -ServicePrincipal -Credential $spncredential -Tenant $env:spntenantId -Subscription $env:subscriptionId
az config set extension.use_dynamic_install=yes_without_prompt
}

Describe "<cluster>" -ForEach $clusters {
BeforeAll {
$cluster = $_
}
It "Cluster exists" {
$clusterObject = Get-AzConnectedKubernetes -ClusterName $cluster -ResourceGroupName $env:resourceGroup -SubscriptionId $env:subscriptionId
$clusterObject | Should -Not -BeNullOrEmpty
}
It "Azure Arc Connected cluster is connected" {
$connectedCluster = Get-AzConnectedKubernetes -Name $cluster -ResourceGroupName $env:resourceGroup -SubscriptionId $env:subscriptionId
$connectedCluster.ConnectivityStatus | Should -Be "Connected"
}
}

Describe "<customLocation>" -ForEach $customLocations {
BeforeAll {
$customLocation = $_
}
It "Custom Location exists" {
$customLocationObject = Get-AzCustomLocation -Name $customLocation -ResourceGroupName $env:resourceGroup -SubscriptionId $env:subscriptionId
$customLocationObject | Should -Not -BeNullOrEmpty
}
It "Custom Location is connected" {
$customLocationObject = Get-AzCustomLocation -Name $customLocation -ResourceGroupName $env:resourceGroup -SubscriptionId $env:subscriptionId
$customLocationObject.ProvisioningState | Should -Be "Succeeded"
}
}

Describe "<dataController>" -ForEach $dataControllers {
BeforeAll {
$dataController = $_
}
It "Data Controller exists" {
$dataControllerObject = $(az arcdata dc status show --resource-group $env:resourceGroup --name $dataController --query "{name:name,state:properties.k8SRaw.status.state}")
($dataControllerObject | ConvertFrom-Json).Name | Should -Not -BeNullOrEmpty
}
It "Data Controller is connected" {
$dataControllerObject = $(az arcdata dc status show --resource-group $env:resourceGroup --name $dataController --query "{name:name,state:properties.k8SRaw.status.state}")
($dataControllerObject | ConvertFrom-Json).State | Should -Be "Ready"
}
}

Describe "<sqlInstance>" -ForEach $sqlMiInstances {
BeforeAll {
$sqlInstance = $_
}
It "SQL Managed Instance exists" {
$sqlMiInstanceObject = $(az sql mi-arc show --resource-group $env:resourceGroup --name $sqlInstance --query "{name:name,state:properties.k8SRaw.status.state}")
($sqlMiInstanceObject| ConvertFrom-Json).Name | Should -Not -BeNullOrEmpty
}
It "SQL Managed Instance is connected" {
$sqlMiInstanceObject = $(az sql mi-arc show --resource-group $env:resourceGroup --name $sqlInstance --query "{name:name,state:properties.k8SRaw.status.state}")
($sqlMiInstanceObject| ConvertFrom-Json).State | Should -Be "Ready"
}
}

Describe "<drPartner>" -ForEach $drPartners{
BeforeAll {
$drPartner = $_
}
It "DR configuration exists" {
$drConfig = $(az sql instance-failover-group-arc list --resource-group $env:resourceGroup --mi $drPartner)
$drConfig | Should -Not -Be "Found 0 failover group(s)."
}
It "DR configuration is healthy" {
$drConfig = $(az sql mi-arc show --resource-group $env:resourceGroup --name $drPartner --query "{name:name,state:properties.k8SRaw.status.highAvailability.healthState}")
($drConfig| ConvertFrom-Json).state | Should -Be "Ok"
}
}

Describe "<vm>" -ForEach $VMs {
BeforeAll {
$vm = $_
}
It "VM exists" {
$vmobject = Get-VM -Name $vm
$vmobject | Should -Not -BeNullOrEmpty
}
It "VM is running" {
$vmobject = Get-VM -Name $vm
$vmobject.State | Should -Be "Running"
}
It "Azure Arc Connected Machine exists" {
$connectedMachine = Get-AzConnectedMachine -Name $vm -ResourceGroupName $env:resourceGroup -SubscriptionId $env:subscriptionId
$connectedMachine | Should -Not -BeNullOrEmpty
}
It "Azure Arc Connected Machine is connected" {
$connectedMachine = Get-AzConnectedMachine -Name $vm -ResourceGroupName $env:resourceGroup -SubscriptionId $env:subscriptionId
$connectedMachine.Status | Should -Be "Connected"
}
}