Skip to content

Commit

Permalink
Merge branch 'main' into gtrevi/reg-api
Browse files Browse the repository at this point in the history
  • Loading branch information
gtrevi authored Jul 31, 2023
2 parents a391fa2 + 021c4d9 commit 6d4ec80
Show file tree
Hide file tree
Showing 36 changed files with 3,196 additions and 2,582 deletions.
43 changes: 43 additions & 0 deletions .github/workflows/release-scheduler.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# Copyright (c) Microsoft Corporation
# SPDX-License-Identifier: MIT

# This workflow is triggered on the first day of every month, and creates an issue for releasing eBPF for Windows.

name: Release scheduler
on:
schedule:
- cron: 0 0 1 * *

jobs:
create_release_issue:
name: Create release task
runs-on: ubuntu-latest
permissions:
issues: write
steps:
- name: Create release task
uses: actions/github-script@d7906e4ad0b1822421a7e6a35d5ca353c962f410
env:
TITLE: Scheduled eBPF release is due
LABELS: release
BODY: |
This is an automated reminder that a new release for eBPF is scheduled to be created:
1. Follow instructions in [ReleaseProcess.md](https://github.com/microsoft/ebpf-for-windows/blob/main/docs/ReleaseProcess.md).
1. Sync the internal `mscodehub` and follow the internal release instructions in `\.internal\docs\ReleaseEbpfRedistProcess.md`.
with:
script: |
const owner = process.env.GITHUB_REPOSITORY.split('/')[0]
const repo = process.env.GITHUB_REPOSITORY.split('/')[1]
const body = process.env.BODY;
const title = process.env.TITLE;
const labels = process.env.LABELS;
const label_array = labels ? labels.split(',') : [];
console.log(`Creating issue ${title}`);
await github.rest.issues.create({
owner: owner,
repo: repo,
title: title,
body: body,
labels: label_array,
});
2 changes: 1 addition & 1 deletion .github/workflows/reusable-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -385,7 +385,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Harden Runner
uses: step-security/harden-runner@55d479fb1c5bcad5a4f9099a5d9f37c8857b2845 # v2.4.1
uses: step-security/harden-runner@cba0d00b1fc9a034e1e642ea0f1103c282990604 # v2.5.0
with:
disable-sudo: true
egress-policy: block
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/scorecards-analysis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ jobs:

steps:
- name: Harden Runner
uses: step-security/harden-runner@55d479fb1c5bcad5a4f9099a5d9f37c8857b2845 # v2.4.1
uses: step-security/harden-runner@cba0d00b1fc9a034e1e642ea0f1103c282990604 # v2.5.0
with:
disable-sudo: true
egress-policy: block
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/update-docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ jobs:
steps:

- name: Harden Runner
uses: step-security/harden-runner@55d479fb1c5bcad5a4f9099a5d9f37c8857b2845
uses: step-security/harden-runner@cba0d00b1fc9a034e1e642ea0f1103c282990604
with:
egress-policy: block
allowed-endpoints: >
Expand Down
24 changes: 10 additions & 14 deletions docs/SelfHostedRunnerSetup.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
# Setup instructions for self-hosted runners

The CI/CD tests for `eBPF for Windows` requires installing kernel drivers, that are not supported in Github-hosted runners.
That is why self-host runners are needed to run those tests. The `driver` job in the `CI/CD` Github workflow (`cicd.yml`) runs on self-host runners that use Hyper-V VMs to deploy the eBPF components and run the CI/CD tests on. Using Hyper-V VMs enable the Github workflow to start from a clean state every time the test runs by restoring the VMs to a "baseline" snapshot.
This document discusses the steps to set up such a self-hosted actions-runner that can run the workflow for CI/CD tests on a fork of the eBPF for Windows repo.
Some CI/CD tests for `eBPF for Windows` require installing kernel drivers, but the Github-hosted runners cannot be used to collect kernel-mode dumps should a crash occur. Self-hosted runners allow us to use nested VMs and collect KM dumps. We use them to run the `driver` job in the `CI/CD` Github workflow (`cicd.yml`). On each test run, we restore the VM to a saved checkpoint for a clean run.
Since these runners only run jobs for PRs in the official `eBPF for Windows` repo, you can follow the steps below to set one up for your fork.

1) Install Windows Server 2019 - build 17763.
1) [Windows Server 2019 Azure VM](https://portal.azure.com/#create/Microsoft.WindowsServer2019Datacenter-ARM)
Expand All @@ -17,21 +16,18 @@ This document discusses the steps to set up such a self-hosted actions-runner th
`NetworkService` by default. However, the `Kernel_Test_VM` workflow performs operations on a test VM that requires
administrator privilege. So, the credentials of an account with administrator privilege must be supplied in
`windowslogonaccount` and `windowslogonpassword` parameters.
6) Follow the [VM Installation Instructions](vm-setup.md) to set up **two test VMs** and perform one-time setup steps. Then create a snapshot named **baseline** for each of the VMs.
7) Connect the two test VMs.
1) Create a new VMSwitch instance: `New-VMSwitch -Name <VMSwitch Name> -SwitchType Private`
2) Add two VM NICs on the first VM and one VM NIC on the other VM. Connect the NICs to the private VMSwitch: `Add-VMNetworkAdapter -VMName <VMName> -SwitchName <VMSwitch Name>`
8) Edit test configuration JSON files.
1) Edit `test_execution.json` file. Add the name of the one of the VMs in `BasicTest` section. Add the names of both the VMs in `MultiVMTest` section along with the "Interfaces" section that must contain information about the network interfaces created in step 7.2 Above. This section must contain the interface alias, and IPv4 and IPv6 addresses. The CICD test automation would apply these IP addresses on the NICs of the test VM.
2) Edit `vm_list.json` with the names of the two test VMs.
9) Store the VM administrator and standard user credentials:
6) Follow the [VM Installation Instructions](vm-setup.md) to set up **one test VM** and perform one-time setup steps. Then create a snapshot named **baseline** for the VMs.
7) Edit test configuration JSON files.
1) Edit `test_execution.json` file. Update the name of the VM in `BasicTest` section. Update the name of the VM in `MultiVMTest` section.
2) Edit `vm_list.json` with the name of the test VM.
8) Store the VM administrator and standard user credentials on the runner machine:
1) `Install-Module CredentialManager -force`
2) `New-StoredCredential -Target `**`TEST_VM`**` -Username <VM Administrator> -Password <VM Administrator account password> -Persist LocalMachine`
3) `New-StoredCredential -Target `**`TEST_VM_STANDARD`**` -Username <VM Standard User Name> -Password <VM Standard User account password> -Persist LocalMachine`

10) Modify the environment of the VM as needed. Create new checkpoints using **Hyper-V**. Rename the new checkpoint as `baseline`, and remove the old baseline.
11) Set up Windows Error Reporting [Local Dump Collection](https://docs.microsoft.com/en-us/windows/win32/wer/collecting-user-mode-dumps) on the VMs with the following commands.
9) Modify the environment of the VM as needed. Create a new checkpoint using **Hyper-V**. Rename the new checkpoint to `baseline`, and remove the old baseline.
10) Set up Windows Error Reporting [Local Dump Collection](https://docs.microsoft.com/en-us/windows/win32/wer/collecting-user-mode-dumps) on the VM with the following commands.
```New-Item -Path "HKLM:\SOFTWARE\Microsoft\Windows\Windows Error Reporting\LocalDumps" -ErrorAction SilentlyContinue```
```New-ItemProperty -Path "HKLM:\SOFTWARE\Microsoft\Windows\Windows Error Reporting\LocalDumps" -Name "DumpType" -Value 2 -PropertyType DWord -ErrorAction SilentlyContinue```
```New-ItemProperty -Path "HKLM:\SOFTWARE\Microsoft\Windows\Windows Error Reporting\LocalDumps" -Name "DumpFolder" -Value "c:\dumps" -PropertyType ExpandString -ErrorAction SilentlyContinue -Force```
12) Reboot the runner.
11) Reboot the runner.
90 changes: 89 additions & 1 deletion ebpf-for-windows.sln
Original file line number Diff line number Diff line change
Expand Up @@ -221,7 +221,9 @@ Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "ebpf_store_helper_um", "lib
EndProject
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "usersim", "external\usersim\src\usersim.vcxproj", "{030A7AC6-14DC-45CF-AF34-891057AB1402}"
EndProject
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "libbtf", "external\ebpf-verifier\build\external\libbtf\libbtf\libbtf.vcxproj", "{BC45D39C-F956-3951-BD46-44EAE20EC7C9}"
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "libbtf", "external\ebpf-verifier\build\external\libbtf\libbtf\libbtf.vcxproj", "{249E3FBC-0098-3E3E-80AE-C8F8EF6AB0E5}"
EndProject
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "usersim_dll_skeleton", "external\usersim\usersim_dll_skeleton\usersim_dll_skeleton.vcxproj", "{1937DB41-F3EB-4955-A636-6386DCB394F6}"
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
Expand Down Expand Up @@ -2836,6 +2838,90 @@ Global
{BC45D39C-F956-3951-BD46-44EAE20EC7C9}.RelWithDebInfo|x64.Build.0 = RelWithDebInfo|x64
{BC45D39C-F956-3951-BD46-44EAE20EC7C9}.RelWithDebInfo|x86.ActiveCfg = RelWithDebInfo|x64
{BC45D39C-F956-3951-BD46-44EAE20EC7C9}.RelWithDebInfo|x86.Build.0 = RelWithDebInfo|x64
{249E3FBC-0098-3E3E-80AE-C8F8EF6AB0E5}.Debug|ARM64.ActiveCfg = Debug|x64
{249E3FBC-0098-3E3E-80AE-C8F8EF6AB0E5}.Debug|ARM64.Build.0 = Debug|x64
{249E3FBC-0098-3E3E-80AE-C8F8EF6AB0E5}.Debug|x64.ActiveCfg = Debug|x64
{249E3FBC-0098-3E3E-80AE-C8F8EF6AB0E5}.Debug|x64.Build.0 = Debug|x64
{249E3FBC-0098-3E3E-80AE-C8F8EF6AB0E5}.Debug|x86.ActiveCfg = Debug|x64
{249E3FBC-0098-3E3E-80AE-C8F8EF6AB0E5}.Debug|x86.Build.0 = Debug|x64
{249E3FBC-0098-3E3E-80AE-C8F8EF6AB0E5}.FuzzerDebug|ARM64.ActiveCfg = Debug|x64
{249E3FBC-0098-3E3E-80AE-C8F8EF6AB0E5}.FuzzerDebug|ARM64.Build.0 = Debug|x64
{249E3FBC-0098-3E3E-80AE-C8F8EF6AB0E5}.FuzzerDebug|x64.ActiveCfg = FuzzerDebug|x64
{249E3FBC-0098-3E3E-80AE-C8F8EF6AB0E5}.FuzzerDebug|x64.Build.0 = FuzzerDebug|x64
{249E3FBC-0098-3E3E-80AE-C8F8EF6AB0E5}.FuzzerDebug|x86.ActiveCfg = Debug|x64
{249E3FBC-0098-3E3E-80AE-C8F8EF6AB0E5}.FuzzerDebug|x86.Build.0 = Debug|x64
{249E3FBC-0098-3E3E-80AE-C8F8EF6AB0E5}.MinSizeRel|ARM64.ActiveCfg = MinSizeRel|x64
{249E3FBC-0098-3E3E-80AE-C8F8EF6AB0E5}.MinSizeRel|ARM64.Build.0 = MinSizeRel|x64
{249E3FBC-0098-3E3E-80AE-C8F8EF6AB0E5}.MinSizeRel|x64.ActiveCfg = MinSizeRel|x64
{249E3FBC-0098-3E3E-80AE-C8F8EF6AB0E5}.MinSizeRel|x64.Build.0 = MinSizeRel|x64
{249E3FBC-0098-3E3E-80AE-C8F8EF6AB0E5}.MinSizeRel|x86.ActiveCfg = MinSizeRel|x64
{249E3FBC-0098-3E3E-80AE-C8F8EF6AB0E5}.MinSizeRel|x86.Build.0 = MinSizeRel|x64
{249E3FBC-0098-3E3E-80AE-C8F8EF6AB0E5}.NativeOnlyDebug|ARM64.ActiveCfg = Debug|x64
{249E3FBC-0098-3E3E-80AE-C8F8EF6AB0E5}.NativeOnlyDebug|ARM64.Build.0 = Debug|x64
{249E3FBC-0098-3E3E-80AE-C8F8EF6AB0E5}.NativeOnlyDebug|x64.ActiveCfg = Debug|x64
{249E3FBC-0098-3E3E-80AE-C8F8EF6AB0E5}.NativeOnlyDebug|x64.Build.0 = Debug|x64
{249E3FBC-0098-3E3E-80AE-C8F8EF6AB0E5}.NativeOnlyDebug|x86.ActiveCfg = Debug|x64
{249E3FBC-0098-3E3E-80AE-C8F8EF6AB0E5}.NativeOnlyDebug|x86.Build.0 = Debug|x64
{249E3FBC-0098-3E3E-80AE-C8F8EF6AB0E5}.NativeOnlyRelease|ARM64.ActiveCfg = Release|x64
{249E3FBC-0098-3E3E-80AE-C8F8EF6AB0E5}.NativeOnlyRelease|ARM64.Build.0 = Release|x64
{249E3FBC-0098-3E3E-80AE-C8F8EF6AB0E5}.NativeOnlyRelease|x64.ActiveCfg = Release|x64
{249E3FBC-0098-3E3E-80AE-C8F8EF6AB0E5}.NativeOnlyRelease|x64.Build.0 = Release|x64
{249E3FBC-0098-3E3E-80AE-C8F8EF6AB0E5}.NativeOnlyRelease|x86.ActiveCfg = Release|x64
{249E3FBC-0098-3E3E-80AE-C8F8EF6AB0E5}.NativeOnlyRelease|x86.Build.0 = Release|x64
{249E3FBC-0098-3E3E-80AE-C8F8EF6AB0E5}.Release|ARM64.ActiveCfg = Release|x64
{249E3FBC-0098-3E3E-80AE-C8F8EF6AB0E5}.Release|ARM64.Build.0 = Release|x64
{249E3FBC-0098-3E3E-80AE-C8F8EF6AB0E5}.Release|x64.ActiveCfg = Release|x64
{249E3FBC-0098-3E3E-80AE-C8F8EF6AB0E5}.Release|x64.Build.0 = Release|x64
{249E3FBC-0098-3E3E-80AE-C8F8EF6AB0E5}.Release|x86.ActiveCfg = Release|x64
{249E3FBC-0098-3E3E-80AE-C8F8EF6AB0E5}.Release|x86.Build.0 = Release|x64
{249E3FBC-0098-3E3E-80AE-C8F8EF6AB0E5}.RelWithDebInfo|ARM64.ActiveCfg = RelWithDebInfo|x64
{249E3FBC-0098-3E3E-80AE-C8F8EF6AB0E5}.RelWithDebInfo|ARM64.Build.0 = RelWithDebInfo|x64
{249E3FBC-0098-3E3E-80AE-C8F8EF6AB0E5}.RelWithDebInfo|x64.ActiveCfg = RelWithDebInfo|x64
{249E3FBC-0098-3E3E-80AE-C8F8EF6AB0E5}.RelWithDebInfo|x64.Build.0 = RelWithDebInfo|x64
{249E3FBC-0098-3E3E-80AE-C8F8EF6AB0E5}.RelWithDebInfo|x86.ActiveCfg = RelWithDebInfo|x64
{249E3FBC-0098-3E3E-80AE-C8F8EF6AB0E5}.RelWithDebInfo|x86.Build.0 = RelWithDebInfo|x64
{1937DB41-F3EB-4955-A636-6386DCB394F6}.Debug|ARM64.ActiveCfg = Debug|x64
{1937DB41-F3EB-4955-A636-6386DCB394F6}.Debug|ARM64.Build.0 = Debug|x64
{1937DB41-F3EB-4955-A636-6386DCB394F6}.Debug|x64.ActiveCfg = Debug|x64
{1937DB41-F3EB-4955-A636-6386DCB394F6}.Debug|x64.Build.0 = Debug|x64
{1937DB41-F3EB-4955-A636-6386DCB394F6}.Debug|x86.ActiveCfg = Debug|Win32
{1937DB41-F3EB-4955-A636-6386DCB394F6}.Debug|x86.Build.0 = Debug|Win32
{1937DB41-F3EB-4955-A636-6386DCB394F6}.FuzzerDebug|ARM64.ActiveCfg = Debug|x64
{1937DB41-F3EB-4955-A636-6386DCB394F6}.FuzzerDebug|ARM64.Build.0 = Debug|x64
{1937DB41-F3EB-4955-A636-6386DCB394F6}.FuzzerDebug|x64.ActiveCfg = Debug|x64
{1937DB41-F3EB-4955-A636-6386DCB394F6}.FuzzerDebug|x64.Build.0 = Debug|x64
{1937DB41-F3EB-4955-A636-6386DCB394F6}.FuzzerDebug|x86.ActiveCfg = Debug|Win32
{1937DB41-F3EB-4955-A636-6386DCB394F6}.FuzzerDebug|x86.Build.0 = Debug|Win32
{1937DB41-F3EB-4955-A636-6386DCB394F6}.MinSizeRel|ARM64.ActiveCfg = Debug|x64
{1937DB41-F3EB-4955-A636-6386DCB394F6}.MinSizeRel|ARM64.Build.0 = Debug|x64
{1937DB41-F3EB-4955-A636-6386DCB394F6}.MinSizeRel|x64.ActiveCfg = Debug|x64
{1937DB41-F3EB-4955-A636-6386DCB394F6}.MinSizeRel|x64.Build.0 = Debug|x64
{1937DB41-F3EB-4955-A636-6386DCB394F6}.MinSizeRel|x86.ActiveCfg = Debug|Win32
{1937DB41-F3EB-4955-A636-6386DCB394F6}.MinSizeRel|x86.Build.0 = Debug|Win32
{1937DB41-F3EB-4955-A636-6386DCB394F6}.NativeOnlyDebug|ARM64.ActiveCfg = Debug|x64
{1937DB41-F3EB-4955-A636-6386DCB394F6}.NativeOnlyDebug|ARM64.Build.0 = Debug|x64
{1937DB41-F3EB-4955-A636-6386DCB394F6}.NativeOnlyDebug|x64.ActiveCfg = Debug|x64
{1937DB41-F3EB-4955-A636-6386DCB394F6}.NativeOnlyDebug|x64.Build.0 = Debug|x64
{1937DB41-F3EB-4955-A636-6386DCB394F6}.NativeOnlyDebug|x86.ActiveCfg = Debug|Win32
{1937DB41-F3EB-4955-A636-6386DCB394F6}.NativeOnlyDebug|x86.Build.0 = Debug|Win32
{1937DB41-F3EB-4955-A636-6386DCB394F6}.NativeOnlyRelease|ARM64.ActiveCfg = Release|x64
{1937DB41-F3EB-4955-A636-6386DCB394F6}.NativeOnlyRelease|ARM64.Build.0 = Release|x64
{1937DB41-F3EB-4955-A636-6386DCB394F6}.NativeOnlyRelease|x64.ActiveCfg = Release|x64
{1937DB41-F3EB-4955-A636-6386DCB394F6}.NativeOnlyRelease|x64.Build.0 = Release|x64
{1937DB41-F3EB-4955-A636-6386DCB394F6}.NativeOnlyRelease|x86.ActiveCfg = Release|Win32
{1937DB41-F3EB-4955-A636-6386DCB394F6}.NativeOnlyRelease|x86.Build.0 = Release|Win32
{1937DB41-F3EB-4955-A636-6386DCB394F6}.Release|ARM64.ActiveCfg = Release|x64
{1937DB41-F3EB-4955-A636-6386DCB394F6}.Release|ARM64.Build.0 = Release|x64
{1937DB41-F3EB-4955-A636-6386DCB394F6}.Release|x64.ActiveCfg = Release|x64
{1937DB41-F3EB-4955-A636-6386DCB394F6}.Release|x64.Build.0 = Release|x64
{1937DB41-F3EB-4955-A636-6386DCB394F6}.Release|x86.ActiveCfg = Release|Win32
{1937DB41-F3EB-4955-A636-6386DCB394F6}.Release|x86.Build.0 = Release|Win32
{1937DB41-F3EB-4955-A636-6386DCB394F6}.RelWithDebInfo|ARM64.ActiveCfg = Release|x64
{1937DB41-F3EB-4955-A636-6386DCB394F6}.RelWithDebInfo|ARM64.Build.0 = Release|x64
{1937DB41-F3EB-4955-A636-6386DCB394F6}.RelWithDebInfo|x64.ActiveCfg = Release|x64
{1937DB41-F3EB-4955-A636-6386DCB394F6}.RelWithDebInfo|x64.Build.0 = Release|x64
{1937DB41-F3EB-4955-A636-6386DCB394F6}.RelWithDebInfo|x86.ActiveCfg = Release|Win32
{1937DB41-F3EB-4955-A636-6386DCB394F6}.RelWithDebInfo|x86.Build.0 = Release|Win32
EndGlobalSection
GlobalSection(SolutionProperties) = preSolution
HideSolutionNode = FALSE
Expand Down Expand Up @@ -2905,6 +2991,8 @@ Global
{AA933B9F-B5D8-4AA8-AC18-98FE1A161E8A} = {69CDB6A1-434D-4BC9-9BFF-D12DF7EDBB6B}
{030A7AC6-14DC-45CF-AF34-891057AB1402} = {69CDB6A1-434D-4BC9-9BFF-D12DF7EDBB6B}
{BC45D39C-F956-3951-BD46-44EAE20EC7C9} = {69CDB6A1-434D-4BC9-9BFF-D12DF7EDBB6B}
{249E3FBC-0098-3E3E-80AE-C8F8EF6AB0E5} = {69CDB6A1-434D-4BC9-9BFF-D12DF7EDBB6B}
{1937DB41-F3EB-4955-A636-6386DCB394F6} = {69CDB6A1-434D-4BC9-9BFF-D12DF7EDBB6B}
EndGlobalSection
GlobalSection(ExtensibilityGlobals) = postSolution
SolutionGuid = {3D5F862D-74C6-4357-9F95-0B152E33B7B8}
Expand Down
4 changes: 3 additions & 1 deletion libs/execution_context/ebpf_maps.c
Original file line number Diff line number Diff line change
Expand Up @@ -1795,7 +1795,9 @@ static _Requires_lock_held_(ring_buffer_map->lock) void _ebpf_ring_buffer_map_si
ebpf_ring_buffer_query(
(ebpf_ring_buffer_t*)map->data, &async_query_result->consumer, &async_query_result->producer);
ebpf_list_remove_entry(&context->entry);
ebpf_async_complete(context->async_context, sizeof(*async_query_result), EBPF_SUCCESS);
ebpf_operation_ring_buffer_map_async_query_reply_t* reply =
EBPF_FROM_FIELD(ebpf_operation_ring_buffer_map_async_query_reply_t, async_query_result, async_query_result);
ebpf_async_complete(context->async_context, sizeof(*reply), EBPF_SUCCESS);
ebpf_free(context);
context = NULL;
}
Expand Down
Loading

0 comments on commit 6d4ec80

Please sign in to comment.