Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

resources: update gapbs to use new 24.04 images #46

Open
wants to merge 9 commits into
base: stable
Choose a base branch
from

Conversation

Harshil2107
Copy link
Contributor

No description provided.

@Harshil2107 Harshil2107 marked this pull request as ready for review September 24, 2024 20:42
@erin-le erin-le self-requested a review September 25, 2024 17:16
license: BSD-3-Clause
---

This document provides instructions to create a GAP Benchmark Suite (GAPBS) disk image, which, along with an example script, may be used to run GAPBS within gem5 simulations. The example script uses a pre-built disk-image.

A pre-built disk image, for X86, can be found, gzipped, here: <http://dist.gem5.org/dist/v22-1/images/x86/ubuntu-18-04/gapbs.img.gz>.
A pre-built disk image, for X86, can be found, gzipped, here: <add link>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add link

license: BSD-3-Clause
---

This document provides instructions to create a GAP Benchmark Suite (GAPBS) disk image, which, along with an example script, may be used to run GAPBS within gem5 simulations. The example script uses a pre-built disk-image.

A pre-built disk image, for X86, can be found, gzipped, here: <http://dist.gem5.org/dist/v22-1/images/x86/ubuntu-18-04/gapbs.img.gz>.
A pre-built disk image, for X86, can be found, gzipped, here: <add link>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add link

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If everything looks good then I will upload the disks and add the links.


GAPBS disk image can support both real and synthetic graph inputs. The current pre-built disk image contains only one graph input which includes the New York city road map (with 733K nodes) it can be found: <http://users.diag.uniroma1.it/challenge9/download.shtml>.
This gapbs image uses the prebuilt ubuntu 24.04 image as a base image. The gapbs image also throws the same exit events as the base image.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Elaborate more on exit events in the base disk image and/or add link to base image documentation.

I think you should also say "Look at the Boot Sequences section for information on what exit events will take place", then tell users where they can find configuration scripts with that exit event setup. Since the PR to update the config scripts is still pending, maybe you could point users to the "usage" tab of one of the gem5 resources with the fleshed out exit event handler setup (such as this one: https://resources.gem5.org/resources/x86-ubuntu-24.04-img/usage?database=gem5-resources&version=2.0.0). Or even just put it in a code block somewhere in this document.

Copy link

@mbabaie mbabaie Sep 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with more explanations about two things:

  1. what the path to the programs' executable is in the disk image. This is probably required to setup the command in the run script. I saw in the examples Erin provided below, she has not specified that, and has used them directly as ./cc for this purpose.
  2. number of exit events, their orders, and where they are in the main program.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. I have added the path to the executable, I will also add the links to the workloads on gem5 resources when the changes looks good, and the disks have been uploads. There will be a workload for each executable benchmark in 3 sizes and with two real graphs. Is there any other information that I should add?

  2. The boot sequences section in the readme explains, where the exits will occur. will it be beneficial to mention exact files?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Thanks for adding this. Looks good to me.
  2. That also looks good with the stars marked to see how many exit events must be handled in the runs scripts.

src/gapbs/README.md Outdated Show resolved Hide resolved
src/gapbs/README.md Outdated Show resolved Hide resolved
src/gapbs/README.md Outdated Show resolved Hide resolved
* **--synthetic** refers whether to use a synthetic or a real graph. It accepts a boolean value.
* **--size**, which refers to either the size of a synthetic graph from 1 to 16 nodes, or, a real graph. The real graph included in the pre-built disk-image is `USA-road-d.NY.gr`. Note that `--synthetic True` and `--size USA-road-d.NY.gr` cannot be combined, and, vice versa for real graphs.
This detailed overview provides a foundational understanding of how different boot configurations affect the system's initialization and mode of operation.
By selecting the appropriate parameters, users can customize the boot process for diverse environments, ranging from automated setups to hands-on interactive sessions.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It could be good to add an overview of the benchmarks available on the built disk, or point toward the GAPBS documentation

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will add a section with all the benchmark workloads, when we upload the disk to the cloud and I can push the JSONs to MongoDB.

src/gapbs/README.md Outdated Show resolved Hide resolved
@@ -0,0 +1,10 @@
PACKER_VERSION="1.10.0"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add copyright (?)

src/gapbs/build-x86.sh Outdated Show resolved Hide resolved
src/gapbs/build-x86.sh Outdated Show resolved Hide resolved
dd if=/usr/share/qemu-efi-aarch64/QEMU_EFI.fd of=flash0.img conv=notrunc
cd ..

wget https://storage.googleapis.com/dist.gem5.org/dist/develop/images/arm/ubuntu-24-04/arm-ubuntu-24.04-20240823.gz
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line has a problem when the .gz file already exists. Can you do one of the following?

  1. If the .gz file already exists, wget will download it to .gz.1 or so. Before downloading the .gz file, make sure you remove the old download of .gz file.
  2. If the .gz file already exists, checksum the file. If the checksum does not match, remove the old .gz file and re-download the .gz file.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This also applies to similar wget commands below.

Copy link
Contributor

@erin-le erin-le left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would also be good to add examples of how to set the workload directly into the README. It was something that I struggled with when testing your other disk image refactors. Examples for Arm, RISCV, and X86 are included below:

# for RISCV
board.set_kernel_disk_workload(
    kernel=obtain_resource(
        "riscv-linux-6.6.33-kernel", resource_version="1.0.0"
    ),
    disk_image=DiskImageResource("/path/to/gem5-resources/src/gapbs/riscv-disk-image-ubuntu-24-04/riscv-ubuntu", root_partition="1"),
    bootloader=obtain_resource("riscv-bootloader-opensbi-1.3.1"), 
    readfile_contents="./cc -g 21\n",
)
# for x86
board.set_kernel_disk_workload(
    kernel=obtain_resource(
        "x86-linux-kernel-5.4.0-105-generic", resource_version="1.0.0"
    ),
    disk_image=DiskImageResource("/path/to/gem5-resources/src/gapbs/disk-image-ubuntu-24-04/x86-ubuntu-24-04-gapbs"),
    readfile_contents="./cc -g 23\n",
    kernel_args=[
        "earlyprintk=ttyS0",
        "console=ttyS0",
        "lpj=7999923",
        "root=/dev/sda2",
        "no_systemd=true"
    ]

)
# for arm
disk_img = DiskImageResource("/path/to/gem5-resources/src/gapbs/disk-image-arm-npb/arm-ubuntu", root_partition="2")
board.set_kernel_disk_workload(
        disk_image=disk_img, 
        bootloader=obtain_resource("arm64-bootloader-foundation"), 
        kernel=obtain_resource("arm64-linux-kernel-5.15.36"),
        readfile_contents="./cc -g 21\n",
        )

@mbabaie
Copy link

mbabaie commented Oct 1, 2024

Related to the examples Erin has provided, I think it's good to have a full example of readfile_contents that has everything in it (whether it logs in with sudo, the full path to the binary, etc.)

@Harshil2107
Copy link
Contributor Author

Related to the examples Erin has provided, I think it's good to have a full example of readfile_contents that has everything in it (whether it logs in with sudo, the full path to the binary, etc.)

I am thinking of adding a base example scripts to run the gapbs workloads like we have right now for x86 but for all the ISAs and then point them in the readme. I think those should be good examples? is there anything else specific that I should add?

@mbabaie
Copy link

mbabaie commented Oct 2, 2024

That would be very good and helpful.
Another thing to add is to make sure we remind the users it should be run with sudo in the examples.
I don't have anything else to add at the moment :)

@mbabaie
Copy link

mbabaie commented Oct 2, 2024

This is what I meant (figured it out with trial&error) that worked perfectly:
"echo 12345 | sudo -S ./gapbs/bfs -g 22"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants