Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update to use new version of Kangooroo #90

Open
wants to merge 11 commits into
base: master
Choose a base branch
from

Conversation

cccs-shellyw
Copy link

  • use new Kangooroo with different command argument and result.json format
  • download Kangooroo from its github release
  • update tests to the new result.json format

@cccs-shellyw cccs-shellyw requested a review from gdesmar December 3, 2024 16:36
@cccs-shellyw cccs-shellyw force-pushed the update-kangooroo-version branch from e528b58 to 2f96ac6 Compare December 3, 2024 17:00
@cccs-shellyw cccs-shellyw force-pushed the update-kangooroo-version branch 9 times, most recently from b21cf30 to de239ac Compare December 17, 2024 15:43
@cccs-shellyw cccs-shellyw force-pushed the update-kangooroo-version branch 2 times, most recently from 60044ba to bc3ec1c Compare December 17, 2024 16:00
@cccs-shellyw cccs-shellyw force-pushed the update-kangooroo-version branch from bc3ec1c to c629ef3 Compare December 17, 2024 16:08
Dockerfile Outdated Show resolved Hide resolved
@@ -180,21 +180,21 @@ def execute(self, request: ServiceRequest) -> None:
yaml.dump(kangooroo_config, temp_conf)

kangooroo_args = [
"java",
f"-Xmx{math.floor(self.service_attributes.docker_config.ram_mb*0.75)}m",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had problem without limiting the memory on the java call. This needs to stay.

urldownloader.py Outdated Show resolved Hide resolved
@@ -3,24 +3,37 @@
"drop_file": false,
"score": 0,
"sections": [
{
"auto_collapse": false,
"body": "URL 'https://news.slashdot.org/comments.pl?sid=11253499&cid=55399795' (380369f835d00015f089274107cf114d) was requested but a different URL was fetched (d86e2b62b352d8f9c944980f534f86fe).",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you check all results.json where they have changed? They should not have changed unless you meant to change it like the sample_zip test where you modified the IP address.

@cccs-shellyw cccs-shellyw requested review from cccs-ml and removed request for cccs-ml December 19, 2024 20:41
@@ -35,6 +39,3 @@ RUN pip install --no-cache-dir --user --requirement requirements.txt && rm -rf ~
ARG version=4.0.0.dev1
USER root
RUN sed -i -e "s/\$SERVICE_TAG/$version/g" service_manifest.yml

# Switch to assemblyline user
USER assemblyline
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why was this removed?

USER root

RUN apt update -y && \
apt install -y wget default-jre unzip && \
apt install -y wget wget default-jre unzip ffmpeg build-essential libffi-dev python3 python3-dev python3-pip libfuzzy-dev && \
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You added a duplicate wget

USER root

RUN apt update -y && \
apt install -y wget default-jre unzip && \
apt install -y wget wget default-jre unzip ffmpeg build-essential libffi-dev python3 python3-dev python3-pip libfuzzy-dev && \
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are all of the new libraries really required?

]
if self.no_sandbox:
kangooroo_args.insert(-2, "--no-sandbox")
try:
subprocess.run(kangooroo_args, cwd=KANGOOROO_FOLDER, capture_output=True, timeout=self.request_timeout)
subprocess.run(kangooroo_args, cwd=KANGOOROO_FOLDER,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you use the settings from .vscode/settings.json so that it applies the right formatter?

"fetchResult": {
"response_code": 200,
"connection_success": true,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are those three boolean supposed to be actual booleans or strings? They are stored as strings in other results.json files.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants