Add the ability to remove the Functions container cache from Artifact Registry using the CLI

[fuelkoy]

[REQUIRED] Environment info

firebase-tools: 13.23.1

Platform: win64

[REQUIRED] Test case

Minimum repo: https://github.com/TrustyTechSG/firebase-function-issue

[REQUIRED] Steps to reproduce

1. Deploy the only http function in repo "captureGoogle".
2. Visit http function, get response with google home screenshot.
3. Add one comment so that the function will be deployed again, but don't change anything else, and simply deploy the function again.
4. Visit http function, get Error "Could not find Chrome ", which means the install script from puppeteer not been runned correctly.
5. Update puppeteer ^21.3.4 to ^21.3.5 and deploy the function again, function works again.
6. Do not change anything, simply deploy the function again, function stop working again.

[REQUIRED] Expected behavior

The function should work after redeploy.

[REQUIRED] Actual behavior

The function stopped working after redeploy.

Comment

This is a copy of #6412. The solution(#6412 (comment)) provided by @colerogers does not work if multiple functions are deployed as seemly then cache is not created again even tho it does not exists. And then this shows no npm install was run > same Could not find Chrome error happens. Also the second solution(#6412 (comment)) provided is not good

I actually just remembered that we have an experiments command that will clean up artifacts from artifact registry. To delete the cache, you need to delete the function prior to running it. This should work in automation:

firebase functions:delete <functionName>
firebase experiments:enable deletegcfartifacts
firebase functions:deletegcfartifacts
firebase deploy --only functions

closing out this issue, thanks

I wrote a description there why the provided solution is not good. I will copy the text here to help the reading.

"I think this should be reconsidered to be implemented another way. Deleting a production environment function before deleting the artifacts would resolve to the API endpoint being down in a production environment. This would lead to service not being available for the users. If this would happen every time a function (with puppeteer or similar kind of package) would need to be updated, the service would be unavailable always for a period of time.

Problems:

1. If such a function is not deployed with an empty cache, the deploy will work, but then the function will not work in usage.
2. Production and development environments have separate artifact caches, so this can sneak into the production environment even though development would work. This makes this very error-prone.
3. Reminding the developer always to manually locate to the GCP panel > Artifact registry and find all the functions whose cache needs to be deleted. And to do this always manually on every function deploy is not a good user experience and also does not scale. This approach would bring about other possible problems where, for example, the developer would accidentally delete the function, a folder, or a wrong cache, resulting in the function not working as expected.
   4. Deleting the cache does not ensure npm install is run when deploying multiple functions.

My suggestion:

As you mentioned, you can navigate to the Artifact registry and there find your function cache and delete it with no problems, and then deploy the function buildpacks by running npm install. Then the function works as expected. This solution should be made more straightforward and integrated into the standard workflow. To address this, I propose adding a --no-function-cache flag to the firebase deploy command which would delete the cache for the deployed functions. This would solve all previously mentioned issues. "

[colerogers]

Hi @fuelkoy thanks for opening a new issue. So creating a new command or even just adding a flag to an existing command is actually much more complicated than you'd think since every public surface needs to go through an internal review process. I understand that your workflow would greatly benefit from a firebase built solution, so I'm going to mark this as a feature request. Hopefully others can chime in and +1 this so that we can prioritize this work.

But as a stop gap solution for your automation, you should be able to make a call to this API using the path projects/$PROJECT/locations/$FUNCTION_REGION/repositories/gcf-artifacts/packages/$FUNCTION_NAME to delete the artifact registry images/caches. I know this is not a quick solution, but it's better than navigating to the GCP console manually.

[fuelkoy]

@colerogers Thank you for the information. Could you help me little bit futher with this stop gap please?

I created a script to delete the artifacts as required. Now i made this only for one function. It successfully deletes the package item but not the cache. It outputs Error deleting cache: Error: Failed to delete package: Not Found

The script:

import { exec } from "node:child_process";

/** The Firebase function region. */
const FUNCTION_REGION = "europe-north1";

/** Specific function to clean up artifacts for. */
const TARGET_FUNCTION = "exampleFunction";

/**
 * Executes a shell command and returns the output.
 * @param cmd - The command to execute.
 * @returns Promise resolving with the command's stdout.
 */
const runCommand = (cmd: string): Promise<string> =>
	new Promise((resolve, reject) => {
		exec(cmd, (error, stdout, stderr) => {
			if (error) {
				reject(stderr || error.message);
			} else {
				resolve(stdout.trim());
			}
		});
	});

/**
 * Deletes the artifact registry cache for the target function.
 * @param projectId - The project ID.
 * @returns Promise resolving when the cache is deleted.
 */
const deleteArtifactRegistryCache = async (
	projectId: string,
): Promise<void> => {
	const packageUrl = `https://artifactregistry.googleapis.com/v1/projects/${projectId}/locations/${FUNCTION_REGION}/repositories/gcf-artifacts/packages/${projectId}__europe--north1__${TARGET_FUNCTION}`;
	const cacheUrl = `https://artifactregistry.googleapis.com/v1/projects/${projectId}/locations/${FUNCTION_REGION}/repositories/gcf-artifacts/packages/${projectId}__europe--north1__${TARGET_FUNCTION}/cache`;

	console.log(
		`Deleting artifact registry cache for function ${TARGET_FUNCTION} in project ${projectId}...`,
	);

	try {
		// Get the access token using gcloud auth
		const accessToken = await runCommand("gcloud auth print-access-token");

		console.log(`Attempting to delete package from: ${packageUrl}`);
		const packageResponse = await fetch(packageUrl, {
			method: "DELETE",
			headers: {
				Authorization: `Bearer ${accessToken}`,
			},
		});

		if (!packageResponse.ok) {
			throw new Error(
				`Failed to delete package: ${packageResponse.statusText}`,
			);
		}

		console.log(`Successfully deleted package: ${TARGET_FUNCTION}`);

		// Delete the cache.
		console.log(`Attempting to delete cache from: ${cacheUrl}`);
		const cacheResponse = await fetch(cacheUrl, {
			method: "DELETE",
			headers: {
				Authorization: `Bearer ${accessToken}`,
			},
		});

		if (!cacheResponse.ok) {
			throw new Error(`Failed to delete cache: ${cacheResponse.statusText}`);
		}

		console.log(`Successfully deleted cache for: ${TARGET_FUNCTION}`);
	} catch (error) {
		console.error(`Error deleting cache: ${error}`);
		process.exit(1);
	}
};

/**
 * Extracts the project ID and functions from the command-line arguments.
 * @param args - Command-line arguments passed to the script.
 * @returns An object containing the project ID and list of functions, or null if --only is missing.
 */
const extractArguments = (
	args: string[],
): { projectId: string; functions: string[] | null } => {
	const projectArg = args.find((arg) => arg.startsWith("--project="));
	const onlyArg = args.find((arg) => arg.startsWith("--only="));

	if (!projectArg) {
		throw new Error(
			"Missing --project argument. Please specify the Firebase project.",
		);
	}

	const projectId = projectArg.split("=")[1];
	const functions = onlyArg
		? onlyArg
				.split("=")[1]
				.replace(/functions:/g, "")
				.split(",")
		: null;

	return { projectId, functions };
};

/**
 * Main function to handle the deployment process.
 */
const main = async () => {
	try {
		// Extract project ID and functions to deploy from arguments
		const args = process.argv.slice(2);
		const { projectId, functions } = extractArguments(args);

		console.log(`Project ID: ${projectId}`);
		if (functions) {
			console.log(`Functions to deploy: ${functions.join(", ")}`);
		} else {
			console.log("No specific functions provided. Deploying all functions.");
		}

		// Check if the target function is included in the deployment list or deploying all
		if (!functions || functions.includes(TARGET_FUNCTION)) {
			console.log(
				`Target function "${TARGET_FUNCTION}" detected. Performing cache cleanup...`,
			);
			await deleteArtifactRegistryCache(projectId);
		}

		console.log("Deploying functions...");
		const deployCommand = functions
			? `firebase deploy --only "functions:${functions.join(",functions:")}" --project ${projectId}`
			: `firebase deploy --only functions --project ${projectId}`;

		const deployOutput = await runCommand(deployCommand);

		console.log("Deployment output:");
		console.log(deployOutput);
	} catch (error) {
		console.error(`Error: ${error.message || error}`);
		process.exit(1);
	}
};

main();

[fuelkoy]

Also, which one do I need to delete? There seem to be 2 caches associated with the function, as shown below (with example project names). The naming also seems to follow some weird convention. My function name in this case is just the trpc.

exampleProjectName__europe--north1__trpc
exampleProjectName__europe--north1__trpc/cache
trpc/cache

[fuelkoy]

It appears that simply deleting the function's artifact is sufficient, and there is no need to clear the cache for a successful installation and build.