cargo install --locked is using dev-dependencies

[kushudai]

Problem

I previously opened this issue in addr2line gimli-rs/addr2line#338 and I thought it was a simple matter of updating the lock file as attempted in gimli-rs/addr2line#337
However, @philipc pointed out that the dependency resolution failure was from a dev-dependency which surprised me.

I ran through cargo tree and I verified backtrace-rs is only in dev dependencies.

Steps

Run
cargo install addr2line --locked

Possible Solution(s)

Unclear

Notes

I looked through open issues and it's not clear to me if this is a duplicate of any of these:

• Cargo fails to resolve when adding a git dependency incompatible with a locked crate version but compatible with the Cargo.toml requirement #14916
• Nondeterministic resolution (?) causing dependency to flap in Cargo.lock #13942
• Cargo should skip crates.io check with --locked and all dependencies already resolved #10984
• "all possible versions conflict with previously selected packages" error is not giving the appropriate diagnostic #10189
• cargo install --locked is not really locked #9289

Version

cargo version --verbose
cargo 1.83.0 (5ffbef321 2024-10-29)
release: 1.83.0
commit-hash: 5ffbef3211a8c378857905775a15c5b32a174d3b
commit-date: 2024-10-29
host: x86_64-unknown-linux-gnu
libgit2: 1.8.1 (sys:0.19.0 vendored)
libcurl: 8.9.0-DEV (sys:0.4.74+curl-8.9.0 vendored ssl:OpenSSL/1.1.1w)
ssl: OpenSSL 1.1.1w  11 Sep 2023
os: Ubuntu 22.4.0 (jammy) [64-bit]

[epage]

Oh, interesting.

As mentioned in gimli-rs/addr2line#337, if you download and run cargo build --locked, it works.

My guess as to what is happening is that the indirect dev-dependency on addr2line changes. When its local, it is fine with 0.24.1. When its installing from a registry, addr2line 0.24.2 (package being installed) can be unified with 0.24.1 (transitive dependency) and we attempt to do so but --locked prevents it.

Stepping through the resolve code

cargo/src/cargo/ops/resolve.rs

Line 148 in addcc8c

let (resolve, resolved_with_overrides) = if ws.ignore_lock() {

ws.ignore_lock() is set to false, so we skip this

cargo/src/cargo/ops/resolve.rs

Line 163 in addcc8c

} else if ws.require_optional_deps() {

require_optional_deps is set to false, so we skip this

So that means we then run

cargo/src/cargo/ops/resolve.rs

Lines 207 to 223 in addcc8c

	} else {
	let add_patches = true;
	let resolve = ops::load_pkg_lockfile(ws)?;
	let resolved_with_overrides = resolve_with_previous(
	&mut registry,
	ws,
	cli_features,
	has_dev_units,
	resolve.as_ref(),
	None,
	specs,
	add_patches,
	)?;
	// Skipping `print_lockfile_changes` as there are cases where this prints irrelevant
	// information
	(resolve, resolved_with_overrides)
	};

Looking in my log, I see

   0.342076945s TRACE main:exec:resolve_with_registry:resolve_with_previous:register_previous_locks:register_lock: car
go::core::registry: register_lock: addr2line v0.24.1

which I believe means we got to

cargo/src/cargo/ops/resolve.rs

Line 697 in addcc8c

registry.register_lock(node, deps);

We got there by walking the entire previous resolve and filtering for things we should keep.

The definition of keep is

cargo/src/cargo/ops/resolve.rs

Lines 667 to 671 in addcc8c

	// Alright now that we've got our new, fresh, shiny, and refined `keep`
	// function let's put it to action. Take a look at the previous lock file,
	// filter everything by this callback, and then shove everything else into
	// the registry as a locked dependency.
	let keep = |id: &PackageId| keep(id) && !avoid_locking.contains(id);

We'll not worry about that second definition of keep atm. We'll instead focus on avoid_locking.

For our purposes, avoid_locking is populated at

cargo/src/cargo/ops/resolve.rs

Line 643 in addcc8c

add_deps(resolve, id, &mut avoid_locking);

It seems like we should avoid locking dev-dependencies but we have this preventing us from adding them to avoid_locking

cargo/src/cargo/ops/resolve.rs

Lines 613 to 616 in addcc8c

	// If dev-dependencies aren't being resolved, skip them.
	if !dep.is_transitive() && !dev_deps {
	continue;
	}

@Eh2406 any thoughts on what is going on here?