When trying to install new versions volta hangs indefinitely and uses all available network bandwidth

[effervescentia]

Websites will not load and I have not waited long enough for the command to ever resolve or error (I've waited over 10 minutes)

I was able to solve this temporarily for node downloads by configuring the hooks.json file with a mirror

{
    "node": {
        "index": {
            "template": "https://mirrors.dotsrc.org/nodejs/release/index.json"
        },
        "distro": {
            "template": "https://mirrors.dotsrc.org/nodejs/release/v{{version}}/{{filename}}"
        }
    }
}

this is also happening when trying to install new versions of yarn
I first encountered this close to 2 or 3 weeks ago and it has not yet been resolved
it doesn't looks like a new version of volta has been released, but perhaps some certificates have expired?

if I open Activity Monitor and quit volta-shim my website loads immediately

OS: MacOS 14.4.1

[effervescentia]

as a follow up I wasn't able to find a relevant mirror for yarn but I followed this comment and used the releases from github as distro template

{
    "node": {
        "index": {
            "template": "https://mirrors.dotsrc.org/nodejs/release/index.json"
        },
        "distro": {
            "template": "https://mirrors.dotsrc.org/nodejs/release/v{{version}}/{{filename}}"
        }
    },
    "yarn": {
        "distro": {
            "template": "https://raw.githubusercontent.com/yarnpkg/berry/%40yarnpkg/cli/{{version}}/packages/yarnpkg-cli/bin/yarn.js"
        }
    }
}

[xavidop]

this happens to me as well!

[edison-vflow]

I am also experiencing this issue
@Volta could you please help resolve

[charlespierce]

This is extremely odd, thank you all for reporting! Is this still happening and consistent? And does it happen trying to access the paths directly via curl or the browser (e.g. https://nodejs.org/dist/index.json)?

[gillyb]

Happening to me too..
Trying to reach that url from the browser works for me.

[effervescentia]

this is definitely still happening, and I don't have any issues curling that URL
It's also happening with yarn releases, so nothing specific to the NodeJS download servers

[effervescentia]

@charlespierce anything else I can do to help find the source of the issue?

[JamesPHoughton]

I'm seeing this too, using volta 1.1.1 on macOS 14.5, the verbose output is:

volta install node --verbose              
[verbose] No custom hooks found
[verbose] Found valid cache of Node version index
[verbose] Found newest LTS node version (20.14.0) from https://nodejs.org/dist/index.json
[verbose] Acquiring lock on Volta directory: /Users/me/.volta/volta.lock
[verbose] Downloading [email protected] from https://nodejs.org/dist/v20.14.0/node-v20.14.0-darwin-arm64.tar.gz

This is all that is output before the process hangs.

There doesn't seem to be excessive CPU usage:

...% top -stats pid,command,cpu | grep -i volta
26382  volta            0.0 
26382  volta            0.0 
29336  volta            0.0 
29336  volta            19.3
29336  volta            0.1 
29336  volta            0.0 
29336  volta            0.0 
29336  volta            0.0 

And network traffic is pretty constrained - nettop doesn't show any existing outbound connections

...% nettop | grep -i volta
                                                                                     interface         state        bytes_in       bytes_out   rx_dupe    rx_ooo     re-tx   rtt_avg   rcvsize    tx_win  tc_class    tc_mgt   cc_algo P C R W arch
volta.30865                                                                                                          675 KiB        1033 B      21 KiB     0 B       0 B
   tcp4 127.0.0.1:55871<->localhost:10011                                                  lo0   Established         675 KiB         656 B      21 KiB     0 B       0 B     1.22 ms   630 KiB   398 KiB        BE         -     cubic - - - -   so
   tcp4 127.0.0.1:55873<->localhost:10011                                                  lo0   Established           0 B           377 B       0 B       0 B       0 B     1.22 ms   398 KiB   398 KiB        BE         -     cubic - - - -   so

(but they could have come and gone before i got this snapshot)

I have the same behavior when used for installing as part of a node package.json:

{
 "volta": {
    "node": "20.14.0",
    "npm": "10.7.0"
  }
}

but here the hang eventually resolves with the error:

Volta error: Could not unpack Node v20.14.0
Please ensure the correct version is specified.
Error cause: failed to unpack `/Users/me/Library/Application Support/empirica/volta/tmp/.tmp170xQy/node-v20.14.0-darwin-arm64/lib/node_modules/npm/node_modules/read-package-json-fast/lib/index.js`

Error cause: failed to unpack `/Users/me/Library/Application Support/empirica/volta/tmp/.tmp170xQy/node-v20.14.0-darwin-arm64/lib/node_modules/npm/node_modules/read-package-json-fast/lib/index.js`

Error cause: failed to unpack `node-v20.14.0-darwin-arm64/lib/node_modules/npm/node_modules/read-package-json-fast/lib/index.js` into `/Users/me/Library/Application Support/empirica/volta/tmp/.tmp170xQy/node-v20.14.0-darwin-arm64/lib/node_modules/npm/node_modules/read-package-json-fast/lib/index.js`

Error cause: corrupt deflate stream
...

[DecathectZero]

still getting this behavior, bump

[effervescentia]

@charlespierce is there anything we can do to help you look into this? It's pretty disruptive to our development since it hangs forever without an error message and kills network traffic

[chriskrycho]

Sorry, just saw this (have been busy on other stuff!). From the error messages @JamesPHoughton shared and the description of what people are seeing, it looks like that’s probably coming from some interaction between a bad download and attempting to unpack the result of that.

For others on the Volta team who are spelunking/debugging (or users who want to help), the relevant chunks of code are:

volta/crates/volta-core/src/tool/node/fetch.rs

Lines 180 to 184 in fd9cc3f

	debug!("Downloading {} from {}", tool_version("node", version), url);
	archive::fetch_native(url, staging_path).with_context(download_tool_error(
	tool::Spec::Node(VersionSpec::Exact(version.clone())),
	url,
	))

volta/crates/volta-core/src/tool/node/fetch.rs

Lines 67 to 72 in fd9cc3f

	let archive = fetch_remote_distro(version, &remote_url, staging.path())?;
	(archive, Some(staging))
	}
	};
	let node_version = unpack_archive(archive, version)?;

volta/crates/volta-core/src/tool/node/fetch.rs

Lines 104 to 111 in fd9cc3f

	archive
	.unpack(temp.path(), &mut |_, read| {
	progress.inc(read as u64);
	})
	.with_context(|| ErrorKind::UnpackArchiveError {
	tool: "Node".into(),
	version: version_string.clone(),
	})?;

The relevant code paths haven’t been touched in 5 years (there are a couple Clippy fixes that are unreleased, but that’s it).

I suspect something is happening upstream in archive, given that Volta prints a debug message right before calling archive::fetch_native, you all see a very long timeout, and then the next thing that happens after that very long delay is attempting to unpack the resulting archive. Next step would be to go see whether there is an upstream issue about it, and if so if there is a fix! investigating our own code—I totally thought this was upstream!

volta/crates/archive/src/tarball.rs

Lines 37 to 78 in fd9cc3f

	impl Tarball {
	/// Loads a tarball from the specified file.
	pub fn load(mut source: File) -> Result<Box<dyn Archive>, ArchiveError> {
	let uncompressed_size = load_uncompressed_size(&mut source);
	let compressed_size = source.metadata()?.len();
	Ok(Box::new(Tarball {
	uncompressed_size,
	compressed_size,
	data: Box::new(source),
	origin: Origin::Local,
	}))
	}
	/// Initiate fetching of a tarball from the given URL, returning a
	/// tarball that can be streamed (and that tees its data to a local
	/// file as it streams).
	pub fn fetch(url: &str, cache_file: &Path) -> Result<Box<dyn Archive>, ArchiveError> {
	let (status, headers, response) = attohttpc::get(url).send()?.split();
	if !status.is_success() {
	return Err(ArchiveError::HttpError(status));
	}
	let compressed_size = content_length(&headers)?;
	let uncompressed_size = if accepts_byte_ranges(&headers) {
	fetch_uncompressed_size(url, compressed_size)
	} else {
	None
	};
	ensure_containing_dir_exists(&cache_file)?;
	let file = File::create(cache_file)?;
	let data = Box::new(TeeReader::new(response, file));
	Ok(Box::new(Tarball {
	uncompressed_size,
	compressed_size,
	data,
	origin: Origin::Remote,
	}))
	}
	}

I will keep spelunking if and as I have time, but also welcome other folks digging in a bit.

[chriskrycho]

For anyone following on the thread, and per my edit (which you would not have gotten notified about), this is all in our own code; there is no upstream. 🤦🏻‍♂️

A good next step might be for us to get a build up that folks can run to diagnose, with some additional tracing in Volta’s internal archive crate.

[chriskrycho]

Request for more info from folks on the thread who are hitting this: what operating system are you using? We have at least one confirmed Mac user, but I’d like to know for the rest as part of debugging!