top-{n} is not most downloaded as advertised

[jieyouxu]

The bot usage docs say

top-{n}: run the experiment on the n most downloaded crates on crates.io (e.g. top-100).

However, it seems like this is not accurate, in that the crates that get run are not the top-N most downloaded crates on crates.io.

See for instance the run rust-lang/rust#149291 (comment), this looks nothing like https://crates.io/crates?sort=downloads.

[Skgland]

It appears to currently get the top-n crates based on database rowid?

crater/src/crates/lists.rs

Lines 152 to 155 in a3078e1

	CrateSelect::Top(n) => {
	crates.append(&mut RegistryList::get(db)?);
	crates.truncate(*n as usize);
	}

crater/src/crates/lists.rs

Lines 48 to 58 in a3078e1

	fn get(db: &Database) -> Fallible<Vec<Crate>> {
	let crates_results = db.query(
	"SELECT crate FROM crates WHERE list = ?1 ORDER BY rowid;",
	[&Self::NAME],
	|r| r.get::<_, String>(0),
	)?;
	// Turns Vec<Fallible<Crate>> into Fallible<Vec<Crate>>
	crates_results.into_iter().map(|v| v.parse()).collect()
	}
	}

While the reverse dependencies count is computed (krate.2).

crater/src/crates/sources/registry.rs

Lines 225 to 239 in a3078e1

	for record in rdr.deserialize::<DependencyRecord>() {
	let record = record?;
	// Ignore unless this is for a version we're interested in.
	if !versions_selected.contains(&record.version_id) {
	continue;
	}
	let krate = crates
	.get_mut(&record.crate_id)
	.expect("all crates in crates.csv");
	krate.2 += 1;
	}
	info!("computed reverse dependency counts");

It is kept when converting crates to by_name (v.2)

crater/src/crates/sources/registry.rs

Lines 241 to 248 in a3078e1

	let by_name = crates
	.into_iter()
	.filter(|v| v.1 .1.is_some())
	.map(|(_, v)| (v.0, (v.1.unwrap(), v.2)))
	.collect::<HashMap<_, _>>();
	assert_eq!(by_name.len(), versions_selected.len());
	info!("{} unique crate names", by_name.len());

but when converting by_name to list it is dropped (no use of v.1)

crater/src/crates/sources/registry.rs

Lines 250 to 258 in a3078e1

	let mut list = by_name
	.iter()
	.map(|(name, v)| {
	Crate::Registry(RegistryCrate {
	name: name.clone(),
	version: v.0.num.clone(),
	})
	})
	.collect::<Vec<_>>();