Allow customising number of lines shown before <n lines omitted>

[gibfahn]

Often in CI I find the important details are lost behind:

```
<72 lines omitted>
```

this makes it hard to debug issues. It would be great to have an environment variable or similar to customise how much is printed.

[epage]

I've been hesitant to include env variables for controlling things, so far. Unsure if I'm more open to that or a feature flag.

[gibfahn]

A feature flag that disables truncating would be great for me if that works for you.

[epage]

I'm also not a big fan of feature flags, so that was a bit of "which evil do I prefer?"

[gibfahn]

Fair enough.

I keep hitting issues where truncated output hides the key details of what went wrong in CI, so I'm also weighing up using a git fork (not ideal) vs not using the crate, vs waiting for this to be an option.

I guess this could also be a set of options on the Command struct, so I could do something like:

let mut cmd = Command::cargo_bin("mycli")?;
cmd.truncate(false);
// ...

[gibfahn]

An alternative would be an option to print the full stdout and stderr to a temp file somewhere, so when the output is small one could just see it inline, but when it is large one could go find it if needed.

[s373r]

I guess this could also be a set of options on the Command struct, so I could do something like:

let mut cmd = Command::cargo_bin("mycli")?;
cmd.truncate(false);
// ...

to be able to configure in this way would be very convenient -- also encountered the need to see the full output

[teythoon]

I also like to see a way to disable truncation. I just pulled that from our CI logs, which is evidence of a problem that we have never seen before, so it must be quite rare:

thread 'integration::sq_cert_lint::sha1_authentication_subkey' panicked at /usr/src/rustc-1.82.0/library/core/src/ops/function.rs:250:5:
Unexpected return code, failed var == 0
└── var: 1

command=`cd "tests/data/cert-lint" && SEQUOIA_CRYPTO_POLICY="/tmp/.tmpdb9WUh/empty-policy.toml" "/target/debug/sq" "--batch" "--home" "/tmp/.tmpdb9WUh/home" "--time" "2024-12-03T14:17:03Z" "cert" "lint" "--time" "20220101" "--cert-file" "-"`
stdin=`<143 lines total>
` ``
-----BEGIN PGP PRIVATE KEY BLOCK-----

xcSYBF+r4VEBDACwtLvhy9Rmgeh/Y6Dd4KLAR/j9iERMtXZHNf+G/SvTUeRbIqcu
w4lqqne2r+mAmCqHSDw8qEP4II4rwT2KQ2lEubd8xpeDPRQO4Nl09nhyZiIYwvUT
WUXEcojppOT9hpM5OkQIlZYUdrtuswdgBjmTGs85/tpy/TCVh+ed2SYxOQtNGtpQ
BO8u0aFGme7Id5W72ctgCLqtuklejo/+0QWe37hHYELNCVW3RcRqs3jEGxR2fIa9
gf4jkWK08srPMoFByNdwmAS9cMreckLctAOKz14Pa/80Wb/mXx3hKC2+k2YFki27
V8cdwcZhfCzwlw2wL/J0YQ3LZ+7Zujtp59k8oHd/FHQYlKwgUFi0IhCWjJJEDVaR
gd6EpR0SLoU8nTed/WbWC276fJtm+K8FpwmHVOSgVJmLQwRj0ktfpSIum8yGyKdT
XjJj3PyVDeyU3OP1zOuN08cK9jtugjBqmt7u34wclzdHSi2cfH4Oz5xJaW2se83h
FXneKml82QxzfEUAEQEAAQAL+QGl9BV2j6aOquJJnnaYSSALGpc8DhUxZkcdxKxV
cF9xjMzmXLHvDNa0kDQ59iU9KJVRsMnXv6d6xuAZwvUdSAMXaBPmkbSdmtmZmfAh
SGwG6FINHaUuoogx4GHUT4qAAxXakdGU1XcBtXB6+pCOMgBdgPYbJIpXvI/k0EBY
cmuZbXa2bQZs+ke153wOQnJR8gIlQbMUUIjYAXSUQxRbUzuO1oNvnoEgfbPOoVR5
drSICphIpibyOTxBY+nt3nRseD1ISvDitDVK/nzlr7P4rqo873cifYnP27x4xObn
eOr2dgsHFe5HDyk6rLpMzMiJS2a25f/Yk44ugBh2o2aZIwtBBKC+Xxo+fUduDSA8
0LJ4BSMRZnUlyuHyib1b1jyOjC4ZKLcraSJhqm0c10r4UQ+JVjLv5kbfns0FwKgr
THVNDdh/bxx6n5YSWSGnVV7jCP3nBkXqvvf3S6JzxtqoU1YH+6+AOu2ouqHZBZA6
UIuilf4JNJr+SmJUEtTmdW7etQYAxIkfFt9koaqkuZDKGDfq4FRLfeQo6DF0+lzK
QbRJg52r4RHg/YpM6USCRt8j85+Vn5o+kzCldUAEcvi2EEsa63t4X+qca+LT2aiC
` ``
<83 lines omitted>
` ``
MRud1BVGRYw3aF+Bf6hblkkcAdhuWs8iTuYqMETSpdi4UTv1f1q+NZ9r85qXHsln
pDXoG+O7autnCSZ6On5PvLvU5BQrXykpTJ6R8ZZCsddaUgHo7MaqZiB6Y3VQm1Xj
xy6X7NHA5aZnQzqZNKe5xbVabCYF6CLmN+l4LaiaPG25Gbym3N4UdMq0dWYTBgDR
xd0XoOpQu97V6YU85yU8oCu6NTn/aYyVdyUTf8jsLPYlaVvU2x3nJc8z9DbdnBnJ
2K7ZN53dRbhPR3Hac7nV30jVPgPGsls37fYE/lraDk45vDl3o4raxDrrBZePd2GT
FPPqzeBQeSSRKa80w1Jdn9HIw1yTAVo+H01hKEEmZoSJViizdZ7pa0omOpxxf90w
DOB5yAkNI1t8THDRAQS3iHTlTF8/6fwym4IQnMpk3XnD6QaA8v+enJu/COTFNjbs
tsLBPgQYAQgAcgWCYc+ZgAkQDkbn/dFpDP9HFAAAAAAAHgAgc2FsdEBub3RhdGlv
bnMuc2VxdW9pYS1wZ3Aub3JnSoXPL/o8D6QBLmv4HySzm4cUNQPIW25DXM9lWZzK
59MCGyAWIQRLb0Ys9RdU119plj8ORuf90WkM/wAAKm4L/iH37rRGVHit7uAgmPTY
r3Nu4S/lzv/IRc/GKES/IXEl17z+3RvQZA6fn5+9Ot3ImdgbZjHkRG5fg/09wW62
tMhNSA2xjKr0p+PlugITuQ1aNEzaoIomBB8fqmhSNX+wHXP1MljEP9oHhP9rsdrc
sZafRfJr6t3vQvisuVrgWNqW4wKn5Cb3uRoCxo56pJWIyZctIaUUdxQjJZu6NMiv
/OqMlcG0cxWuqDyt4H2X+OUV9xWCnKJ7HSEGblUhKAVQ/sQtGj1FPlD+CpANTA7t
YysDhCfBU2kZewIkc1QV2uK2UIxkWbX03+K5FTPk4v2N3bI+khRzKAaxYzGTzBzn
7J7MS9UEl5/ftogeBKub235N8JPhcGLAHYRwgECrfD8EzB2++nV/DTwZxpq2tMUe
0lJUIdV5aZqyQ3NHzgpZt3j+D8QYz3XX1C78tG/zPgUrC+VpIeFeEf1+mJb5C0eV
+1Y3CCxN5857lGaDs4Vs+q1MIdqtfvo6dYBuLVFhIH4iRMLBRAQYAQgAeAWCYc+Z
gAUJA8JnAAkQDkbn/dFpDP9HFAAAAAAAHgAgc2FsdEBub3RhdGlvbnMuc2VxdW9p
YS1wZ3Aub3Jn7rji6WGImXlemqtOA/pI28xy0B3KxmI2NdkKAchpoUECGwwWIQRL
b0Ys9RdU119plj8ORuf90WkM/wAAIDsL/3GkHoPpi2id89AojD5zgiXRJJ7mQSr3
14OsqyTWiYQGmobK0ptglq9I5shTdzslNIwJJlxeEcFU5OuzFtCDu5VrqqCqfX5h
FtDp5Nt2RdIUICqaSIaxZm3eCDcbJr14LKALum4vs4EeCdSTxFt5t4xYRLSKgRHN
2uaBbGv+D3VF3BDm9ySoa79bFFcc0SiDbKw4rxdKpO3fqHC0zBLFYq8xsmu0OgW0
TLYH1l1USt5Wy1dj40D/WK4xtPbKkmyPSeKDS6yFyOGxCQnuLFzLnGAhhN4kB9lu
jyNpdwDo84/DtWnTY6XpLwTvdZity/lyLZugqV4cOtaN4HW6EJqBh2Bw1vC/bJlp
LizvBO2Zr5iETlE0G6wYG2VYwt9bIQcqigdupb2+ONNbLCeVg2wPlmKVFA6IloI7
Ca3gulyNGBk44z3nYmUr9ZjPEjY+mVQPPaCJkpeircJjN6TtyV6YifTgqiKT0OuS
9clvqFoBi+mW5Gw4F50/SfiCh+8zsgt87MLA/AQYAQIAJhYhBEtvRiz1F1TXX2mW
Pw5G5/3RaQz/BQJfq+FRAhsMBQkDwmcAAAoJEA5G5/3RaQz/iEQL/2VrUh62pkx6
tLoHObJgmvHD2WZmpTQH+QjR/LtkvwyD3hS+kRlcN09Mb5/LTG25vz2qG7m2ZYXP
CceWcqnXORTd6OrlaPNU45CLmSv1gQUwDdgwF+gH9QZ+GjdlldBTLFfFpOmpFOWq
n4QySXB+k583lJNCDSBoyR1VuSw5CFr36W/Z4NFgM/5DcZaXw9B77QtVRv9hkqhf
xsRanTR2aBTFo+m1LPN3HWwXJE9iuZ7WZ7vAgwrbypas5/kKxzuFncebtaGTZXeI
9aZM1lknaO8S5UQJNtHOzcHrTedfGwiz3Z+doJkjeTLaf6k4GqkfWDCACCURmLC3
yGM3/CN6TBP/fPozdNo43H/XEHIxHkRc/bT5PnocCrwu51k+zpCAvbsNLn/HEM3B
IqZQm9H4yj8wJs7Lqm85iEMXXywCto1N9XyMUlb4lqHm4pD0OX/NkGFU6BfHYu1E
6OUMVjSAuug71XalrwxdCKrUdGvONlM0YfBvsgJlIITIZB9fw+G24w==
=k0Y8
-----END PGP PRIVATE KEY BLOCK-----
` ``
`
code=1
stdout=""
stderr=```
Certificate 0E46E7FDD1690CFF, key 95722DD2AF2BA4B6 uses a SHA-1-protected binding signature.
Examined 1 certificate.
  0 certificates are invalid and were not linted. (GOOD)
  1 certificate was linted.
  1 of the 1 certificates (100%) has at least one issue. (BAD)
0 of the linted certificates were revoked.
  0 of the 0 certificates has revocation certificates that are weaker than the certificate and should be recreated. (GOOD)
0 of the linted certificates were expired.
1 of the non-revoked linted certificate has at least one non-revoked User ID:
  0 have at least one User ID protected by SHA-1. (GOOD)
  0 have all User IDs protected by SHA-1. (GOOD)
1 of the non-revoked linted certificates has at least one non-revoked, live subkey:
  1 has at least one non-revoked, live subkey with a binding signature that uses SHA-1. (BAD)
0 of the non-revoked linted certificates have at least one non-revoked, live, signing-capable subkey:
  0 certificates have at least one non-revoked, live, signing-capable subkey with a strong binding signature, but a backsig that uses SHA-1. (GOOD)

  Error: 1 certificate have at least one issue
` ``

Sadly, the intermediate value has been truncated, so we cannot reproduce the problem locally.

[s373r]

@teythoon , alternatively as a workaround, you can use this fork (https://github.com/kamu-data/assert_cmd):

[patch.crates-io]
assert_cmd = { git = 'https://github.com/kamu-data/assert_cmd', branch = "deactivate-output-truncation" }

(c) https://github.com/kamu-data/kamu-cli/blob/176a781910530d9e73c23415692ebae927c9a3e8/Cargo.toml#L274

We use it on a large volume of tests and large outputs, long time.

[gibfahn]

What I did was add this method to my test utils file:

/// Extensions to `assert_cmd` functions.
pub trait AssertCmdExt {
    /**
    `assert_cmd`'s assert functions truncate the stdout and stderr.
    This is painful in CI, so add a function to always print them in CI.
    Refs: <https://github.com/assert-rs/assert_cmd/issues/180>
    */
    fn eprint_stdout_stderr(self) -> Self;
}

impl AssertCmdExt for assert_cmd::assert::Assert {
    /**
    `assert_cmd`'s assert functions truncate the stdout and stderr.
    This is painful in CI, so add a function to always print them in CI.

    In general instead of `.success()?` and `.stderr()` we use `.try_success()?` and
    `.try_stderr()?`. This is mostly to remind the author to use this method before
    asserting.

    Refs: <https://github.com/assert-rs/assert_cmd/issues/180>
    */
    fn eprint_stdout_stderr(self) -> Self {
        let output = self.get_output();
        eprintln!(
            "COMMAND STDOUT:\n-------------------\n<<<<\n{stdout}\n>>>>\n\COMMAND \
             STDERR:\n-------------------\n<<<<\n{stderr}\n>>>>",
            stdout = String::from_utf8_lossy(&output.stdout),
            stderr = String::from_utf8_lossy(&output.stderr),
        );
        self
    }
}

Then I use this crate like:

    Command::cargo_bin("my-crate")?
        .arg("run")
        .assert()
        .eprint_stdout_stderr()
        .try_success()?
        .try_stdout(expected_value)?;

Downside is it prints the full and then the truncated output (but also prints the full output with proper colour codes). Upside is I don't need to fork the crate, and then have to maintain said fork.

[djmarcin]

While this extension trait helps, being able to get the full output of the command under test, particularly when an expectation does not match, seems like it should be a core part of the library if not the default behavior.

[teythoon]

I'm about to run into this again, would love to see some progress on this issue.

[teythoon]

I've been hesitant to include env variables for controlling things, so far. Unsure if I'm more open to that or a feature flag.

There is another way that requires neither: never omit any lines.