gh pr list doesn't write output anywhere if stdout or stderr is piped

[gibfahn]

Describe the bug

When you pipe the stdout or stderr of gh somewhere, it no longer writes useful output to the stderr.

Steps to reproduce the behavior

❯ gh pr list
X No default remote repository has been set for this directory.

please run `gh repo set-default` to select a default remote repository.

❯ gh pr list | cat

❯

Using latest brew gh version:

❯ gh --version
gh version 2.50.0 (2024-05-29)
https://github.com/cli/cli/releases/tag/v2.50.0

Expected vs actual behavior

I expect to see the error on the stderr even when I save the output on the stdout.

[williammartin]

Hey @gibfahn, this took me a moment to track down so I can understand why you would be confused. This isn't a case of the output disappearing as you suspected but actually it's a behavioural difference in the CLI such that we don't treat the second case as an error.

In the first case, where we know that there is interactivity, the CLI bails out saying "we really want to you to choose a default repo for us to target" (see: gh repo set-default --help for what that means). In the second case, we're not sure whether there ever will be interactivity (e.g. in CI) so we select a remote to act as the default. To be honest I don't have the full picture on this since the code was written some 4 years ago but that's what's going on here. This would have been more visible if the remote being chosen had PRs, where yours presumably has none (you can check the specific API requests using GH_DEBUG=api)

I'd have to check whether there are other issues around the decision above and I'm happy to go searching for them if you still have a problem but I'm going to close this for now since the specific issue is addressed. Cheers.

[gibfahn]

Ahh okay, I guess trying to simplify the output didn't fully repro, how about this one?

❯ GH_REPO=gibfahn/brew GH_HOST=github.com gh pr list --state=open --limit=1
no open pull requests in gibfahn/brew

❯ GH_REPO=gibfahn/brew GH_HOST=github.com gh pr list --state=open --limit=1 | cat

❯

I expect to still see the log message when something goes wrong.

For context I'm seeing this in a script where I'm saving the output to a shell variable to work around #575 . When the command doesn't return any results, it's useful to be able to see what went wrong.

GH_DEBUG doesn't cause this to be shown either for what it's worth:

❯ GH_DEBUG=1 GH_REPO=gibfahn/brew GH_HOST=github.com gh pr list '--state=open' '--limit=1' | cat
* Request at 2024-05-31 15:21:01.749336 +0100 BST m=+0.144867168
* Request to https://api.github.com/graphql
* Request took 427.841208ms

❯

[williammartin]

Thank you for the clarification. I'll have to think about this and a comment from a previous maintainer a bit more.

Consider a workflow "find me all the open PRs and assign me as a reviewer". From a list command point of view, we don't know whether zero results is an error. Only you know if it is an error in your domain as the caller of the script.

When hooked up to a terminal, we try to provide a helpful message. I could see us printing something to stderr when there is no TTY but I suspect the reason Mislav didn't suggest that is because it would be a breaking change for any scripts that might be redirecting or consuming stderr as well. We try to be very careful about what we output to the iostreams when there is no TTY attached. Hyrum's Law and all.

GH_DEBUG doesn't cause this to be shown either for what it's worth:

This was just to highlight the repo that was being targeted, to ensure that it was the one you expected given my explanation of repo set-default.

[gibfahn]

The reasoning behind each of these decisions does make sense (thank you for sharing it!), but overall it leads to quite a confusing debugging experience, where my script fails because there was no output, so I try to change the command, and then it starts printing errors, and I change it again, and now it prints different errors, and none of them are the actual thing I need to debug (which ends up being that the upstream for the current branch is configured wrong, so there was no output). This gets more complex when you start using the --json --jq outputs, as there are more things that could have gone wrong (e.g. maybe the JSON format changed).

At the risk of adding more complexity, I would happily take an option here that made gh pr list always print the error message when nothing was found. In general in CI or automation is where I most want to see errors or more info on stderr, as it's where it's hardest to easily rerun the command to debug.

[ntBre]

I came across this issue while searching for a similar problem with no errors involved. I'm just trying to get the output from gh pr list to merge many PRs at once. The command itself runs fine:

> gh pr list -A 'dependabot[bot]' --json headRefName --jq '.[].headRefName'
dependabot/cargo/clap-4.5.6
dependabot/cargo/spectro-80b1dec
dependabot/cargo/serde_json-1.0.117
dependabot/cargo/nalgebra-0.32.5
dependabot/cargo/psqs-d2ea8fe

But trying to pipe it anywhere (| cat, > out 2> err, var=$(...)) produces no output. I'm also on version 2.50 installed from the Arch repos.

In my case, I think the issue is something with the -A flag because this actually works as expected:

> gh pr list --json headRefName --jq '.[].headRefName' | cat
dependabot/cargo/clap-4.5.6
dependabot/cargo/spectro-80b1dec
dependabot/cargo/serde_json-1.0.117
dependabot/cargo/nalgebra-0.32.5
dependabot/cargo/psqs-d2ea8fe

[williammartin]

That's incredibly strange! Are you saying the following command produces no output for you, and without | cat it does?

➜ gh pr list -A 'dependabot[bot]' --json headRefName --jq '.[].headRefName' | cat
dependabot/github_actions/goreleaser/goreleaser-action-6
dependabot/github_actions/actions/attest-build-provenance-1.2.0

[ntBre]

Yes, that's exactly right!

[ntBre]

I just tried gh pr list -A 'dependabot[bot]' | cat in the Emacs shell running bash instead of my normal st running zsh to rule out a specific shell/terminal issue and had the same result. With | cat produces no output, without | cat runs fine.

Woah, but a different repo does work! This is very strange.

[williammartin]

Woah, but a different repo does work! This is very strange.

That's uhhh interesting. Can you see if there's anything interesting with GH_DEBUG=api turned on? I can't even fathom what's going on here.

[ntBre]

I'm not quite sure what I'm looking for, but the GraphQL query in the cat version is

GraphQL query:
fragment pr on PullRequest{number,title,state,url,headRefName,headRepositoryOwner{id,login,...on User{name}},isCrossRepository,isDraft,createdAt}
    query PullRequestSearch(
      $q: String!,
      $limit: Int!,
      $endCursor: String,
    ) {
      search(query: $q, type: ISSUE, first: $limit, after: $endCursor) {
        issueCount
        nodes {
          ...pr
        }
        pageInfo {
          hasNextPage
          endCursor
        }
      }
    }
GraphQL variables: {"limit":30,"q":"author:dependabot[bot] repo:ntBre/rust-pbqff state:open type:pr"}

and the final data is empty:

{
  "data": {
    "search": {
      "issueCount": 0,
      "nodes": [],
      "pageInfo": {
        "hasNextPage": false,
        "endCursor": null
      }
    }
  }
}

whereas the no-cat version has multiple queries, starting with

GraphQL query:
fragment repo on Repository {
    id
    name
    owner { login }
    viewerPermission
    defaultBranchRef {
      name
    }
    isPrivate
  }
  query RepositoryNetwork {
    viewer { login }

    repo_000: repository(owner: "ntBre", name: "rust-pbqff") {
      ...repo
      parent {
        ...repo
      }
    }

  }
GraphQL variables: null

and then the PullRequest query, which returns data in that case. All of the requests are returning HTTP 200.

I can upload the output from the command if that would help, but redirecting it to a file means I can only get the "cat" version. I guess I could install a terminal with better scrolling and paste the other output if needed.

[williammartin]

and then the PullRequest query, which returns data in that case. All of the requests are returning HTTP 200.

Are the query and variables exactly the same? From your first log:

{"limit":30,"q":"author:dependabot[bot] repo:ntBre/rust-pbqff state:open type:pr"}

The full output would be useful. You may be able to force it to output by setting GH_FORCE_TTY=true in your environment.

[ntBre]

Is there anything I should redact from the output before uploading? The token looks like it's already blocked out, and the only other thing I see that is possibly sensitive is the X-Oauth-Client-Id, but I think even that is probably okay (?)

[williammartin]

Nothing sensitive unless there's anything that you want to redact from your own data (repo names, issues, authors, and that kind of thing).

[ntBre]

Awesome, thanks! These are all public repos and the PRs are all from Dependabot (which is why I'm hoping to automate this a bit). Here's the full | cat version:

[git remote -v]
[git config --get-regexp ^remote\..*\.gh-resolved$]
* Request at 2024-06-07 15:28:45.921113009 -0400 EDT m=+0.042989137
* Request to https://api.github.com/graphql
> POST /graphql HTTP/1.1
> Host: api.github.com
> Accept: application/vnd.github.merge-info-preview+json, application/vnd.github.nebula-preview
> Authorization: token ████████████████████
> Content-Length: 589
> Content-Type: application/json; charset=utf-8
> Graphql-Features: merge_queue
> Time-Zone: America/New_York
> User-Agent: GitHub CLI v2.50.0

GraphQL query:
fragment pr on PullRequest{number,title,state,url,headRefName,headRepositoryOwner{id,login,...on User{name}},isCrossRepository,isDraft,createdAt}
    query PullRequestSearch(
      $q: String!,
      $limit: Int!,
      $endCursor: String,
    ) {
      search(query: $q, type: ISSUE, first: $limit, after: $endCursor) {
        issueCount
        nodes {
          ...pr
        }
        pageInfo {
          hasNextPage
          endCursor
        }
      }
    }
GraphQL variables: {"limit":30,"q":"author:dependabot[bot] repo:ntBre/rust-pbqff state:open type:pr"}

< HTTP/2.0 200 OK
< Access-Control-Allow-Origin: *
< Access-Control-Expose-Headers: ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset
< Content-Security-Policy: default-src 'none'
< Content-Type: application/json; charset=utf-8
< Date: Fri, 07 Jun 2024 19:28:46 GMT
< Referrer-Policy: origin-when-cross-origin, strict-origin-when-cross-origin
< Server: GitHub.com
< Strict-Transport-Security: max-age=31536000; includeSubdomains; preload
< Vary: Accept-Encoding, Accept, X-Requested-With
< X-Accepted-Oauth-Scopes: repo
< X-Content-Type-Options: nosniff
< X-Frame-Options: deny
< X-Github-Media-Type: github.v4; param=merge-info-preview.nebula-preview; format=json
< X-Github-Request-Id: 8D40:2CA7EE:1442AD:21996E:66635F6E
< X-Oauth-Client-Id: 178c6fc778ccc68e1d6a
< X-Oauth-Scopes: gist, read:org, repo
< X-Ratelimit-Limit: 5000
< X-Ratelimit-Remaining: 4939
< X-Ratelimit-Reset: 1717791391
< X-Ratelimit-Resource: graphql
< X-Ratelimit-Used: 61
< X-Xss-Protection: 0

{
  "data": {
    "search": {
      "issueCount": 0,
      "nodes": [],
      "pageInfo": {
        "hasNextPage": false,
        "endCursor": null
      }
    }
  }
}

* Request took 1.06807743s

and no cat:

[git remote -v]
[git config --get-regexp ^remote\..*\.gh-resolved$]
* Request at 2024-06-07 15:29:39.297894772 -0400 EDT m=+0.043812634
* Request to https://api.github.com/graphql
> POST /graphql HTTP/1.1
> Host: api.github.com
> Accept: application/vnd.github.merge-info-preview+json, application/vnd.github.nebula-preview
> Authorization: token ████████████████████
> Content-Length: 390
> Content-Type: application/json; charset=utf-8
> Graphql-Features: merge_queue
> Time-Zone: America/New_York
> User-Agent: GitHub CLI v2.50.0

GraphQL query:
fragment repo on Repository {
    id
    name
    owner { login }
    viewerPermission
    defaultBranchRef {
      name
    }
    isPrivate
  }
  query RepositoryNetwork {
    viewer { login }

    repo_000: repository(owner: "ntBre", name: "rust-pbqff") {
      ...repo
      parent {
        ...repo
      }
    }

  }
GraphQL variables: null

< HTTP/2.0 200 OK
< Access-Control-Allow-Origin: *
< Access-Control-Expose-Headers: ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset
< Content-Security-Policy: default-src 'none'
< Content-Type: application/json; charset=utf-8
< Date: Fri, 07 Jun 2024 19:29:39 GMT
< Referrer-Policy: origin-when-cross-origin, strict-origin-when-cross-origin
< Server: GitHub.com
< Strict-Transport-Security: max-age=31536000; includeSubdomains; preload
< Vary: Accept-Encoding, Accept, X-Requested-With
< X-Accepted-Oauth-Scopes: repo
< X-Content-Type-Options: nosniff
< X-Frame-Options: deny
< X-Github-Media-Type: github.v4; param=merge-info-preview.nebula-preview; format=json
< X-Github-Request-Id: 844A:2237C8:1A28B83:2F3DFC9:66635FA3
< X-Oauth-Client-Id: 178c6fc778ccc68e1d6a
< X-Oauth-Scopes: gist, read:org, repo
< X-Ratelimit-Limit: 5000
< X-Ratelimit-Remaining: 4934
< X-Ratelimit-Reset: 1717791391
< X-Ratelimit-Resource: graphql
< X-Ratelimit-Used: 66
< X-Xss-Protection: 0

{
  "data": {
    "viewer": {
      "login": "ntBre"
    },
    "repo_000": {
      "id": "R_kgDOHToUKw",
      "name": "pbqff",
      "owner": {
        "login": "ntBre"
      },
      "viewerPermission": "ADMIN",
      "defaultBranchRef": {
        "name": "master"
      },
      "isPrivate": false,
      "parent": null
    }
  }
}

* Request took 263.323694ms
* Request at 2024-06-07 15:29:39.561795284 -0400 EDT m=+0.307713077
* Request to https://api.github.com/graphql
> POST /graphql HTTP/1.1
> Host: api.github.com
> Accept: application/vnd.github.merge-info-preview+json, application/vnd.github.nebula-preview
> Authorization: token ████████████████████
> Content-Length: 584
> Content-Type: application/json; charset=utf-8
> Graphql-Features: merge_queue
> Time-Zone: America/New_York
> User-Agent: GitHub CLI v2.50.0

GraphQL query:
fragment pr on PullRequest{number,title,state,url,headRefName,headRepositoryOwner{id,login,...on User{name}},isCrossRepository,isDraft,createdAt}
    query PullRequestSearch(
      $q: String!,
      $limit: Int!,
      $endCursor: String,
    ) {
      search(query: $q, type: ISSUE, first: $limit, after: $endCursor) {
        issueCount
        nodes {
          ...pr
        }
        pageInfo {
          hasNextPage
          endCursor
        }
      }
    }
GraphQL variables: {"limit":30,"q":"author:dependabot[bot] repo:ntBre/pbqff state:open type:pr"}

< HTTP/2.0 200 OK
< Access-Control-Allow-Origin: *
< Access-Control-Expose-Headers: ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset
< Content-Security-Policy: default-src 'none'
< Content-Type: application/json; charset=utf-8
< Date: Fri, 07 Jun 2024 19:29:40 GMT
< Referrer-Policy: origin-when-cross-origin, strict-origin-when-cross-origin
< Server: GitHub.com
< Strict-Transport-Security: max-age=31536000; includeSubdomains; preload
< Vary: Accept-Encoding, Accept, X-Requested-With
< X-Accepted-Oauth-Scopes: repo
< X-Content-Type-Options: nosniff
< X-Frame-Options: deny
< X-Github-Media-Type: github.v4; param=merge-info-preview.nebula-preview; format=json
< X-Github-Request-Id: 844A:2237C8:1A28BEA:2F3E07D:66635FA3
< X-Oauth-Client-Id: 178c6fc778ccc68e1d6a
< X-Oauth-Scopes: gist, read:org, repo
< X-Ratelimit-Limit: 5000
< X-Ratelimit-Remaining: 4933
< X-Ratelimit-Reset: 1717791391
< X-Ratelimit-Resource: graphql
< X-Ratelimit-Used: 67
< X-Xss-Protection: 0

{
  "data": {
    "search": {
      "issueCount": 5,
      "nodes": [
        {
          "number": 269,
          "title": "Bump clap from 4.5.0 to 4.5.6",
          "state": "OPEN",
          "url": "https://github.com/ntBre/pbqff/pull/269",
          "headRefName": "dependabot/cargo/clap-4.5.6",
          "headRepositoryOwner": {
            "id": "MDQ6VXNlcjM2Nzc4Nzg2",
            "login": "ntBre",
            "name": "Brent Westbrook"
          },
          "isCrossRepository": false,
          "isDraft": false,
          "createdAt": "2024-06-07T02:45:05Z"
        },
        {
          "number": 268,
          "title": "Bump spectro from `97f9040` to `80b1dec`",
          "state": "OPEN",
          "url": "https://github.com/ntBre/pbqff/pull/268",
          "headRefName": "dependabot/cargo/spectro-80b1dec",
          "headRepositoryOwner": {
            "id": "MDQ6VXNlcjM2Nzc4Nzg2",
            "login": "ntBre",
            "name": "Brent Westbrook"
          },
          "isCrossRepository": false,
          "isDraft": false,
          "createdAt": "2024-06-06T02:32:37Z"
        },
        {
          "number": 261,
          "title": "Bump serde_json from 1.0.113 to 1.0.117",
          "state": "OPEN",
          "url": "https://github.com/ntBre/pbqff/pull/261",
          "headRefName": "dependabot/cargo/serde_json-1.0.117",
          "headRepositoryOwner": {
            "id": "MDQ6VXNlcjM2Nzc4Nzg2",
            "login": "ntBre",
            "name": "Brent Westbrook"
          },
          "isCrossRepository": false,
          "isDraft": false,
          "createdAt": "2024-05-08T02:54:16Z"
        },
        {
          "number": 255,
          "title": "Bump nalgebra from 0.32.3 to 0.32.5",
          "state": "OPEN",
          "url": "https://github.com/ntBre/pbqff/pull/255",
          "headRefName": "dependabot/cargo/nalgebra-0.32.5",
          "headRepositoryOwner": {
            "id": "MDQ6VXNlcjM2Nzc4Nzg2",
            "login": "ntBre",
            "name": "Brent Westbrook"
          },
          "isCrossRepository": false,
          "isDraft": false,
          "createdAt": "2024-03-29T02:56:53Z"
        },
        {
          "number": 250,
          "title": "Bump psqs from `e2ec9d9` to `d2ea8fe`",
          "state": "OPEN",
          "url": "https://github.com/ntBre/pbqff/pull/250",
          "headRefName": "dependabot/cargo/psqs-d2ea8fe",
          "headRepositoryOwner": {
            "id": "MDQ6VXNlcjM2Nzc4Nzg2",
            "login": "ntBre",
            "name": "Brent Westbrook"
          },
          "isCrossRepository": false,
          "isDraft": false,
          "createdAt": "2024-02-27T03:00:21Z"
        }
      ],
      "pageInfo": {
        "hasNextPage": false,
        "endCursor": "Y3Vyc29yOjU="
      }
    }
  }
}

* Request took 838.656256ms

Showing 5 of 5 pull requests in ntBre/pbqff that match your search

ID    TITLE                                     BRANCH                               CREATED AT
#269  Bump clap from 4.5.0 to 4.5.6             dependabot/cargo/clap-4.5.6          about 16 hours ago
#268  Bump spectro from `97f9040` to `80b1dec`  dependabot/cargo/spectro-80b1dec     about 1 day ago
#261  Bump serde_json from 1.0.113 to 1.0.117   dependabot/cargo/serde_json-1.0.117  about 1 month ago
#255  Bump nalgebra from 0.32.3 to 0.32.5       dependabot/cargo/nalgebra-0.32.5     about 2 months ago
#250  Bump psqs from `e2ec9d9` to `d2ea8fe`     dependabot/cargo/psqs-d2ea8fe        about 3 months ago

[williammartin]

Well, these are indeed searching different repos:

{"limit":30,"q":"author:dependabot[bot] repo:ntBre/rust-pbqff state:open type:pr"}

vs

{"limit":30,"q":"author:dependabot[bot] repo:ntBre/pbqff state:open type:pr"}

I'm thinking that there's a redirect going on here that is causing some confusion. Would it be fair to say that your git remote is still set to the old version? What's in git remote -v?

[ntBre]

Ahhh, that makes sense. I did rename this repo a while back. git remote -v reports ntBre/rust-pbqff (the old version) like you expected!

[williammartin]

It seems like the search infra doesn't account for redirects while indexing. Would you mind creating an issue on this repo quickly summarising what we've found here (I would but it's pretty late here already!). I'm not sure what our options are here but I'd like to think about it next week.

Sorry for all the noise @gibfahn ! 😬

[ntBre]

Will do, thanks for the help! And yes, sorry for the noise. I thought this would be more relevant at first!

[andyfeller]

When hooked up to a terminal, we try to provide a helpful message. I could see us printing something to stderr when there is no TTY but I suspect the reason Mislav didn't suggest that is because it would be a breaking change for any scripts that might be redirecting or consuming stderr as well. We try to be very careful about what we output to the iostreams when there is no TTY attached. Hyrum's Law and all.

Hoping to ground this somewhere in design artifacts, I think the only 2 I could really find are:

• Primer Style > CLI > Components > Header

• Primer Style > CLI > Foundations > Differences to note in machine output

  • No color or styling
  • State is explicitly written, not implied from color
  • Tabs between columns instead of table layout, since cut uses tabs as a delimiter
  • No truncation
  • Exact date format
  • No header 👈

• YouTube "Designing for the command line - Amanda Pinsker"

So it is a small comfort at least this much is documented though not necessarily the reasoning why. That said, I don't think screen output is a hard contract as structured data would be the less brittle route.

[benelan]

Bringing the discussion back to the original report, @gibfahn you could do this:

GH_FORCE_TTY=true NO_COLOR=1 gh pr list 2>&1 1>/dev/null | cat

EDITS:

• Added NO_COLOR to avoid ANSI escape sequences (see gh help environment for more info)
• Redirected stdout to /dev/null so it only outputs the log messages.

[benelan]

@andyfeller This is a good discussion, thanks for the resources! It seems those guidelines are related to machine-readability of output when there is no TTY, which gh does well. But writing to stderr doesn't effect machine-readability of stdout.

I think in general it is good practice to print to stderr, even when there is no TTY. I usually reference clig.dev for design conventions, and their Output section mentions:

It’s rare that printing nothing at all is the best default behavior, but it’s usually best to err on the side of less.

For instances where you do want no output (for example, when used in shell scripts), to avoid clumsy redirection of stderr to /dev/null, you can provide a -q option to suppress all non-essential output.

Additionally, when discussing where color should be disabled, one of the points is:

stdout or stderr is not an interactive terminal (a TTY). It’s best to individually check—if you’re piping stdout to another program, it’s still useful to get colors on stderr.

That implies it's good practice to print to stderr, even when there's no TTY.

A good example would be checking the logs when using gh in GitHub Actions and other continuous integration.

However, I also agree that breaking changes are bad and I'm not sure if this is worth one. It could be worth considering the CI environment variable, which is automatically set in GitHub Actions and other tools. Or GH_CI to be cautious of breaking changes. For example, when CI is set to any value and there is no TTY, I'd suggest:

• Printing to stderr as if there were a TTY, including color. Color could still be disabled with NO_COLOR CLICOLOR, etc.
• Keeping stdout machine readable like it currently is when there is no TTY