The Wayback Machine - http://web.archive.org/web/20201123040736/https://github.com/sharkdp/hyperfine/issues/348
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Export exit codes to JSON #348

Open
sharkdp opened this issue Nov 5, 2020 · 0 comments
Open

Export exit codes to JSON #348

sharkdp opened this issue Nov 5, 2020 · 0 comments

Comments

@sharkdp
Copy link
Owner

@sharkdp sharkdp commented Nov 5, 2020

If running a benchmark with --ignore-failure, it might be interesting to know if some of the commands actually failed.

For example, consider a long running compile-time benchmark:

hyperfine \
  --runs 3 \
  --ignore-failure \
  --parameter-list git_tag "v0.1,v0.2,v0.3,v0.4,v0.4.1,v0.5" \
  --prepare "git checkout --force {git_tag}; make clean" \
  --export-json "compile_time.json" \
  "make -j8"

We pass --ignore-failure because we don't want all benchmark results to be lost if one of the tags does not compile cleanly. However, at the end of the run, we would like to know if one of the commands actually failed.

We could add a exit_codes: Vec<i32> field to BenchmarkResult such that the exit codes would be exported to the JSON file.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
1 participant
You can’t perform that action at this time.