Rust insta based tests - #210
Conversation
This patch series introduces the rust insta crate based testing upgrade. The insta crate allows for snapshot comparison which is more powerful than the current bash script based testing framework. For documentation https://docs.rs/insta/latest/insta/ This patch introduces a new crate named qemu-tests. * The qemu-tests has a mod named common, which lists the bins available in each of the examples * Once bins are populated, we use rust's command execution from std, and call the `cargo run` with specific flags and --bin * Before validating the test suite needs to be bootstrapped for the insta to load the snapshot for comparsion please refer to justfile for notes on how to initialize the test suite. Signed-off-by: Sriram Raghunathan <sriram@hcoop.net>
… with insta crate Signed-off-by: Sriram Raghunathan <sriram@hcoop.net>
…rkflow to use updated test-qemu Signed-off-by: Sriram Raghunathan <sriram@hcoop.net>
Signed-off-by: Sriram Raghunathan <sriram@hcoop.net>
jonathanpallant
left a comment
jonathanpallant
left a comment
There was a problem hiding this comment.
Thank you for these changes. I think the proposed solution is going to work well and I left you some thoughts.
| @@ -0,0 +1,10 @@ | |||
| [package] | |||
| name = "qemu-tests" | |||
There was a problem hiding this comment.
Maybe call this 'run-tests'? I don't think there's anything QEMU specific about it, and we might use other runners in future.
There was a problem hiding this comment.
I would rename the environment variables too.
There was a problem hiding this comment.
'run-tests' sounded generic, perhaps! Something like 'aarch32-tests' ? Following the other crates aarch32-rt , aarch32-rt-macros.
Does that sound ok?
| - test-qemu-v7a-zynq | ||
| - test-qemu-v8r | ||
| - test-qemu-v8r-el2 | ||
| - test-qemu-versatileab |
There was a problem hiding this comment.
I'd prefer to leave the justfile target names as they were - if I'm working on something ARMv6 specific I wouldn't want to keep running the tests for the other architectures, even if they run on the same machine.
| #[test] | ||
| #[ignore = "requires QEMU_TARGET; run via the justfile"] | ||
| fn test_examples() { | ||
| let target = std::env::var("QEMU_TARGET") |
There was a problem hiding this comment.
Would it make sense to have one test per target, instead of passing in the target as an environment variable? Then bare cargo test would work, and you'd see:
armv4t-none-eabi PASS
armv5te-none-eabi PASS
armv6-none-eabi PASS
etc etc
Signed-off-by: Sriram Raghunathan <sriram@hcoop.net>
2 participants
This patch series introduces rust insta based testing framework. The insta framework documentation is available (https://docs.rs/insta/latest/insta/).
The PR address #138