Fetching and Using Blob Data in React Native: Demystified

Today we are excited to release flakiness detection in Waldo.
For those not familiar with Waldo: our product allows nontechnical people to create, maintain and orchestrate automation testing for their mobile apps right from the web browser. Waldo is built as a “script-less” automation engine. This means that end users are not burdened with writing scripts; neither does Waldo conceptualize tests as scripts internally. This leads to interesting benefits, including how to address the problem of test flakiness.
Flakiness is probably the worst thing that can happen to automated end-to-end testing. A “flaky” test is simply one that yields different outcomes when played multiple times.
Flakiness causes most teams to doubt their automation results and leads them to consider errors to be “probably because of a blip or something”.
Because Waldo is an automation platform, we have always had the problem of flakiness in mind, and thus we have discovered that there are 2 ways we can address it:
We are just now releasing that second part, and we think it is a fundamental addition to the Waldo platform and how people can now evaluate their test results.
Most importantly, Waldo needs to able to tell that a test is flaky.
Before this change, Waldo would partition automation results into 2 conceptual categories:
This meant that any time a test resulted in an error, one could understandably raise an eyebrow: “Wait, is that test really broken?“.
Instead, Waldo now partitions automation results into 3 categories:
This immediately provides 2 important benefits:
If a test is considered flaky, Waldo displays something like this:
The Waldo platform has always been very visual. We at Waldo believe that a picture is worth a thousand words (and probably a million cryptic lines to stdout). When it comes to a flaky test, it is no different. Waldo shows the state in plain sight, with all the different outcomes that occurred while replaying it, so that it is straightforward for you to analyze what caused the flakiness.