We run our full suite which gives our engineers more time to work on building new features or improving existing ones.
The Challenge
Keepsafe's QA was manual, tedious & time-consuming.
Keepsafe provides additional privacy and security protection for over 75 million smartphone users. Their entire business relies on upholding the military grade encryptions that prevent anyone, including Keepsafe employees, from accessing customer assets stored within their app.
The team can't afford any regressions, given that any interference with their app performance could compromise sensitive assets that customers are paying them to protect.
As a result, testing is of utmost importance to the Keepsafe team, and they need to make sure features in their app works as intended when any changes are made. One of their most popular services is their Photo Vault: which adds the complexity of testing camera access and the device's photo gallery.
To tackle testing, their team would gather once every 3 weeks before release, pick up their own devices, and go through a spreadsheet with more than 500 end-to-end test scenarios.
That meant time spent preparing the physical devices, manually testing user flows, manually documenting each test scenario and outcome…and if a regression was discovered: the team had to fix that issue and then repeat the testing process all over again.
This process was incredibly expensive: requiring time from every team member across the org just to meet basic release requirements. Keepsafe needed a solution that would promise testing coverage on tight deadlines.
The Solution
Saving thousands of hours across the team
With over 500 end to end test scenarios, it was clear that the only sustainable, long term testing solution for Keepsafe was automation.
They selected Waldo to allow every team member to contribute to test automation, gain access to fully maintained out of the box testing infrastructure, and produce better quality results with every test run.
Waldo empowered every existing member of the Keepsafe team to build fully automated, E2E tests in minutes…even ones that involve accessing the camera roll and camera functions. Creating a fully automated test in Waldo is as easy as using your app: move through the test scenario on a simulated device, directly in your browser.
The team no longer needs to spend time setting up their own testing devices. Waldo provides Keepsafe (and all customers) with the testing infrastructure required for their tests: supporting a wide variety of both iOS and Android simulated devices.
Instead of manually documenting the test scenario and outcomes, the Keepsafe team can click into the Waldo Result’s Studio after every run to view a full, video Replay of the test, examine step by step breakdowns of every screen, and quickly surface any issues. Any team member can login to the Waldo account to view these artifacts.
The Result
From a week to test the whole app, to just two hours
With Waldo, Keepsafe is running 700+ tests across iOS and Android covering 95% of their testing needs.
Currently, they run their suite 2-4 times a month, and all of the scenarios are completed in just 2 hours. Any errors send notifications to the relevant team, who make necessary adjustments, and then will re-run the suite to ensure everything is passing.
The team has full documentation of every single test that is run, every outcome, and a comprehensive overview of their testing history.
What used to take a week, now takes a couple of hours. And as of today, Keepsafe has run 500,000+ tests, saving 1,000+ hours of QA.
The FUTURE
With Waldo, we’re able to test anything around the gallery, which is key to us, without a single line of code.
ABOUT THE CUSTOMER
Keepsafe is a leader in cloud data security. Their encryption is among the most secure in the world, and since 2012, they've enabled their users to keep photos, apps, and browsing private.