We all should try that our End-to-End test should NOT considerably reiterate the test efforts of our Unit test and API test. Ideally, our end-to-end should be meant to confirm that user(s) can use our app in the way it’s intended to do so, perform interactions with it without hitting any issues, and always work when performing any E2E transaction(s). On the other hand, our Unit and API test should test and cover business logic.
Many a time, we go overboard to reach 100% coverage, create an automated mess in terms of freezing the scope of our End-to-End tests and we might reach to a point:
– Where our test source code becomes even the same or in fact, more than the application codebase.
– And/Or automation execution run time will likely take the same time as it takes to write the test case, etc.
There’s no silver bullet here and it’s all about trying, failing, and finally succeeding. One such approach can be:
Divide your test cases broadly into two major groups: Fat and Thin. Your Fat test cases should be what a maximum number of users perform repeatedly on your application. First target these and spend your max energy, effort, and time on these because if these aren’t working- as Russell Peters said- Somebody is going to get hurt real bad.
Then only jump to thin cases. What are thin cases? These are those cases where users use our application mostly in an unexpected and unforeseen way but again as these can create issues in your app these are also important. Still, obviously, these take a second priority.
Based on the execution test time allowed (on our final test cycle) and regression impact, QE can recommend running both Fat and Thin tests (recommended) OR only Fat ones on the last cycle.
P.S. As part of our daily CI/CD test automation run, we should try to run both Fat and Thin tests.