In order to find a workflow which will allow us to easily find automated tests for a given feature, I have come up with the solution described below.
The feature’s name should include similar keywords as we decided to use as labels on Jira e.g. if we’re writing a functional test scenario for adding a new processing period, the feature’s name should be like: “Adding processing periods”. Unlike functional tests, contract tests are grouped by resource, so the scenario for adding and editing is in one feature. In this case, I see two ways to improve searching:
- use a little bit different pattern, and name features like: “Processing periods” or
- split them into smaller features (for adding/updating etc.), and use the same convention as for functional tests.
For both types of tests, the scenario’s name should follow the convention: <user_name> should be able to <action_description>.
Moreover, functional test results are not grouped. In my opinion, it makes a mess and slows down searching on Jenkins, so I would also suggest to update them to be grouped by resource or screen.
Suggested next steps:
- update documentation to include the pattern
- create a ticket to apply the pattern
- create a ticket to add grouping in functional tests results
Let me know, if you have some questions or suggestions.
To be honest, I’m a little lost on what is the suggested pattern/naming convention you are proposing. Are you saying that both contract tests and functional tests would be using the exact same one?
Could you give a complete example of a contract test and a functional test that would have the naming following the new convention (and an old name - for comparison).
Here you can find an example of the updated contract test: https://github.com/OpenLMIS/openlmis-contract-tests/commit/be48facca2e4ae3d1249ac95c1cf4bae16caedf5 and functional test https://github.com/OpenLMIS/openlmis-functional-tests/blob/master/src/features/administration/reason-add/reason-add.feature in which I would update only Feature line to say Adding reasons. Let me know if it’s more clear than my previous description.
Thank you for those examples, Klaudia. I have some doubts on whether this naming convention alone will be enough to improve searching for the specific use cases.
One thing I’d recommend, if we haven’t done that yet, is to also document how the test scenarios/features map to the code. In case of contract tests, is there a 1:1 mapping betweeen a scenario and an endpoint? For functional tests, does a single feature always cover the related UI screen?
Thanks for your suggestion.
As for contract tests, sometimes a scenario checks whole workflow (requisition), not only specific endpoint. In functional tests, single feature covers only one (related) UI screen but sometimes we have more than one feature for the screen.
I agree with all your ideas - I think that changing the naming conventions for automated tests to the ones that you proposed, and thus similar to the ones in the keywords that I proposed, will be enough to facilitate searching for the tests covering specific features, regardless of whether the test covers a whole workflow, or only a single feature. Also, the mentioned next-step tickets seem a good solution to the problem.
I think this makes sense and we can proceed with the tickets. I’m not sure whether those changes alone will be sufficient, but we will asses once we have more functional tests developed. For now, having this standardized naming convention is a step in the right direction in my eyes.
Please proceed with the next steps. As for the new tickets, add them to this epic: https://openlmis.atlassian.net/browse/OLMIS-5651