Suggestion: use cucumber + rest-assured for contract testing

Hi all,

I have created a contract test repo: https://github.com/OpenLMIS/openlmis-contract-tests

I wrote a sanity test using cucumber + rest-assured.

Cucumber allows us to write tests in a more readable manner, behind it, rest-assured would sending http requests and verifying response.

Please take a look and let me know if you have any questions.

Thanks

Thanks Pengfei – I really like the readability of the tests.

SolDevelo guys – want to take some of the Postman tests and try them out in Cucumber/gherkin?

···

On Wednesday, August 24, 2016 at 3:34:28 AM UTC-7, pfcui@thoughtworks.com wrote:

Hi all,

I have created a contract test repo: https://github.com/OpenLMIS/openlmis-contract-tests

I wrote a sanity test using cucumber + rest-assured.

Cucumber allows us to write tests in a more readable manner, behind it, rest-assured would sending http requests and verifying response.

Please take a look and let me know if you have any questions.

Thanks

Hello,

  When in Seattle we decided on Postman as opposed to the RestAssured tests we were starting as it would be easier for less technical users to write tests. From what I can see Cucumber is a BDD framework, so the tests (at least the spec part) would be Java code, then we would write these test cases as specs. Am I getting that right?

How would you see the flow of us using it? If the product owner/QAs would write the specs - who do you see writing the steps? Do you think this would ultimately fall more on the shoulders of the development team or the QA?

Regards,

Paweł
···

On 25.08.2016 01:40, Jake Watson wrote:

    Thanks Pengfei -- I really like the readability of the tests.
      SolDevelo guys -- want to take some of the Postman tests and try them out in Cucumber/gherkin?



      On Wednesday, August 24, 2016 at 3:34:28 AM UTC-7, wrote:

  You received this message because you are subscribed to the Google Groups "OpenLMIS Dev" group.

  To unsubscribe from this group and stop receiving emails from it, send an email to openlmis-dev+unsubscribe@googlegroups.com.

  To post to this group, send email to openlmis-dev@googlegroups.com.

  To view this discussion on the web visit [https://groups.google.com/d/msgid/openlmis-dev/6ffbdc93-91e7-48f5-a9d8-15b08557b619%40googlegroups.com](https://groups.google.com/d/msgid/openlmis-dev/6ffbdc93-91e7-48f5-a9d8-15b08557b619%40googlegroups.com?utm_medium=email&utm_source=footer).

  For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

pfcui@thoughtworks.com

Hi all,

I have created a contract test repo: https://github.com/OpenLMIS/openlmis-contract-tests

I wrote a sanity test using cucumber + rest-assured.

            Cucumber allows us to write tests in a more readable manner, behind it, rest-assured would sending http requests and verifying response.
            Please take a look and let me know if you have any questions.

Thanks

Hi Paweł,

Cucumber specs can be written in gherkin, which is its DSL.

But the step definition will be written in Java code, which I think will be more of developer’s workload.

Postman would be ok too if it could be incorporated into Jenkins build.

Is there a Postman Collection already? If so, I can try to run that as contract test and see how it goes.

And Cucumber has a JS version as well, we can still use it to write specs and behind the specs there are step definitions filled with postman collections.

The major benefit I see in Cucumber is that it makes it easier for less technical people to see what is being covered by tests.

···

On Thursday, August 25, 2016 at 8:50:32 PM UTC+8, Paweł Gesek wrote:

Hello,

  When in Seattle we decided on Postman as opposed to the RestAssured tests we were starting as it would be easier for less technical users to write tests. From what I can see Cucumber is a BDD framework, so the tests (at least the spec part) would be Java code, then we would write these test cases as specs. Am I getting that right?
How would you see the flow of us using it? If the product owner/QAs would write the specs - who do you see writing the steps? Do you think this would ultimately fall more on the shoulders of the development team or the QA?



Regards,

Paweł

On 25.08.2016 01:40, Jake Watson wrote:

    Thanks Pengfei -- I really like the readability of the tests.
      SolDevelo guys -- want to take some of the Postman tests and try them out in Cucumber/gherkin?



      On Wednesday, August 24, 2016 at 3:34:28 AM UTC-7, pf...@thoughtworks.com wrote:

Hi all,

I have created a contract test repo: https://github.com/OpenLMIS/openlmis-contract-tests

I wrote a sanity test using cucumber + rest-assured.

            Cucumber allows us to write tests in a more readable manner, behind it, rest-assured would sending http requests and verifying response.
            Please take a look and let me know if you have any questions.

Thanks

  You received this message because you are subscribed to the Google Groups "OpenLMIS Dev" group.

  To unsubscribe from this group and stop receiving emails from it, send an email to openlmis-dev...@googlegroups.com.

  To post to this group, send email to openlm...@googlegroups.com.

  To view this discussion on the web visit [https://groups.google.com/d/msgid/openlmis-dev/6ffbdc93-91e7-48f5-a9d8-15b08557b619%40googlegroups.com](https://groups.google.com/d/msgid/openlmis-dev/6ffbdc93-91e7-48f5-a9d8-15b08557b619%40googlegroups.com?utm_medium=email&utm_source=footer).

  For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

Interesting, in considering Cucumber, there are some tests from v1 used with Selenium:

https://github.com/OpenLMIS/open-lmis/tree/master/test-modules/functional-tests/src/test/resources/org/openlmis/functional

I’d be curious to see how well Cucumber makes clear the JSON needed at the endpoints, and what if any overlap there is with example JSON that could/should be in RAML?

Thanks,

Josh

···

On Aug 25, 2016, at 7:06 PM, > pfcui@thoughtworks.com wrote:

Hi Paweł,

Cucumber specs can be written in gherkin, which is its DSL.

But the step definition will be written in Java code, which I think will be more of developer’s workload.

Postman would be ok too if it could be incorporated into Jenkins build.

Is there a Postman Collection already? If so, I can try to run that as contract test and see how it goes.

And Cucumber has a JS version as well, we can still use it to write specs and behind the specs there are step definitions filled with postman collections.

The major benefit I see in Cucumber is that it makes it easier for less technical people to see what is being covered by tests.

On Thursday, August 25, 2016 at 8:50:32 PM UTC+8, Paweł Gesek wrote:

Hello,

When in Seattle we decided on Postman as opposed to the RestAssured tests we were starting as it would be easier for less technical users to write tests. From what I can see Cucumber is a BDD framework, so the tests (at least the spec part) would be Java code, then we would write these test cases as specs. Am I getting that right?

How would you see the flow of us using it? If the product owner/QAs would write the specs - who do you see writing the steps? Do you think this would ultimately fall more on the shoulders of the development team or the QA?

Regards,

Paweł

On 25.08.2016 01:40, Jake Watson wrote:

Thanks Pengfei – I really like the readability of the tests.

SolDevelo guys – want to take some of the Postman tests and try them out in Cucumber/gherkin?

On Wednesday, August 24, 2016 at 3:34:28 AM UTC-7, > > > pf...@thoughtworks.com wrote:

Hi all,

I have created a contract test repo: https://github.com/OpenLMIS/openlmis-contract-tests

I wrote a sanity test using cucumber + rest-assured.

Cucumber allows us to write tests in a more readable manner, behind it, rest-assured would sending http requests and verifying response.

Please take a look and let me know if you have any questions.

Thanks

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to
openlmis-dev...@googlegroups.com.

To post to this group, send email to
openlm...@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/openlmis-dev/6ffbdc93-91e7-48f5-a9d8-15b08557b619%40googlegroups.com
.

For more options, visit
https://groups.google.com/d/optout
.

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to
openlmis-dev@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/openlmis-dev/ead88b05-d4f1-4e61-abc4-fde99fd355f1%40googlegroups.com
.

For more options, visit https://groups.google.com/d/optout.

I just tried to integrate postman with cucumber.js, it did not go well.

A postman collection is basically a json file that describes the end point under test, the expected response.

{
“id”: “04e3f4e4-1b93-7f3f-0fd7-9af00f57161a”,
“name”: “cuke”,
“description”: “try to use this in cucumber”,
“order”: [
“d12a2691-e5d4-b83d-a6e2-7fa576c72313”
],
“folders”: [],
“timestamp”: 1472192231695,
“owner”: 0,
“public”: false,
“published”: false,
“requests”: [
{
“id”: “d12a2691-e5d4-b83d-a6e2-7fa576c72313”,
“headers”: “Content-Type: application/json\n”,
“url”: “http://jsonplaceholder.typicode.com/posts”,
“preRequestScript”: “”,
“pathVariables”: {},
“method”: “POST”,
“data”: [],
“dataMode”: “raw”,
“version”: 2,
“tests”: “var jsonData = JSON.parse(responseBody);\ntests[“has title”] = jsonData.title === ‘foo’;”,
“currentHelper”: “normal”,
“helperAttributes”: {},
“time”: 1472192692688,
“name”: “post to json place holder”,
“description”: “send post to json placeholder”,
“collectionId”: “04e3f4e4-1b93-7f3f-0fd7-9af00f57161a”,
“responses”: [],
“rawModeData”: “{“title”:“foo”}”
}
]
}

``

It could be ran with newman, the postman cli tool.

Or it could be ran with newman’s API, like:

var newman = require(‘newman’);

newman.run({
collection: require(’./some_test_collection.json’),//it could be url here as well
reporters: ‘cli’
}, function (err) {
if (err) { throw err; }
console.log(‘collection run complete!’);
});

``

As we can see here, the smallest granularity of control that newman gives us is a collection, which contains all steps of a test already.

So defining cucumber steps(given, when, then, smaller steps) with postman collection would not work out.

Due to the granularity of control that newman exposes, any BDD framework that requires us to define the setup, the action, the assertion separately will not work with postman.

···

On Friday, August 26, 2016 at 10:06:17 AM UTC+8, pf...@thoughtworks.com wrote:

Hi Paweł,

Cucumber specs can be written in gherkin, which is its DSL.

But the step definition will be written in Java code, which I think will be more of developer’s workload.

Postman would be ok too if it could be incorporated into Jenkins build.

Is there a Postman Collection already? If so, I can try to run that as contract test and see how it goes.

And Cucumber has a JS version as well, we can still use it to write specs and behind the specs there are step definitions filled with postman collections.

The major benefit I see in Cucumber is that it makes it easier for less technical people to see what is being covered by tests.

On Thursday, August 25, 2016 at 8:50:32 PM UTC+8, Paweł Gesek wrote:

Hello,

  When in Seattle we decided on Postman as opposed to the RestAssured tests we were starting as it would be easier for less technical users to write tests. From what I can see Cucumber is a BDD framework, so the tests (at least the spec part) would be Java code, then we would write these test cases as specs. Am I getting that right?
How would you see the flow of us using it? If the product owner/QAs would write the specs - who do you see writing the steps? Do you think this would ultimately fall more on the shoulders of the development team or the QA?



Regards,

Paweł

On 25.08.2016 01:40, Jake Watson wrote:

    Thanks Pengfei -- I really like the readability of the tests.
      SolDevelo guys -- want to take some of the Postman tests and try them out in Cucumber/gherkin?



      On Wednesday, August 24, 2016 at 3:34:28 AM UTC-7, pf...@thoughtworks.com wrote:

Hi all,

I have created a contract test repo: https://github.com/OpenLMIS/openlmis-contract-tests

I wrote a sanity test using cucumber + rest-assured.

            Cucumber allows us to write tests in a more readable manner, behind it, rest-assured would sending http requests and verifying response.
            Please take a look and let me know if you have any questions.

Thanks

  You received this message because you are subscribed to the Google Groups "OpenLMIS Dev" group.

  To unsubscribe from this group and stop receiving emails from it, send an email to openlmis-dev...@googlegroups.com.

  To post to this group, send email to openlm...@googlegroups.com.

  To view this discussion on the web visit [https://groups.google.com/d/msgid/openlmis-dev/6ffbdc93-91e7-48f5-a9d8-15b08557b619%40googlegroups.com](https://groups.google.com/d/msgid/openlmis-dev/6ffbdc93-91e7-48f5-a9d8-15b08557b619%40googlegroups.com?utm_medium=email&utm_source=footer).

  For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

Yes, I agree that Postman and Cucumber are two separate technologies that are not made for each other and I doubt anyone is mixing them currently, so we will have to decide on either one.

FYI there was a sample Postman collection posted in the comments of Regards, Paweł
···

https://openlmis.atlassian.net/secure/RapidBoard.jspa?rapidView=46&view=detail&selectedIssue=OLMIS-965&sprint=70

   On 26.08.2016 09:45, wrote:
    I just tried to integrate postman with cucumber.js, it did not go well.
      A postman collection is basically a json file that describes the end point under test, the expected response.

{

               "id": "04e3f4e4-1b93-7f3f-0fd7-9af00f57161a",

               "name": "cuke",

               "description":                   "try to use this in cucumber",

               "order": [

               "d12a2691-e5d4-b83d-a6e2-7fa576c72313"

               ],

               "folders": [],

               "timestamp": 1472192231695,

               "owner": 0,

               "public": false,

               "published": false,

               "requests": [

               {

               "id": "d12a2691-e5d4-b83d-a6e2-7fa576c72313",

               "headers":                   "Content-Type: application/json\n",

               "url": ,

               "preRequestScript": "",

               "pathVariables": {},

               "method": "POST",

               "data": [],

               "dataMode": "raw",

               "version": 2,

               "tests":                   "var jsonData = JSON.parse(responseBody);\ntests[\"has title\"] = jsonData.title === 'foo';",

               "currentHelper": "normal",

               "helperAttributes": {},

               "time": 1472192692688,

               "name":                   "post to json place holder",

               "description":                   "send post to json placeholder",

               "collectionId": "04e3f4e4-1b93-7f3f-0fd7-9af00f57161a",

               "responses": [],

               "rawModeData": "{\"title\":\"foo\"}"

               }

               ]

            }

``

It could be ran with newman , the postman cli tool.

Or it could be ran with newman’s API, like:

var
newman = require(‘newman’);

                newman.run({

                    collection: require('./some_test_collection.json'),                    //it could be url here as well

                    reporters: 'cli'

              }, function (err) {

                    if (err) { throw err; }

                    console.log(                    'collection run complete!');

              });

``

As we can see here, the ** smallest granularity of control** that newman gives us is a collection, which contains all steps of a test already.

          So defining cucumber steps(given, when, then, smaller steps) with postman collection would not work out.
        Due to the granularity of control that newman exposes, any BDD framework that requires us to define the setup, the action, the assertion separately will not work with postman.
      On Friday, August 26, 2016 at 10:06:17 AM UTC+8, wrote:

  You received this message because you are subscribed to the Google Groups "OpenLMIS Dev" group.

  To unsubscribe from this group and stop receiving emails from it, send an email to openlmis-dev+unsubscribe@googlegroups.com.

  To post to this group, send email to openlmis-dev@googlegroups.com.

  To view this discussion on the web visit [https://groups.google.com/d/msgid/openlmis-dev/2deb0054-1f1b-412c-a705-594c7213e004%40googlegroups.com](https://groups.google.com/d/msgid/openlmis-dev/2deb0054-1f1b-412c-a705-594c7213e004%40googlegroups.com?utm_medium=email&utm_source=footer).

  For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

pfcui@thoughtworks.com
"http://jsonplaceholder.typicode.com/posts"pf...@thoughtworks.com

Hi Paweł,

            Cucumber specs can be written in gherkin, which is its DSL.
            But the step definition will be written in Java code, which I think will be more of developer's workload.
            Postman would be ok too if it could be incorporated into Jenkins build.
            Is there a Postman Collection already? If so, I can try to run that as contract test and see how it goes.
            And Cucumber has a JS version as well, we can still use it to write specs and behind the specs there are step definitions filled with postman collections.
            The major benefit I see in Cucumber is that it makes it easier for less technical people to see what is being covered by tests.
            On Thursday, August 25, 2016 at 8:50:32 PM UTC+8, Paweł Gesek wrote:

Hello,

                  When in Seattle we decided on Postman as opposed to the RestAssured tests we were starting as it would be easier for less technical users to write tests. From what I can see Cucumber is a BDD framework, so the tests (at least the spec part) would be Java code, then we would write these test cases as specs. Am I getting that right?
                How would you see the flow of us using it? If the product owner/QAs would write the specs - who do you see writing the steps? Do you think this would ultimately fall more on the shoulders of the development team or the QA?



                Regards,

                Paweł

On 25.08.2016 01:40, Jake Watson wrote:

                    Thanks Pengfei -- I really like the readability of the tests.
                      SolDevelo guys -- want to take some of the Postman tests and try them out in Cucumber/gherkin?



                      On Wednesday, August 24, 2016 at 3:34:28 AM UTC-7, pf...@thoughtworks.com
                      wrote:

Hi all,

I have created a contract test repo: https://github.com/OpenLMIS/openlmis-contract-tests

                            I wrote a sanity test using cucumber + rest-assured.
                            Cucumber allows us to write tests in a more readable manner, behind it, rest-assured would sending http requests and verifying response.
                            Please take a look and let me know if you have any questions.

Thanks

                  You received this message because you are subscribed to the Google Groups "OpenLMIS Dev" group.

                  To unsubscribe from this group and stop receiving emails from it, send an email to openlmis-dev...@googlegroups.com.

                  To post to this group, send email to openlm...@googlegroups.com.

                  To view this discussion on the web visit [https://groups.google.com/d/msgid/openlmis-dev/6ffbdc93-91e7-48f5-a9d8-15b08557b619%40googlegroups.com](https://groups.google.com/d/msgid/openlmis-dev/6ffbdc93-91e7-48f5-a9d8-15b08557b619%40googlegroups.com?utm_medium=email&utm_source=footer).

                  For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).