Orderable edit design


(Nikodem Graczewski) #1

Hi everyone,

as per our Sprint Q&A meeting I’m starting a discussion about how do we want to approach the Orderables edit as allowing this action can possibly result is some unforeseen consequences in some of the modules depending on them.

Here’s a list of modules that are somehow related to Orderables:

  • referencedata - holds the Orderables
  • requisition
  • fulfillment
  • stockmanagement
    I’ve done some research and I could only find one thing that would break after we implement Orderables edit. Requisition line item uses the ProgramOrderable to actually set the price per pack of itself. If we modify the ProgramsOrderable price per pack after the requisition initialization it won’t be reflected in the requisition line item.

Here’s a couple of ideas on how we could approach the problem:

  • We could create a new endpoint for updating the ProgramOrderables only, without touching the Orderable itself as proposed by Paweł Gesek here.
  • We could use the FHIR approach by versioning the Orderable. This approach would use two IDs, one for tracking the Orderable itself, so the ID of Orderable would never change and a second for tracking the Orderable version.
    This would allow us to:
  • Retrieve the newest version of Orderable simply by calling a GET endpoint with the ID. This would be used by the administration screens so they always display the newest Orderable. Modifying the Orderable would result in creation of a new version of it, and storing the previous one as history.
  • All the module that would require the older version of the Orderable would simply pass the version ID as a request parameter to the GET endpoint. This would result in retrieving a history version of an Orderable instead of the newest one.
    If you have any ideas, comments or concerns about this feature, please post them in this thread.

Best Regards,
Nikodem


(Josh Zamor) #2

Thanks for getting this started here Nikodem,

For reference the ticket, including some initial discussion and a proposed pattern is located here: https://openlmis.atlassian.net/browse/OLMIS-3167

I don’t think we’re on the same page here, if I’m reading this as you intended:

I’ve done some research and I could only find one thing that would break after we implement Orderables edit. Requisition line item uses the ProgramOrderable to actually set the price per pack of itself. If we modify the ProgramsOrderable price per pack after the requisition initialization it won’t be reflected in the requisition line item.

The functionality that is desired is that if you change the price per pack of the Orderable for a program today, it should not change the price of that Orderable on a line item in a Requisition or Order that has already been created. e.g. if a Requisition was created 2 years ago for a price of X, and the price today of line items has increased by 25%, we wouldn’t expect the price in that 2 year old Requisition to suddenly increase. It should rather stay the same price as when the Requisition was initiated, and this should be immutable. This is true for the rest of the fields in Orderable and Program Orderable

I also noticed a couple things:

  • The link you included for Pawel’s idea of building a new API for just changing Program Orderables appears to go to an unrelated code review.
    • We’ve expanded the topic to include updating Orderables, not just solving for updating Program Orderables.
  • Thanks for linking to the FHIR proposal, for background for people considering I have related points in the issue.

Let’s make sure we’re on the same page about desired functionality, and then take a crack at evaluating what would break, including expected functionality.

Best,

Josh

···

On Sep 29, 2017, at 9:09 AM, Nikodem Graczewski ngraczewski@soldevelo.com wrote:

Hi everyone,

as per our Sprint Q&A meeting I’m starting a discussion about how do we want to approach the Orderables edit as allowing this action can possibly result is some unforeseen consequences in some of the modules depending on them.

Here’s a list of modules that are somehow related to Orderables:

  • referencedata - holds the Orderables
  • requisition
  • fulfillment
  • stockmanagement
    I’ve done some research and I could only find one thing that would break after we implement Orderables edit. Requisition line item uses the ProgramOrderable to actually set the price per pack of itself. If we modify the ProgramsOrderable price per pack after the requisition initialization it won’t be reflected in the requisition line item.

Here’s a couple of ideas on how we could approach the problem:

  • We could create a new endpoint for updating the ProgramOrderables only, without touching the Orderable itself as proposed by Paweł Gesek here.

  • We could use the FHIR approach by versioning the Orderable. This approach would use two IDs, one for tracking the Orderable itself, so the ID of Orderable would never change and a second for tracking the Orderable version.

    This would allow us to:

    • Retrieve the newest version of Orderable simply by calling a GET endpoint with the ID. This would be used by the administration screens so they always display the newest Orderable. Modifying the Orderable would result in creation of a new version of it, and storing the previous one as history.
    • All the module that would require the older version of the Orderable would simply pass the version ID as a request parameter to the GET endpoint. This would result in retrieving a history version of an Orderable instead of the newest one.
      If you have any ideas, comments or concerns about this feature, please post them in this thread.

Best Regards,

Nikodem

**

SolDevelo** Sp. z o.o. [LLC] /
www.soldevelo.com

Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland

Phone: +48 58 782 45 40 / Fax: +48 58 782 45 41

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to
openlmis-dev@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/openlmis-dev/6ce639ba-ac8e-48df-a201-3f3d4455316a%40googlegroups.com
.

For more options, visit https://groups.google.com/d/optout.


(Josh Zamor) #3

Quick correction, I typed “FHIR proposal” when what I meant to say is that the idea is to borrow a pattern seen in FHIR. I’ve not proposed FHIR at this point for our Orderable representations.

···

On Sep 29, 2017, at 10:22 AM, Josh Zamor josh.zamor@villagereach.org wrote:

Thanks for getting this started here Nikodem,

For reference the ticket, including some initial discussion and a proposed pattern is located here: https://openlmis.atlassian.net/browse/OLMIS-3167

I don’t think we’re on the same page here, if I’m reading this as you intended:

I’ve done some research and I could only find one thing that would break after we implement Orderables edit. Requisition line item uses the ProgramOrderable to actually set the price per pack of itself. If we modify the ProgramsOrderable price per pack after the requisition initialization it won’t be reflected in the requisition line item.

The functionality that is desired is that if you change the price per pack of the Orderable for a program today, it should not change the price of that Orderable on a line item in a Requisition or Order that has already been created. e.g. if a Requisition was created 2 years ago for a price of X, and the price today of line items has increased by 25%, we wouldn’t expect the price in that 2 year old Requisition to suddenly increase. It should rather stay the same price as when the Requisition was initiated, and this should be immutable. This is true for the rest of the fields in Orderable and Program Orderable

I also noticed a couple things:

  • The link you included for Pawel’s idea of building a new API for just changing Program Orderables appears to go to an unrelated code review.
    • We’ve expanded the topic to include updating Orderables, not just solving for updating Program Orderables.
  • Thanks for linking to the FHIR proposal, for background for people considering I have related points in the issue.

Let’s make sure we’re on the same page about desired functionality, and then take a crack at evaluating what would break, including expected functionality.

Best,

Josh

On Sep 29, 2017, at 9:09 AM, Nikodem Graczewski ngraczewski@soldevelo.com wrote:

Hi everyone,

as per our Sprint Q&A meeting I’m starting a discussion about how do we want to approach the Orderables edit as allowing this action can possibly result is some unforeseen consequences in some of the modules depending on them.

Here’s a list of modules that are somehow related to Orderables:

  • referencedata - holds the Orderables
  • requisition
  • fulfillment
  • stockmanagement
    I’ve done some research and I could only find one thing that would break after we implement Orderables edit. Requisition line item uses the ProgramOrderable to actually set the price per pack of itself. If we modify the ProgramsOrderable price per pack after the requisition initialization it won’t be reflected in the requisition line item.

Here’s a couple of ideas on how we could approach the problem:

  • We could create a new endpoint for updating the ProgramOrderables only, without touching the Orderable itself as proposed by Paweł Gesek here.

  • We could use the FHIR approach by versioning the Orderable. This approach would use two IDs, one for tracking the Orderable itself, so the ID of Orderable would never change and a second for tracking the Orderable version.

    This would allow us to:

    • Retrieve the newest version of Orderable simply by calling a GET endpoint with the ID. This would be used by the administration screens so they always display the newest Orderable. Modifying the Orderable would result in creation of a new version of it, and storing the previous one as history.
    • All the module that would require the older version of the Orderable would simply pass the version ID as a request parameter to the GET endpoint. This would result in retrieving a history version of an Orderable instead of the newest one.
      If you have any ideas, comments or concerns about this feature, please post them in this thread.

Best Regards,

Nikodem

**

SolDevelo** Sp. z o.o. [LLC] /
www.soldevelo.com

Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland

Phone: +48 58 782 45 40 / Fax: +48 58 782 45 41

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to
openlmis-dev@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/openlmis-dev/6ce639ba-ac8e-48df-a201-3f3d4455316a%40googlegroups.com
.

For more options, visit https://groups.google.com/d/optout.

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to
openlmis-dev@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/openlmis-dev/FC141E2F-3489-42F6-B0AB-1C5577AEC431%40villagereach.org
.

For more options, visit https://groups.google.com/d/optout.


(Sebastian Brudziński) #4

Hi,

just a quick comment, Nikodem.

  When updating an orderable to remove one of the program-orderables association, the UI currently fails to load all requisitions that were initiated at the time the removed program-orderable was still supported (It hangs at the step that tries to find correct program on the program-orderables list). This is one of the major thing that breaks when editing an Orderable at the moment. Versioning of the Orderables (read/vread) would potentially fix this problem as well as made sure the orderables are immute at the initiate (by additionally storing and using the version of an orderable).

Best regards,

  Sebastian.
···

On 29.09.2017 19:25, Josh Zamor wrote:

  Quick correction, I typed “FHIR proposal” when what I meant to say is that the idea is to borrow a pattern seen in FHIR.  I’ve not proposed FHIR at this point for our Orderable representations.

On Sep 29, 2017, at 10:22 AM, Josh Zamor <josh.zamor@villagereach.org > wrote:

Thanks for getting this started here Nikodem,

              For reference the ticket, including some initial discussion and a proposed pattern is located here:  [https://openlmis.atlassian.net/browse/OLMIS-3167](https://openlmis.atlassian.net/browse/OLMIS-3167)
              I don’t think we’re on the same page here, if I’m reading this as you intended:
                    I've done some research and I could only find one thing that would break after we implement Orderables edit. Requisition line item uses the ProgramOrderable to actually set the price per pack of itself. If we modify the ProgramsOrderable price per pack after the requisition initialization it won't be reflected in the requisition line item.

The functionality that is desired is that if you change the price per pack of the Orderable for a program today, it should not change the price of that Orderable on a line item in a Requisition or Order that has already been created. e.g. if a Requisition was created 2 years ago for a price of X, and the price today of line items has increased by 25%, we wouldn’t expect the price in that 2 year old Requisition to suddenly increase. It should rather stay the same price as when the Requisition was initiated, and this should be immutable . This is true for the rest of the fields in Orderable and Program Orderable

I also noticed a couple things:

  •                   The link you included for Pawel’s idea of building a new API for just changing Program Orderables appears to go to an unrelated code review.
    
    •                       We’ve expanded the topic to include updating Orderables, not just solving for updating Program Orderables.
      
  •                   Thanks for linking to the FHIR proposal, for background for people considering I have related points in the [issue](https://openlmis.atlassian.net/browse/OLMIS-3167?focusedCommentId=44924&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-44924).
    
              Let’s make sure we’re on the same page about desired functionality, and then take a crack at evaluating what would break, including expected functionality.

Best,

Josh

                    On Sep 29, 2017, at 9:09 AM, Nikodem Graczewski <ngraczewski@soldevelo.com                        > wrote:

Hi everyone,

                      as per our Sprint Q&A meeting I'm starting a discussion about how do we want to approach the Orderables edit as allowing this action can possibly result is some unforeseen consequences in some of the modules depending on them.



                      Here's a list of modules that are somehow related to Orderables:
  •                           referencedata - holds the Orderables
    
  • requisition

  • fulfillment

  • stockmanagement
    I’ve done some research and I could only find one thing that would break after we implement Orderables edit. Requisition line item uses the ProgramOrderable to actually set the price per pack of itself. If we modify the ProgramsOrderable price per pack after the requisition initialization it won’t be reflected in the requisition line item.

                          Here's a couple of ideas on how we could approach the problem:
    
  •                             We could create a new endpoint for updating the ProgramOrderables only, without touching the Orderable itself as proposed by Paweł Gesek [here](https://review.openlmis.org/cru/FEOLMIS-1960#details).
    
  • We could use the FHIR approach by versioning the Orderable. This approach would use two IDs, one for tracking the Orderable itself, so the ID of Orderable would never change and a second for tracking the Orderable version.

                                This would allow us to:
    
    •                                 Retrieve the newest version of Orderable simply by calling a GET endpoint with the ID. This would be used by the administration screens so they always display the newest Orderable. Modifying the Orderable would result in creation of a new version of it, and storing the previous one as history.
      
    •                                 All the module that would require the older version of the Orderable would simply pass the version ID as a request parameter to the GET endpoint. This would result in retrieving a history version of an Orderable instead of the newest one.
                          If you have any ideas, comments or concerns about this feature, please post them in this thread.
      
      
      
                          Best Regards,
      
                          Nikodem
      
                    **![](http://www.soldevelo.com/sites/default/files/Soldevelo_logo_EPS_CMYK.png)

                        SolDevelo** Sp. z o.o. [LLC] / [
                        www.soldevelo.com](http://www.soldevelo.com/)

                      Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland

                      Phone:                             +48 58 782 45 40 / Fax:                             +48 58 782 45 41

                    You received this message because you are subscribed to the Google Groups "OpenLMIS Dev" group.

                    To unsubscribe from this group and stop receiving emails from it, send an email to openlmis-dev+unsubscribe@googlegroups.com.

                    To post to this group, send email to
                      openlmis-dev@googlegroups.com.

                    To view this discussion on the web visit [

https://groups.google.com/d/msgid/openlmis-dev/6ce639ba-ac8e-48df-a201-3f3d4455316a%40googlegroups.com](https://groups.google.com/d/msgid/openlmis-dev/6ce639ba-ac8e-48df-a201-3f3d4455316a%40googlegroups.com?utm_medium=email&utm_source=footer).

                    For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

          You received this message because you are subscribed to the Google Groups "OpenLMIS Dev" group.

          To unsubscribe from this group and stop receiving emails from it, send an email to openlmis-dev+unsubscribe@googlegroups.com.

          To post to this group, send email to
            openlmis-dev@googlegroups.com.

          To view this discussion on the web visit [

https://groups.google.com/d/msgid/openlmis-dev/FC141E2F-3489-42F6-B0AB-1C5577AEC431%40villagereach.org](https://groups.google.com/d/msgid/openlmis-dev/FC141E2F-3489-42F6-B0AB-1C5577AEC431%40villagereach.org?utm_medium=email&utm_source=footer).

          For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

  You received this message because you are subscribed to the Google Groups "OpenLMIS Dev" group.

  To unsubscribe from this group and stop receiving emails from it, send an email to openlmis-dev+unsubscribe@googlegroups.com.

  To post to this group, send email to openlmis-dev@googlegroups.com.

  To view this discussion on the web visit [https://groups.google.com/d/msgid/openlmis-dev/B61C618B-A25C-4F3B-89FE-909E3D7C8104%40villagereach.org](https://groups.google.com/d/msgid/openlmis-dev/B61C618B-A25C-4F3B-89FE-909E3D7C8104%40villagereach.org?utm_medium=email&utm_source=footer).

  For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).


Sebastian Brudziński

    Software Developer / Team Leader


SolDevelo
Sp. z o.o. [LLC] / www.soldevelo.com
Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland
Phone: +48 58 782 45 40 / Fax: +48 58 782 45 41
sbrudzinski@soldevelo.com


(Nikodem Graczewski) #5

Hello Everyone,

first of all I would like to thank Josh for the clarification. I’ve made some digging and here’s the list of domain classes that has a reference to the Orderable and would probably have to be updated:

  • direct DB relation
  • FacilityTypeApprovedProduct
  • ProgramOrderable
  • ID of the orderable
  • OrderLineItem
  • RequisitionLineItem
  • StockCard
  • VmmApplicable interface
  • StockEventLineItem
  • IdentifiableByOrderableLot interface
  • OrderableLotIdentity
  • PhysicalInventoryLineItem
    I was thinking how we should include the version ID in the classes above and here are three solutions I would like to propose:
  1. Leave the ID of the orderable table unique, so it can be used for fetching them from the referencedata the same way we do it now. This would require adding a separate ID for tracking entries that relate to the same orderable. This would probably require less effort as we wouldn’t have to modify all the domain classes mention above.
  2. Add a version ID to the orderable and modify all the mentioned classes to actually have information about both orderableId and orderableVersionId. This would be the exact behavior proposed by Josh.
  3. Add a version ID to the orderable and make it table unique so we could call the GET /orderables?versionId endpoint and don’t have to keep both orderableId and orderableVersionId in the mentioned classes.
    Best regards,
    Nikodem
···

On Monday, October 2, 2017 at 11:08:34 AM UTC+2, Sebastian Brudziński wrote:

Hi,

just a quick comment, Nikodem.

  When updating an orderable to remove one of the program-orderables association, the UI currently fails to load all requisitions that were initiated at the time the removed program-orderable was still supported (It hangs at the step that tries to find correct program on the program-orderables list). This is one of the major thing that breaks when editing an Orderable at the moment. Versioning of the Orderables (read/vread) would potentially fix this problem as well as made sure the orderables are immute at the initiate (by additionally storing and using the version of an orderable).

Best regards,

  Sebastian.

On 29.09.2017 19:25, Josh Zamor wrote:

  Quick correction, I typed “FHIR proposal” when what I meant to say is that the idea is to borrow a pattern seen in FHIR.  I’ve not proposed FHIR at this point for our Orderable representations.

On Sep 29, 2017, at 10:22 AM, Josh Zamor <josh....@villagereach.org > wrote:

Thanks for getting this started here Nikodem,

              For reference the ticket, including some initial discussion and a proposed pattern is located here:  [https://openlmis.atlassian.net/browse/OLMIS-3167](https://openlmis.atlassian.net/browse/OLMIS-3167)
              I don’t think we’re on the same page here, if I’m reading this as you intended:
                    I've done some research and I could only find one thing that would break after we implement Orderables edit. Requisition line item uses the ProgramOrderable to actually set the price per pack of itself. If we modify the ProgramsOrderable price per pack after the requisition initialization it won't be reflected in the requisition line item.

The functionality that is desired is that if you change the price per pack of the Orderable for a program today, it should not change the price of that Orderable on a line item in a Requisition or Order that has already been created. e.g. if a Requisition was created 2 years ago for a price of X, and the price today of line items has increased by 25%, we wouldn’t expect the price in that 2 year old Requisition to suddenly increase. It should rather stay the same price as when the Requisition was initiated, and this should be immutable . This is true for the rest of the fields in Orderable and Program Orderable

I also noticed a couple things:

  •                   The link you included for Pawel’s idea of building a new API for just changing Program Orderables appears to go to an unrelated code review.
    
    •                       We’ve expanded the topic to include updating Orderables, not just solving for updating Program Orderables.
      
  •                   Thanks for linking to the FHIR proposal, for background for people considering I have related points in the [issue](https://openlmis.atlassian.net/browse/OLMIS-3167?focusedCommentId=44924&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-44924).
    
              Let’s make sure we’re on the same page about desired functionality, and then take a crack at evaluating what would break, including expected functionality.

Best,

Josh

                    On Sep 29, 2017, at 9:09 AM, Nikodem Graczewski <ngrac...@soldevelo.com                        > wrote:

Hi everyone,

                      as per our Sprint Q&A meeting I'm starting a discussion about how do we want to approach the Orderables edit as allowing this action can possibly result is some unforeseen consequences in some of the modules depending on them.



                      Here's a list of modules that are somehow related to Orderables:
  •                           referencedata - holds the Orderables
    
  • requisition

  • fulfillment

  • stockmanagement
    I’ve done some research and I could only find one thing that would break after we implement Orderables edit. Requisition line item uses the ProgramOrderable to actually set the price per pack of itself. If we modify the ProgramsOrderable price per pack after the requisition initialization it won’t be reflected in the requisition line item.

                          Here's a couple of ideas on how we could approach the problem:
    
  •                             We could create a new endpoint for updating the ProgramOrderables only,                                 without touching the Orderable itself as proposed by Paweł Gesek [here](https://review.openlmis.org/cru/FEOLMIS-1960#details).
    
  • We could use the FHIR approach by versioning the Orderable. This approach would use two IDs, one for tracking the Orderable itself, so the ID of Orderable would never change and a second for tracking the Orderable version.

                                This would allow us to:
    
    •                                 Retrieve the newest version of Orderable simply by calling a GET endpoint with the ID. This would be used by the administration screens so they always display the newest Orderable. Modifying the Orderable would result in creation of a new version of it, and storing the previous one as history.
      
    •                                 All the module that would require the older version of the Orderable would simply pass the version ID as a request parameter to the GET endpoint. This would result in retrieving a history version of an Orderable instead of the newest one.
                          If you have any ideas, comments or concerns about this feature, please post them in this thread.
      
      
      
                          Best Regards,
      
                          Nikodem
      
                    **<img src="https://lh3.googleusercontent.com/proxy/Kq7icsK3MEQjYLwBW84HwuYjQ8aFuyyirMOzt_ENZ5BgyyRVaFVgsbO-vnqZPEKtkcm1Gs2mKJSDSi59adej7wSt1KiO3u5QJa2SfrMvGRh8cyONHJScEXiljA=w5000-h5000" height="35" width="145">

                        SolDevelo** Sp. z o.o. [LLC] / [
                        www.soldevelo.com](http://www.soldevelo.com/)

                      Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland

                      Phone:                             +48 58 782 45 40 / Fax:                             +48 58 782 45 41

                    You received this message because you are subscribed to the Google Groups "OpenLMIS Dev" group.

                    To unsubscribe from this group and stop receiving emails from it, send an email to openlmis-dev...@googlegroups.com.

                    To post to this group, send email to
              ...@googlegroups.com.

                    To view this discussion on the web visit [

https://groups.google.com/d/msgid/openlmis-dev/6ce639ba-ac8e-48df-a201-3f3d4455316a%40googlegroups.com](https://groups.google.com/d/msgid/openlmis-dev/6ce639ba-ac8e-48df-a201-3f3d4455316a%40googlegroups.com?utm_medium=email&utm_source=footer).

                    For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

          You received this message because you are subscribed to the Google Groups "OpenLMIS Dev" group.

          To unsubscribe from this group and stop receiving emails from it, send an email to openlmis-dev...@googlegroups.com.

          To post to this group, send email to
         ...@googlegroups.com.

          To view this discussion on the web visit [

https://groups.google.com/d/msgid/openlmis-dev/FC141E2F-3489-42F6-B0AB-1C5577AEC431%40villagereach.org](https://groups.google.com/d/msgid/openlmis-dev/FC141E2F-3489-42F6-B0AB-1C5577AEC431%40villagereach.org?utm_medium=email&utm_source=footer).

          For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

  You received this message because you are subscribed to the Google Groups "OpenLMIS Dev" group.

  To unsubscribe from this group and stop receiving emails from it, send an email to openlmis-dev...@googlegroups.com.

  To post to this group, send email to openlm...@googlegroups.com.

  To view this discussion on the web visit [https://groups.google.com/d/msgid/openlmis-dev/B61C618B-A25C-4F3B-89FE-909E3D7C8104%40villagereach.org](https://groups.google.com/d/msgid/openlmis-dev/B61C618B-A25C-4F3B-89FE-909E3D7C8104%40villagereach.org?utm_medium=email&utm_source=footer).

  For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).


Sebastian Brudziński

    Software Developer / Team Leader


     sbrud...@soldevelo.com


SolDevelo
Sp. z o.o. [LLC] / www.soldevelo.com
Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland
Phone: +48 58 782 45 40 / Fax: +48 58 782 45 41


(Brandon Bowersox-Johnson) #6

Hi All,

The conversation quickly moved to ‘versioning the orderables’. There are cool ideas to do that using FHIR read and vread in the ticket*.

The ticket* makes the point that the ability to edit and update Orderables used to work in v3*!* Of course we did not have ‘versioning of the Orderables’ or FHIR support to do that.

So is there a simpler way to go back and get that feature working again? Is that a smaller baby step we can take to fix the broken functionality before we take the larger step to versioning and FHIR?

The versioning and FHIR work is probably too large to introduce before 3.2.1 (release is only 3 weeks away).

-Brandon


(Josh Zamor) #7

Hi Brandon,

A quick clarification: when I said it did work, I meant that the simple action worked - not that the overall user expectations would be met. Early on we had an intention of using a snapshotting approach in client services. This is the approach used in previous versions of OpenLMIS, however we have identified two issues:

  * In v3 we have yet to fully snapshot the Orderable and associated entities: see for example this requisition table<http://ci.openlmis.org/erd-requisition/requisition/tables/requisition_line_items.html> doesn’t snapshot things like Orderable.packSize.
  * Not only have we not yet fully achieved snapshotting in Requisition or other services, I now think it’s less desirable to strive for this if one of our performance goals is to help Orderables, and other infrequently changed meta (reference) data, be cached by clients. Snapshotting bloats the size of things like Requisition line items, and can’t take advantage of pre-caching reference data. In versions prior to v3, we never went below the district and so this bloat was, presumably, more acceptable. With our goal of reaching more network limited users, I don’t believe we should invest more in the snapshotting approach.

I do think if we’re going to implement any code changes, that we need to make sure we meet the overall user expectations.

One of the things we talked about on the call, that I said I’d repost here, is that some OpenLMIS concepts do have an expectation that the Orderable they reference never changes (immutable), and other’s have the opposite expectation. The example from this morning’s tech call:

  * The Requisition workflow does have the expectation that an Orderable (and the FTAP, ProgramOrderable, etc), once used in the Requisition’s initiate process, never changes. Additionally when a Requisition is turned into an Order, the Order placed should be for the same Orderable that the Requisition used.
  * A concept such as FTAP, doesn’t place the same expectation on an Orderable. If a user creates an FTAP, they’re intending to say that an Orderable at a facility for a program has certain additional characteristics, currently: max, min and emergency order point for stock. Put another way, if an Orderable’s packSize changes, we don’t want to force user’s to create new FTAPs for that new Orderable as FTAP doesn’t have the same expectation of Orderables as a Requisition does.
  * We didn’t discuss, but also I believe that a Stock Card doesn’t have an expectation that an Orderable never changes - if an Orderable’s code or packSize changes, it shouldn’t effect the Stock Card’s function of tracking stock quantity.

The next steps for gathering the requirements would be to extend the above reasoning to all the other concepts we’ve identified thus far that would be effected by an Orderable changing.

I also agree that using a versioned type approach, as I proposed, would take considerable work that likely wouldn’t make it in a v3.2.1 release in a few weeks here. I’d love to hear some other ideas - so long as we’re setting ourselves up to meet user expectations and also which aligns with our performance goals related to caching, reducing bloat, and ultimately reaching beyond the district’s network connectivity.

Best,
Josh

···

On Oct 3, 2017, at 7:43 PM, Brandon Bowersox-Johnson <brandon.bowersox-johnson@villagereach.org<mailto:brandon.bowersox-johnson@villagereach.org>> wrote:

Hi All,

The conversation quickly moved to ‘versioning the orderables’. There are cool ideas to do that using FHIR read and vread in the ticket*.

The ticket* makes the point that the ability to edit and update Orderables used to work in v3! Of course we did not have ‘versioning of the Orderables’ or FHIR support to do that.

* https://openlmis.atlassian.net/browse/OLMIS-3167

So is there a simpler way to go back and get that feature working again? Is that a smaller baby step we can take to fix the broken functionality before we take the larger step to versioning and FHIR?

The versioning and FHIR work is probably too large to introduce before 3.2.1 (release is only 3 weeks away).

-Brandon

--
You received this message because you are subscribed to the Google Groups "OpenLMIS Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openlmis-dev+unsubscribe@googlegroups.com<mailto:openlmis-dev+unsubscribe@googlegroups.com>.
To post to this group, send email to openlmis-dev@googlegroups.com<mailto:openlmis-dev@googlegroups.com>.
To view this discussion on the web visit https://groups.google.com/d/msgid/openlmis-dev/8A6AB6B9-DB2D-4586-9C66-738877F05665%40villagereach.org<https://groups.google.com/d/msgid/openlmis-dev/8A6AB6B9-DB2D-4586-9C66-738877F05665%40villagereach.org?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.


(Nikodem Graczewski) #8

Hello everyone!

As Josh has covered most of the cases here are my two cents:

  • VvmApplicable - it doesn’t really care about the details of the orderable as long as the orderable represents the same product,
  • IdentifiableByOrderableLot - same as above, if we are looking for a stock card or stock even line item we don’t really care about the packSize or code, all we care is that it’s the same product
  • PhysicalInventoryLineItem - it don’t expect the orderable to change, want to keep track of how many strips of X tabs we have

FacilityTypeApprovedProduct, ProgramOrderable, OrderLineItem, RequisitionLineItem and StockEventLineItem have already been covered.

The other thing I would like to bring up is what fields should be editable. Do we want to make the whole orderable mutable, or do we want to leave some of the fields, like name, immutable? Łukasz came with an excellent edge case for allowing name change. Let’s say we have a product “X” that has been used for some time but it has been discontinued. Instead we have a new product like “X Maxx” that is an improved version of the “X” product. The edge case would be a person responsible for managing orderables deciding that, instead of adding new orderable, editing an existing orderable would be a better idea. This action would break the FTAP as orderable would describe a different product, this action would break all the concepts that wouldn’t use the orderable versioning. In my opinion the best way do deal with it would be making some of the fields immutable so we make sure that orderables with the same ID always describe the same product.

Best regards,
Nikodem

···

On Wednesday, October 4, 2017 at 7:36:25 AM UTC+2, Josh Zamor wrote:

Hi Brandon,

A quick clarification: when I said it did work, I meant that the simple action worked - not that the overall user expectations would be met. Early on we had an intention of using a snapshotting approach in client services. This is the approach used in previous versions of OpenLMIS, however we have identified two issues:

  • In v3 we have yet to fully snapshot the Orderable and associated entities: see for example this requisition table doesn’t snapshot things like Orderable.packSize.
  • Not only have we not yet fully achieved snapshotting in Requisition or other services, I now think it’s less desirable to strive for this if one of our performance goals is to help Orderables, and other infrequently changed meta (reference) data, be cached by clients. Snapshotting bloats the size of things like Requisition line items, and can’t take advantage of pre-caching reference data. In versions prior to v3, we never went below the district and so this bloat was, presumably, more acceptable. With our goal of reaching more network limited users, I don’t believe we should invest more in the snapshotting approach.

I do think if we’re going to implement any code changes, that we need to make sure we meet the overall user expectations.

One of the things we talked about on the call, that I said I’d repost here, is that some OpenLMIS concepts do have an expectation that the Orderable they reference never changes (immutable), and other’s have the opposite expectation. The example from this morning’s tech call:

  • The Requisition workflow does have the expectation that an Orderable (and the FTAP, ProgramOrderable, etc), once used in the Requisition’s initiate process, never changes. Additionally when a Requisition is turned into an Order, the Order placed should be for the same Orderable that the Requisition used.
  • A concept such as FTAP, doesn’t place the same expectation on an Orderable. If a user creates an FTAP, they’re intending to say that an Orderable at a facility for a program has certain additional characteristics, currently: max, min and emergency order point for stock. Put another way, if an Orderable’s packSize changes, we don’t want to force user’s to create new FTAPs for that new Orderable as FTAP doesn’t have the same expectation of Orderables as a Requisition does.
  • We didn’t discuss, but also I believe that a Stock Card doesn’t have an expectation that an Orderable never changes - if an Orderable’s code or packSize changes, it shouldn’t effect the Stock Card’s function of tracking stock quantity.

The next steps for gathering the requirements would be to extend the above reasoning to all the other concepts we’ve identified thus far that would be effected by an Orderable changing.

I also agree that using a versioned type approach, as I proposed, would take considerable work that likely wouldn’t make it in a v3.2.1 release in a few weeks here. I’d love to hear some other ideas - so long as we’re setting ourselves up to meet user expectations and also which aligns with our performance goals related to caching, reducing bloat, and ultimately reaching beyond the district’s network connectivity.

Best,

Josh

On Oct 3, 2017, at 7:43 PM, Brandon Bowersox-Johnson brandon.bowe...@villagereach.org wrote:

Hi All,

The conversation quickly moved to ‘versioning the orderables’. There are cool ideas to do that using FHIR read and vread in the ticket*.

The ticket* makes the point that the ability to edit and update Orderables used to work in v3*!* Of course we did not have ‘versioning of the Orderables’ or FHIR support to do that.

So is there a simpler way to go back and get that feature working again? Is that a smaller baby step we can take to fix the broken functionality before we take the larger step to versioning and FHIR?

The versioning and FHIR work is probably too large to introduce before 3.2.1 (release is only 3 weeks away).

-Brandon

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to openlmis-dev...@googlegroups.com.

To post to this group, send email to openlm...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openlmis-dev/8A6AB6B9-DB2D-4586-9C66-738877F05665%40villagereach.org.

For more options, visit https://groups.google.com/d/optout.


(Brandon Bowersox-Johnson) #9

Josh, thanks for that. The explanation totally makes sense to me. I agree that we should proceed with planning this right and doing the work after 3.2.1.

-Brandon

···

From: Josh Zamor josh.zamor@villagereach.org

Date: Tuesday, October 3, 2017 at 10:36 PM

To: Brandon Bowersox-Johnson brandon.bowersox-johnson@villagereach.org

Cc: OpenLMIS Dev openlmis-dev@googlegroups.com, Mary Jo Kochendorfer maryjo.kochendorfer@villagereach.org

Subject: Re: [openlmis-dev] Orderable edit design

Hi Brandon,

A quick clarification: when I said it did work, I meant that the simple action worked - not that the overall user expectations would be met. Early on we had an intention of using a snapshotting approach in client services. This is the approach used in previous versions of OpenLMIS, however we have identified two issues:

  • In v3 we have yet to fully snapshot the Orderable and associated entities: see for example this requisition table doesn’t snapshot things like Orderable.packSize.

  • Not only have we not yet fully achieved snapshotting in Requisition or other services, I now think it’s less desirable to strive for this if one of our performance goals is to help Orderables, and other infrequently changed meta (reference) data, be cached by clients. Snapshotting bloats the size of things like Requisition line items, and can’t take advantage of pre-caching reference data. In versions prior to v3, we never went below the district and so this bloat was, presumably, more acceptable. With our goal of reaching more network limited users, I don’t believe we should invest more in the snapshotting approach.

I do think if we’re going to implement any code changes, that we need to make sure we meet the overall user expectations.

One of the things we talked about on the call, that I said I’d repost here, is that some OpenLMIS concepts do have an expectation that the Orderable they reference never changes (immutable), and other’s have the opposite expectation. The example from this morning’s tech call:

  • The Requisition workflow does have the expectation that an Orderable (and the FTAP, ProgramOrderable, etc), once used in the Requisition’s initiate process, never changes. Additionally when a Requisition is turned into an Order, the Order placed should be for the same Orderable that the Requisition used.

  • A concept such as FTAP, doesn’t place the same expectation on an Orderable. If a user creates an FTAP, they’re intending to say that an Orderable at a facility for a program has certain additional characteristics, currently: max, min and emergency order point for stock. Put another way, if an Orderable’s packSize changes, we don’t want to force user’s to create new FTAPs for that new Orderable as FTAP doesn’t have the same expectation of Orderables as a Requisition does.

  • We didn’t discuss, but also I believe that a Stock Card doesn’t have an expectation that an Orderable never changes - if an Orderable’s code or packSize changes, it shouldn’t effect the Stock Card’s function of tracking stock quantity.

The next steps for gathering the requirements would be to extend the above reasoning to all the other concepts we’ve identified thus far that would be effected by an Orderable changing.

I also agree that using a versioned type approach, as I proposed, would take considerable work that likely wouldn’t make it in a v3.2.1 release in a few weeks here. I’d love to hear some other ideas - so long as we’re setting ourselves up to meet user expectations and also which aligns with our performance goals related to caching, reducing bloat, and ultimately reaching beyond the district’s network connectivity.

Best,

Josh


(Nikodem Graczewski) #10

Hello everyone!

I would like to restart the orderables edit discussion. In order to do it I’ve created the following recap on the problem and proposed a solution for dealing with it.

The Problem

Making Orderables editable to allow users to fix typos and pricing adjustments.

The Pain Points

Currently we have some classes that have the associations on the orderables. We can split them into two groups that:

  • expect orderables to be immutable. These classes care about the orderable details and don’t expect the orderable to change. Here’s a list of the classes that behave this way:
  • RequisitionLineItem
  • OrderLineItem
  • don’t care if orderable has changed. All they care that the object with the specific UUID still represents the same orderable, meaning the identity of the orderable hasn’t changed. Here’s a list of class that meet this condition:
  • FacilityTypeApprovedProduct
  • VmmApplicable
  • IdentifiableByOrderableLot
  • PhysicalInventoryLineItem
  • StockCard

What should be editable?

The main concern behind this question is that allowing the users to update the orderable could lead to an orderable object actually changing identity, meaning it would represent a different orderable than it did originally. The solution for this problem would be making the product code immutable. This would enforce that the orderable can not change its identity. If the user would like to change the code she/he would most likely be doing something she/he shouldn’t. As the problem we’re trying to solve here is rather about fixing the errors and updating the price this most likely won’t be a problem, but he possibility would still remain.

By the way, this concern is most likely also valid for any other entity that is mutable in the OpenLMIS.

The Solutions

I would like to propose three possible solutions I see from which I have chosen one. I will describe them in more details below

Archiving

With this approach we would be archiving the orderable before updating it and creating a new entry within the database giving it a new UUID.

Pros:

  • No changes would have to be made to the entities that treat the orderable as immutable, the UUID would point to the exact orderable version they want.

Cons:

  • We would have to update entities that treat orderable as mutable on every update as they don’t really care about orderable details but about its identity (meaning they should point to the newest possible version of the orderable). FTAP is a great example here. In Malawi implementation there is around 60 FTAPs for a single orderable, which would have to be updated/recreated every time we update the orderable. If we picked up the recreation path the orderables table in the database could grow really huge really fast. We would also have to deal with entities that associate the orderable and update them too, and this could propagate.
  • This would be hard to maintain (see the previous bullet point).
  • The database constraints would be non-trivial, we would have to restrict the situation when there is a couple of orderables that have the same product code and are not archived (this wouldn’t be true for archived orderables).

Open questions:

  • How would we track the orderable identity if the UUID could change?
    Product code is the natural candidate for that.

  • How would we track the newest orderable version?
    We would probably have to introduce the archived flag, which should be set to false for the newest version of the orderable.

Versioning

With this approach we would add a new field to the orderable named version/versionId/something similar. The identity of the orderable would still be held by the UUID attribute.

Pros:

  • No changes need outside the endpoint during the initial implementation.
  • No changes needed at all to the entities that treat the orderable as mutable, the UUID would still point to the newest version.
  • All versions stored in one place (referencedata), no need for snapshots in any module - all they would need would be the UUID and version.
  • We’re not breaking backward combability, the currently implemented endpoint still returns the latest orderable version based on the UUID. To get the older version of the orderable we would have to pass an optional request parameter. This would lead to the endpoint being extended rather than modified.

Cons:

  • The need to identify and update all the places that use orderables and expect them to be immutable.

Open questions:

  • Should we use GUUID or int to track version?
    By using a simple int we could easily retrieve the latest version of the orderable from the database.

  • Where should we store the version in object referencing a specific version of the orderable?
    ReferencedataObject class is a perfect candidate for that.

  • Would this break the contract and lead to the major version being increased?
    The endpoint itself stays the same, but introduction of the versioning would probably lead to the behavior change meaning this would most likely break the contract and require a major version bump.

  • Which changes should spawn new version?
    Ideally, typo fixes should not lead to a version bump, but those things might be hard to detect. We could give the users ability to decide whether the update should spawn a new version of the orderable or update the newest one, but this feature would most likely confuse the end user.

Snapshooting

With this approach we would be snapshooting properties of the orderable which are expected to be immutable by the associating classes.

Pros:

  • Easy to implement.
  • This would break some dependencies leading to a lesser coupling of the fulfillment/requisition and referencedata modules at the cost of code duplication.
  • This would lead to a temporary performance improvement (less calls to the referencedata i.e. during requisition retrieval)…

Cons:

  • …which would become a hit to the performance in the long run (bigger database).
  • This would be a little WET as we would be duplicating some of the properties, but we could snapshot only those fields we want immutable and ignore orderable beyond the point of creation of the object that was snapshooting the orderable.

The Proposed Solution

I would like to propose Versioning as the go-to solution for this problem. It has the most pros with only one con, which is true for all the solutions - eventually, we will have to change something somewhere to make orderables editable flawlessly. This approach would also require least work outside the orderable endpoint. I’ve diagnosed only two places that expect the orderable to be immutable, which are requisition and order creation, rest of the associations rely on the identity only and don’t really care about the orderable details changing.

The Proposed Strategy

This is a proposed strategy for implementing the orderable edit feature. These steps should make the implementation as smooth as possible.

  1. Add the version property to the orderable and modify endpoint to respect it. Create a migration to set the version to 0 by default. Don’t implement orderable edit feature yet.
  2. Update the Requisition and Order creation and classes to actually use ReferencedataObject instead of an ID, the ReferencedataObject should be extended to include the version property (VersionedReferencedataObject class perhaps?). This step could also include removal of the snapshoot properties from the RequisitionLineItem, as the specific version of the orderable wouldn’t be editable (a little DRY is always good).
  3. Implement the orderable edit feature with automated version updating.
  4. Watch everything going smooth.

The Concern

Implementing edit feature on an entity that is treated both as mutable and immutable depending on the associating class will leave a gate for bugs to creep in that might be hard to deal with. My main concern is a situation in which user decides that editing a discontinued orderable and overriding it with a new one is a good idea. This means that the identity of the object has changed and I couldn’t come up with a way to prevent it other than making some of the fields immutable.

Kind regards,
Nikodem

···

On Friday, September 29, 2017 at 6:09:15 PM UTC+2, Nikodem Graczewski wrote:

Hi everyone,

as per our Sprint Q&A meeting I’m starting a discussion about how do we want to approach the Orderables edit as allowing this action can possibly result is some unforeseen consequences in some of the modules depending on them.

Here’s a list of modules that are somehow related to Orderables:

  • referencedata - holds the Orderables
  • requisition
  • fulfillment
  • stockmanagement
    I’ve done some research and I could only find one thing that would break after we implement Orderables edit. Requisition line item uses the ProgramOrderable to actually set the price per pack of itself. If we modify the ProgramsOrderable price per pack after the requisition initialization it won’t be reflected in the requisition line item.

Here’s a couple of ideas on how we could approach the problem:

  • We could create a new endpoint for updating the ProgramOrderables only, without touching the Orderable itself as proposed by Paweł Gesek here.
  • We could use the FHIR approach by versioning the Orderable. This approach would use two IDs, one for tracking the Orderable itself, so the ID of Orderable would never change and a second for tracking the Orderable version.
    This would allow us to:
  • Retrieve the newest version of Orderable simply by calling a GET endpoint with the ID. This would be used by the administration screens so they always display the newest Orderable. Modifying the Orderable would result in creation of a new version of it, and storing the previous one as history.
  • All the module that would require the older version of the Orderable would simply pass the version ID as a request parameter to the GET endpoint. This would result in retrieving a history version of an Orderable instead of the newest one.
    If you have any ideas, comments or concerns about this feature, please post them in this thread.

Best Regards,
Nikodem


(Sebastian Brudziński) #11

Thanks for reviving this discussion and compiling the summary, Nikodem. I think it’s important that we start making progress with the orderables edit while we are still far from the next release, since the impact of this change will be huge.

  My comment on the "identity", under "what should be editable" and "the concern" would be that it is not something that we (as a core product) need or should worry about too much. No matter how hard we try, it's ultimately up to the implementer to ensure that the edits they are making, to any instance in the system, are not causing that instance to represent a different thing. If we wanted to enforce that on our end, we would pretty much need to disable the ability to edit most of the properties for all our entities (eg. a name, a code...).

  You are proposing that we make the product code immutable. I'd personally tend towards giving more, rather than less freedom in the edits. While the code is something that generally shouldn't change, we already had an issue with an incorrect facility code assigned in Malawi. I believe this will/should be the only case where a code of any instance is edited, but it's good to have that option if necessary, especially that editing the code doesn't have any impact on the system as far as I know (references are by the id). It would be great to hear what other people think though.

  +1 from me for "versioning" as our strategy to edit orderables. I think it's the cleanest solution out of the proposed ones. It can also be implemented incrementally, without affecting other services (support for versioning first, migrating services one-by-one to use the orderable version second).

Best regards,

  Sebastian.
···

On 28.11.2017 19:07, Nikodem Graczewski wrote:

Hello everyone!

    I would like to restart the orderables edit discussion. In order to do it I've created the following recap on the problem and proposed a solution for dealing with it.

The Problem

      Making Orderables editable to allow users to fix typos and pricing adjustments.

The Pain Points

      Currently we have some classes that have the associations on the orderables. We can split them into two groups that:
  •           expect orderables to be immutable. These classes care about the orderable details and don't expect the orderable to change. Here's a list of the classes that behave this way:
    
  • RequisitionLineItem
  • OrderLineItem
  •           don't care if orderable has changed. All they care that the object with the specific UUID still represents the same orderable, meaning the identity of the orderable hasn't changed. Here's a list of class that meet this condition:
    
  • FacilityTypeApprovedProduct
  • VmmApplicable
  • IdentifiableByOrderableLot
  • PhysicalInventoryLineItem
  • StockCard

What should be editable?

      The main concern behind this question is that allowing the users to update the orderable could lead to an orderable object actually changing identity, meaning it would represent a different orderable than it did originally. The solution for this problem would be making the product code immutable. This would enforce that the orderable can not change its identity. If the user would like to change the code she/he would most likely be doing something she/he shouldn't. As the problem we're trying to solve here is rather about fixing the errors and updating the price this most likely won't be a problem, but he possibility would still remain.
      By the way, this concern is most likely also valid for any other entity that is mutable in the OpenLMIS.

The Solutions

      I would like to propose three possible solutions I see from which I have chosen one. I will describe them in more details below

Archiving

      With this approach we would be archiving the orderable before updating it and creating a new entry within the database giving it a new UUID.

Pros:

  •           No changes would have to be made to the entities that treat the orderable as immutable, the UUID would point to the exact orderable version they want.
    

Cons:

  •           We would have to update entities that treat orderable as mutable on every update as they don't really care about orderable details but about its identity (meaning they should point to the newest possible version of the orderable). FTAP is a great example here. In Malawi implementation there is around 60 FTAPs for a single orderable, which would have to be updated/recreated every time we update the orderable. If we picked up the recreation path the orderables table in the database could grow really huge really fast. We would also have to deal with entities that associate the orderable and update them too, and this could propagate.
    
  •           This would be hard to maintain (see the previous bullet point).
    
  •           The database constraints would be non-trivial, we would have to restrict the situation when there is a couple of orderables that have the same product code and are not archived (this wouldn't be true for archived orderables).
    

Open questions:

  •           How would we track the orderable identity if the UUID could change?
    
              Product code is the natural candidate for that.
    
  • How would we track the newest orderable version?

              We would probably have to introduce the archived flag, which should be set to false for the newest version of the orderable.
    

Versioning

      With this approach we would add a new field to the orderable named version/versionId/something similar. The identity of the orderable would still be held by the UUID attribute.

Pros:

  •           No changes need outside the endpoint during the initial implementation.
    
  •           No changes needed at all to the entities that treat the orderable as mutable, the UUID would still point to the newest version.
    
  •           All versions stored in one place (referencedata), no need for snapshots in any module - all they would need would be the UUID and version.
    
  •           We're not breaking backward combability, the currently implemented endpoint still returns the latest orderable version based on the UUID. To get the older version of the orderable we would have to pass an optional request parameter. This would lead to the endpoint being extended rather than modified.
    

Cons:

  •           The need to identify and update all the places that use orderables and expect them to be immutable.
    

Open questions:

  • Should we use GUUID or int to track version?

              By using a simple int we could easily retrieve the latest version of the orderable from the database.
    
  •           Where should we store the version in object referencing a specific version of the orderable?
    
              ReferencedataObject class is a perfect candidate for that.
    
  •           Would this break the contract and lead to the major version being increased?
    
              The endpoint itself stays the same, but introduction of the versioning would probably lead to the behavior change meaning this would most likely break the contract and require a major version bump.
    
  • Which changes should spawn new version?

              Ideally, typo fixes should not lead to a version bump, but those things might be hard to detect. We could give the users ability to decide whether the update should spawn a new version of the orderable or update the newest one, but this feature would most likely confuse the end user.
    

Snapshooting

      With this approach we would be snapshooting properties of the orderable which are expected to be immutable by the associating classes.

Pros:

  • Easy to implement.
  •           This would break some dependencies leading to a lesser coupling of the fulfillment/requisition and referencedata modules at the cost of code duplication.
    
  •           This would lead to a temporary performance improvement (less calls to the referencedata i.e. during requisition retrieval)...
    

Cons:

  •           ...which would become a hit to the performance in the long run (bigger database).
    
  •           This would be a little WET as we would be duplicating some of the properties, but we could snapshot only those fields we want immutable and ignore orderable beyond the point of creation of the object that was snapshooting the orderable.
    

The Proposed Solution

I would like to propose Versioning as the go-to solution for this problem. It has the most pros with only one con, which is true for all the solutions - eventually, we will have to change something somewhere to make orderables editable flawlessly. This approach would also require least work outside the orderable endpoint. I’ve diagnosed only two places that expect the orderable to be immutable, which are requisition and order creation, rest of the associations rely on the identity only and don’t really care about the orderable details changing.

The Proposed Strategy

      This is a proposed strategy for implementing the orderable edit feature. These steps should make the implementation as smooth as possible.
  1.           Add the version property to the orderable and modify endpoint to respect it. Create a migration to set the version to 0 by default. Don't implement orderable edit feature yet.
    
  2.           Update the Requisition and Order creation and classes to actually use ReferencedataObject instead of an ID, the ReferencedataObject should be extended to include the version property (VersionedReferencedataObject class perhaps?). This step could also include removal of the snapshoot properties from the RequisitionLineItem, as the specific version of the orderable wouldn't be editable (a little DRY is always good).
    
  3.           Implement the orderable edit feature with automated version updating.
    
  4. Watch everything going smooth.

The Concern

      Implementing edit feature on an entity that is treated both as mutable and immutable depending on the associating class will leave a gate for bugs to creep in that might be hard to deal with. My main concern is a situation in which user decides that editing a discontinued orderable and overriding it with a new one is a good idea. This means that the identity of the object has changed and I couldn't come up with a way to prevent it other than making some of the fields immutable.

Kind regards,

      Nikodem
    On Friday, September 29, 2017 at 6:09:15 PM UTC+2, Nikodem Graczewski wrote:

Hi everyone,

        as per our Sprint Q&A meeting I'm starting a discussion about how do we want to approach the Orderables edit as allowing this action can possibly result is some unforeseen consequences in some of the modules depending on them.



        Here's a list of modules that are somehow related to Orderables:
  • referencedata - holds the Orderables

  • requisition

  • fulfillment

  • stockmanagement
    I’ve done some research and I could only find one thing that would break after we implement Orderables edit. Requisition line item uses the ProgramOrderable to actually set the price per pack of itself. If we modify the ProgramsOrderable price per pack after the requisition initialization it won’t be reflected in the requisition line item.

            Here's a couple of ideas on how we could approach the problem:
    
  •               We could create a new endpoint for updating the ProgramOrderables only,                   without touching the Orderable itself as proposed by Paweł Gesek [here](https://review.openlmis.org/cru/FEOLMIS-1960#details).
    
  • We could use the FHIR approach by versioning the Orderable. This approach would use two IDs, one for tracking the Orderable itself, so the ID of Orderable would never change and a second for tracking the Orderable version.

                  This would allow us to:
    
  •                 Retrieve the newest version of Orderable simply by calling a GET endpoint with the ID. This would be used by the administration screens so they always display the newest Orderable. Modifying the Orderable would result in creation of a new version of it, and storing the previous one as history.
    
  •                 All the module that would require the older version of the Orderable would simply pass the version ID as a request parameter to the GET endpoint. This would result in retrieving a history version of an Orderable instead of the newest one.
            If you have any ideas, comments or concerns about this feature, please post them in this thread.
    
    
    
            Best Regards,
    
            Nikodem
    
  **![](http://www.soldevelo.com/sites/default/files/Soldevelo_logo_EPS_CMYK.png)

      SolDevelo** Sp. z o.o. [LLC] / [www.soldevelo.com](http://www.soldevelo.com)

    Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland

    Phone: +48 58 782 45 40 / Fax: +48 58 782 45 41

  --

  You received this message because you are subscribed to the Google Groups "OpenLMIS Dev" group.

  To unsubscribe from this group and stop receiving emails from it, send an email to openlmis-dev+unsubscribe@googlegroups.com.

  To post to this group, send email to openlmis-dev@googlegroups.com.

  To view this discussion on the web visit [https://groups.google.com/d/msgid/openlmis-dev/e423e6a1-d69c-4348-93de-fa5d8a619129%40googlegroups.com](https://groups.google.com/d/msgid/openlmis-dev/e423e6a1-d69c-4348-93de-fa5d8a619129%40googlegroups.com?utm_medium=email&utm_source=footer).

  For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).


Sebastian Brudziński

    Software Developer / Team Leader


SolDevelo
Sp. z o.o. [LLC] / www.soldevelo.com
Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland
Phone: +48 58 782 45 40 / Fax: +48 58 782 45 41
sbrudzinski@soldevelo.com


(Brandon Bowersox-Johnson) #12

Of the 3 proposed approaches, Versioning is probably also the one I would also lean towards if we want to make an Orderable editable. I agree with Sebastian that implementers should be allowed to change the code and name. In the case of a typo in the code, it is fine to change, and the rest of our software references the UUID and not the code so I believe it does not break anything. Same with changing an orderable name.

The 4 open questions about Versioning still need some proposals, so I have a few responses to those:

• “Should we use GUUID or int to track version?”

GUUID sounds more robust to me and just as easy. The newer version could store a reference to the previous version.

• “Which changes should spawn new version?”

I think any changes should always spawn a new version. That is a safer and simpler solution rather than having a complex logic where certain field edits do NOT change the version number while other field edits DO change the version number. We might as well always make each edit a new version. Any big downsides to that?

• “Where should we store the version in object referencing a specific version of the orderable?”

We need to be clear on whether the other parts of OpenLMIS that reference an Orderable are also storing a reference to a specific version of the Orderable. To take one specific example, I think the Requisition object might want to store a specific version reference for each orderable. At the time of requisition initiation, it builds the Requisition using the latest version of each Orderable from the FTAP list. It also saves that version used. Then, if an Orderable changes over time (maybe code, maybe name), if I view an old requisition from 5 years ago, the on-screen view will show me the name and code from the exact version that was used back then.

I also have a few other points:

• We will need guidance or hard rules on which Orderable properties are NOT allowed to be mutable. Can anyone name an example?

• For GS1 Trade Item-based Orderables, what are the GS1 GDSN rules about mutability? If Tylenol ever changes its PackSize, its UPC Barcode or changes name to “Tylenol Zero Sugar” must they issue a whole new Trade Item?

• For UNSPSC/GPC/Commodity-Type-based Orderables, do those systems ever change the name of something (like how “Autism” was changed to “Autism Spectrum Disorder” in the DSM-5)?

• Finally, for now I believe we are just talking about making an individual Orderable editable. As a separate issue (later) we will also talk about how the FTAP list should be editable. But that’s a separate topic for later. (Correct me if I’m wrong.)

···

From: openlmis-dev@googlegroups.com on behalf of Sebastian Brudziński sbrudzinski@soldevelo.com

Date: Wednesday, November 29, 2017 at 10:18 AM

To: "openlmis-dev@googlegroups.com" openlmis-dev@googlegroups.com

Subject: Re: [openlmis-dev] Re: Orderable edit design

Thanks for reviving this discussion and compiling the summary, Nikodem. I think it’s important that we start making progress with the orderables edit while we are still far from the next release, since the impact of this change will be huge.

My comment on the “identity”, under “what should be editable” and “the concern” would be that it is not something that we (as a core product) need or should worry about too much. No matter how hard we try, it’s ultimately up to the implementer to ensure that the edits they are making, to any instance in the system, are not causing that instance to represent a different thing. If we wanted to enforce that on our end, we would pretty much need to disable the ability to edit most of the properties for all our entities (eg. a name, a code…).

You are proposing that we make the product code immutable. I’d personally tend towards giving more, rather than less freedom in the edits. While the code is something that generally shouldn’t change, we already had an issue with an incorrect facility code assigned in Malawi. I believe this will/should be the only case where a code of any instance is edited, but it’s good to have that option if necessary, especially that editing the code doesn’t have any impact on the system as far as I know (references are by the id). It would be great to hear what other people think though.

+1 from me for “versioning” as our strategy to edit orderables. I think it’s the cleanest solution out of the proposed ones. It can also be implemented incrementally, without affecting other services (support for versioning first, migrating services one-by-one to use the orderable version second).

Best regards,

Sebastian.

On 28.11.2017 19:07, Nikodem Graczewski wrote:

Hello everyone!

I would like to restart the orderables edit discussion. In order to do it I’ve created the following recap on the problem and proposed a solution for dealing with it.

The Problem

Making Orderables editable to allow users to fix typos and pricing adjustments.

The Pain Points

Currently we have some classes that have the associations on the orderables. We can split them into two groups that:

  • expect orderables to be immutable. These classes care about the orderable details and don’t expect the orderable to change. Here’s a list of the classes that behave this way:
  • RequisitionLineItem
  • OrderLineItem
  • don’t care if orderable has changed. All they care that the object with the specific UUID still represents the same orderable, meaning the identity of the orderable hasn’t changed. Here’s a list of class that meet this condition:
  • FacilityTypeApprovedProduct
  • VmmApplicable
  • IdentifiableByOrderableLot
  • PhysicalInventoryLineItem
  • StockCard

What should be editable?

The main concern behind this question is that allowing the users to update the orderable could lead to an orderable object actually changing identity, meaning it would represent a different orderable than it did originally. The solution for this problem would be making the product code immutable. This would enforce that the orderable can not change its identity. If the user would like to change the code she/he would most likely be doing something she/he shouldn’t. As the problem we’re trying to solve here is rather about fixing the errors and updating the price this most likely won’t be a problem, but he possibility would still remain.

By the way, this concern is most likely also valid for any other entity that is mutable in the OpenLMIS.

The Solutions

I would like to propose three possible solutions I see from which I have chosen one. I will describe them in more details below

Archiving

With this approach we would be archiving the orderable before updating it and creating a new entry within the database giving it a new UUID.

Pros:

  • No changes would have to be made to the entities that treat the orderable as immutable, the UUID would point to the exact orderable version they want.

Cons:

  • We would have to update entities that treat orderable as mutable on every update as they don’t really care about orderable details but about its identity (meaning they should point to the newest possible version of the orderable). FTAP is a great example here. In Malawi implementation there is around 60 FTAPs for a single orderable, which would have to be updated/recreated every time we update the orderable. If we picked up the recreation path the orderables table in the database could grow really huge really fast. We would also have to deal with entities that associate the orderable and update them too, and this could propagate.
  • This would be hard to maintain (see the previous bullet point).
  • The database constraints would be non-trivial, we would have to restrict the situation when there is a couple of orderables that have the same product code and are not archived (this wouldn’t be true for archived orderables).

Open questions:

  • How would we track the orderable identity if the UUID could change?

    Product code is the natural candidate for that.

  • How would we track the newest orderable version?

    We would probably have to introduce the archived flag, which should be set to false for the newest version of the orderable.

Versioning

With this approach we would add a new field to the orderable named version/versionId/something similar. The identity of the orderable would still be held by the UUID attribute.

Pros:

  • No changes need outside the endpoint during the initial implementation.
  • No changes needed at all to the entities that treat the orderable as mutable, the UUID would still point to the newest version.
  • All versions stored in one place (referencedata), no need for snapshots in any module - all they would need would be the UUID and version.
  • We’re not breaking backward combability, the currently implemented endpoint still returns the latest orderable version based on the UUID. To get the older version of the orderable we would have to pass an optional request parameter. This would lead to the endpoint being extended rather than modified.

Cons:

  • The need to identify and update all the places that use orderables and expect them to be immutable.

Open questions:

  • Should we use GUUID or int to track version?

    By using a simple int we could easily retrieve the latest version of the orderable from the database.

  • Where should we store the version in object referencing a specific version of the orderable?

    ReferencedataObject class is a perfect candidate for that.

  • Would this break the contract and lead to the major version being increased?

    The endpoint itself stays the same, but introduction of the versioning would probably lead to the behavior change meaning this would most likely break the contract and require a major version bump.

  • Which changes should spawn new version?

    Ideally, typo fixes should not lead to a version bump, but those things might be hard to detect. We could give the users ability to decide whether the update should spawn a new version of the orderable or update the newest one, but this feature would most likely confuse the end user.

Snapshooting

With this approach we would be snapshooting properties of the orderable which are expected to be immutable by the associating classes.

Pros:

  • Easy to implement.
  • This would break some dependencies leading to a lesser coupling of the fulfillment/requisition and referencedata modules at the cost of code duplication.
  • This would lead to a temporary performance improvement (less calls to the referencedata i.e. during requisition retrieval)…

Cons:

  • …which would become a hit to the performance in the long run (bigger database).
  • This would be a little WET as we would be duplicating some of the properties, but we could snapshot only those fields we want immutable and ignore orderable beyond the point of creation of the object that was snapshooting the orderable.

The Proposed Solution

I would like to propose Versioning as the go-to solution for this problem. It has the most pros with only one con, which is true for all the solutions - eventually, we will have to change something somewhere to make orderables editable flawlessly. This approach would also require least work outside the orderable endpoint. I’ve diagnosed only two places that expect the orderable to be immutable, which are requisition and order creation, rest of the associations rely on the identity only and don’t really care about the orderable details changing.

The Proposed Strategy

This is a proposed strategy for implementing the orderable edit feature. These steps should make the implementation as smooth as possible.

  1. Add the version property to the orderable and modify endpoint to respect it. Create a migration to set the version to 0 by default. Don’t implement orderable edit feature yet.
  1. Update the Requisition and Order creation and classes to actually use ReferencedataObject instead of an ID, the ReferencedataObject should be extended to include the version property (VersionedReferencedataObject class perhaps?). This step could also include removal of the snapshoot properties from the RequisitionLineItem, as the specific version of the orderable wouldn’t be editable (a little DRY is always good).
  1. Implement the orderable edit feature with automated version updating.
  1. Watch everything going smooth.

The Concern

Implementing edit feature on an entity that is treated both as mutable and immutable depending on the associating class will leave a gate for bugs to creep in that might be hard to deal with. My main concern is a situation in which user decides that editing a discontinued orderable and overriding it with a new one is a good idea. This means that the identity of the object has changed and I couldn’t come up with a way to prevent it other than making some of the fields immutable.

Kind regards,

Nikodem

On Friday, September 29, 2017 at 6:09:15 PM UTC+2, Nikodem Graczewski wrote:

Hi everyone,

as per our Sprint Q&A meeting I’m starting a discussion about how do we want to approach the Orderables edit as allowing this action can possibly result is some unforeseen consequences in some of the modules depending on them.

Here’s a list of modules that are somehow related to Orderables:

  • referencedata - holds the Orderables
  • requisition
  • fulfillment
  • stockmanagement

I’ve done some research and I could only find one thing that would break after we implement Orderables edit. Requisition line item uses the ProgramOrderable to actually set the price per pack of itself. If we modify the ProgramsOrderable price per pack after the requisition initialization it won’t be reflected in the requisition line item.

Here’s a couple of ideas on how we could approach the problem:

  • We could create a new endpoint for updating the ProgramOrderables only, without touching the Orderable itself as proposed by Paweł Gesek here.
  • We could use the FHIR approach by versioning the Orderable. This approach would use two IDs, one for tracking the Orderable itself, so the ID of Orderable would never change and a second for tracking the Orderable version.

    This would allow us to:

  • Retrieve the newest version of Orderable simply by calling a GET endpoint with the ID. This would be used by the administration screens so they always display the newest Orderable. Modifying the Orderable would result in creation of a new version of it, and storing the previous one as history.
  • All the module that would require the older version of the Orderable would simply pass the version ID as a request parameter to the GET endpoint. This would result in retrieving a history version of an Orderable instead of the newest one.

If you have any ideas, comments or concerns about this feature, please post them in this thread.

Best Regards,

Nikodem

**

SolDevelo** Sp. z o.o. [LLC] /
www.soldevelo.com

Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland

Phone: +48 58 782 45 40 / Fax: +48 58 782 45 41

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to
openlmis-dev@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/openlmis-dev/e423e6a1-d69c-4348-93de-fa5d8a619129%40googlegroups.com
.

For more options, visit https://groups.google.com/d/optout.

Sebastian Brudziński

Software Developer / Team Leader

sbrudzinski@soldevelo.com

**

SolDevelo** Sp. z o.o. [LLC] /
www.soldevelo.com

Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland

Phone: +48 58 782 45 40 / Fax: +48 58 782 45 41

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to
openlmis-dev@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/openlmis-dev/0fc3491d-3734-3bee-b1ac-6d852517ead3%40soldevelo.com
.

For more options, visit https://groups.google.com/d/optout.


(Josh Zamor) #13

I love where this thread is going and some of the issues that various folks have explored and brought to the fore. Thanks everyone for the effort.

Of course, I also think Versioning is our best option. Some quick responses:

  • I agree with Sebastian and Brandon that we shouldn’t worry (much) about immutable fields. I’ve been burned in an old system where an admin was updating user profiles to “create” new users. It was a learning experience, but mostly on the UI side of things. Once the UI directed the admin to what changes meant, that mistake was corrected.
  • I also agree with Brandon’s comments on what types of changes create new versions (always - and FHIR does this as well) and also on where the version should be stored in a referencing entity.
  • I disagree that we should use a UUID for the version, and instead recommend we use a serial integer starting at 1 (and make sure in the DB it has plenty of room to grow). Reasoning: FHIR STU 3 uses a serial integer starting at 1. The UUID is important for uniquely identifying things. The version of that uniquely identified thing is something that is bounded by ACID, and therefore should always increase in a predictable fashion. Note what this means is that when you change an entity (say in the UI), you never increment the version and send it back, that’s a function of the write operation on the server from which it was read.

A few responses to Brandon’s other points:

For GS1 Trade Item-based Orderables, what are the GS1 GDSN rules about mutability? If Tylenol ever changes its PackSize, its UPC Barcode or changes name to “Tylenol Zero Sugar” must they issue a whole new Trade Item?

Great question. The actual answer to this is complex. The simple explanation I got in the GTIN training was this: if you care about identifying yellow shirts, blue shirts and orange shirts in the supply chain, they each need a GTIN. If you just care about shirts, one GTIN will do. The document I linked is aimed at manufacturers / brand owners and when they should issue new GTINs. The key for us however is that we’re supposed to trust what they do. If they make a mistake, as recipients of their information our system can’t automatically decide the change is in error or not. GS1 I believe works to assist those creating GTINs do so correctly. Put simply: OpenLMIS should allow any attribute change which is defined in the GDSN and trust that the author understands the effect of that master data in the supply chain.

For UNSPSC/GPC/Commodity-Type-
based Orderables, do those systems ever change the name of something (like how “Autism” was changed to “Autism Spectrum Disorder” in the DSM-5)?

Another great question. This one I don’t know, and I suspect it might vary depending on the author of the classification system (WHO, GS1, country-specific). The flexibility to where these classification systems come from is intentional in our design, even though we’d hope a country chooses mature systems. Because of this flexibility though, we might err on the side of caution and version CommodityType. I’m not 100% sure and I’ve created a ticket to track it: https://openlmis.atlassian.net/browse/OLMIS-3790

Finally, for now I believe we are just talking about making an individual Orderable editable. As a separate issue (later) we will also talk about how the FTAP list should be editable. But that’s a separate topic for later. (Correct me if I’m wrong.)

I think we are/could include FTAP in this discussion. It might have been how all this got started. Either way I think we know the basics:

  • FTAP should reference non-versioned Orderable, Program, and Facility Type.
  • Changes to a FTAP should be versioned.
  • Entities such as Requisition would reference the versioned Orderable and FTAP.
  • Sound correct?

Lets keep this moving. Next steps I think are:

  • Java and REST patterns created/documented for versioned Resources. We should look at FHIR STU 3 for REST and perhaps http://hapifhir.io/ for Java
  • Pattern in Java and REST for Resources which reference versioned Resources. Again same sources as above for guidance and inspiration.
  • Choose an area to start incrementally applying this. The product model is likely it, however I’d ask Sebastian’s input as an implementer here to re-iterate what he thinks we should approach first in an incremental fashion.
    Best,

Josh

···

On Thursday, December 7, 2017 at 7:26:32 PM UTC-8, brandon.bowersox-johnson wrote:

Of the 3 proposed approaches, Versioning is probably also the one I would also lean towards if we want to make an Orderable editable. I agree with Sebastian that implementers should be allowed to change the code and name. In the case of a typo in the code, it is fine to change, and the rest of our software references the UUID and not the code so I believe it does not break anything. Same with changing an orderable name.

The 4 open questions about Versioning still need some proposals, so I have a few responses to those:

• “Should we use GUUID or int to track version?”

GUUID sounds more robust to me and just as easy. The newer version could store a reference to the previous version.

• “Which changes should spawn new version?”

I think any changes should always spawn a new version. That is a safer and simpler solution rather than having a complex logic where certain field edits do NOT change the version number while other field edits DO change the version number. We might as well always make each edit a new version. Any big downsides to that?

• “Where should we store the version in object referencing a specific version of the orderable?”

We need to be clear on whether the other parts of OpenLMIS that reference an Orderable are also storing a reference to a specific version of the Orderable. To take one specific example, I think the Requisition object might want to store a specific version reference for each orderable. At the time of requisition initiation, it builds the Requisition using the latest version of each Orderable from the FTAP list. It also saves that version used. Then, if an Orderable changes over time (maybe code, maybe name), if I view an old requisition from 5 years ago, the on-screen view will show me the name and code from the exact version that was used back then.

I also have a few other points:

• We will need guidance or hard rules on which Orderable properties are NOT allowed to be mutable. Can anyone name an example?

• For GS1 Trade Item-based Orderables, what are the GS1 GDSN rules about mutability? If Tylenol ever changes its PackSize, its UPC Barcode or changes name to “Tylenol Zero Sugar” must they issue a whole new Trade Item?

• For UNSPSC/GPC/Commodity-Type-based Orderables, do those systems ever change the name of something (like how “Autism” was changed to “Autism Spectrum Disorder” in the DSM-5)?

• Finally, for now I believe we are just talking about making an individual Orderable editable. As a separate issue (later) we will also talk about how the FTAP list should be editable. But that’s a separate topic for later. (Correct me if I’m wrong.)

From: openlmis-dev@googlegroups.com on behalf of Sebastian Brudziński sbrudzinski@soldevelo.com

Date: Wednesday, November 29, 2017 at 10:18 AM

To: "openlmis-dev@googlegroups.com" openlmis-dev@googlegroups.com

Subject: Re: [openlmis-dev] Re: Orderable edit design

Thanks for reviving this discussion and compiling the summary, Nikodem. I think it’s important that we start making progress with the orderables edit while we are still far from the next release, since the impact of this change will be huge.

My comment on the “identity”, under “what should be editable” and “the concern” would be that it is not something that we (as a core product) need or should worry about too much. No matter how hard we try, it’s ultimately up to the implementer to ensure that the edits they are making, to any instance in the system, are not causing that instance to represent a different thing. If we wanted to enforce that on our end, we would pretty much need to disable the ability to edit most of the properties for all our entities (eg. a name, a code…).

You are proposing that we make the product code immutable. I’d personally tend towards giving more, rather than less freedom in the edits. While the code is something that generally shouldn’t change, we already had an issue with an incorrect facility code assigned in Malawi. I believe this will/should be the only case where a code of any instance is edited, but it’s good to have that option if necessary, especially that editing the code doesn’t have any impact on the system as far as I know (references are by the id). It would be great to hear what other people think though.

+1 from me for “versioning” as our strategy to edit orderables. I think it’s the cleanest solution out of the proposed ones. It can also be implemented incrementally, without affecting other services (support for versioning first, migrating services one-by-one to use the orderable version second).

Best regards,

Sebastian.

On 28.11.2017 19:07, Nikodem Graczewski wrote:

Hello everyone!

I would like to restart the orderables edit discussion. In order to do it I’ve created the following recap on the problem and proposed a solution for dealing with it.

The Problem

Making Orderables editable to allow users to fix typos and pricing adjustments.

The Pain Points

Currently we have some classes that have the associations on the orderables. We can split them into two groups that:

  • expect orderables to be immutable. These classes care about the orderable details and don’t expect the orderable to change. Here’s a list of the classes that behave this way:
  • RequisitionLineItem
  • OrderLineItem
  • don’t care if orderable has changed. All they care that the object with the specific UUID still represents the same orderable, meaning the identity of the orderable hasn’t changed. Here’s a list of class that meet this condition:
  • FacilityTypeApprovedProduct
  • VmmApplicable
  • IdentifiableByOrderableLot
  • PhysicalInventoryLineItem
  • StockCard

What should be editable?

The main concern behind this question is that allowing the users to update the orderable could lead to an orderable object actually changing identity, meaning it would represent a different orderable than it did originally. The solution for this problem would be making the product code immutable. This would enforce that the orderable can not change its identity. If the user would like to change the code she/he would most likely be doing something she/he shouldn’t. As the problem we’re trying to solve here is rather about fixing the errors and updating the price this most likely won’t be a problem, but he possibility would still remain.

By the way, this concern is most likely also valid for any other entity that is mutable in the OpenLMIS.

The Solutions

I would like to propose three possible solutions I see from which I have chosen one. I will describe them in more details below

Archiving

With this approach we would be archiving the orderable before updating it and creating a new entry within the database giving it a new UUID.

Pros:

  • No changes would have to be made to the entities that treat the orderable as immutable, the UUID would point to the exact orderable version they want.

Cons:

  • We would have to update entities that treat orderable as mutable on every update as they don’t really care about orderable details but about its identity (meaning they should point to the newest possible version of the orderable). FTAP is a great example here. In Malawi implementation there is around 60 FTAPs for a single orderable, which would have to be updated/recreated every time we update the orderable. If we picked up the recreation path the orderables table in the database could grow really huge really fast. We would also have to deal with entities that associate the orderable and update them too, and this could propagate.
  • This would be hard to maintain (see the previous bullet point).
  • The database constraints would be non-trivial, we would have to restrict the situation when there is a couple of orderables that have the same product code and are not archived (this wouldn’t be true for archived orderables).

Open questions:

  • How would we track the orderable identity if the UUID could change?

    Product code is the natural candidate for that.

  • How would we track the newest orderable version?

    We would probably have to introduce the archived flag, which should be set to false for the newest version of the orderable.

Versioning

With this approach we would add a new field to the orderable named version/versionId/something similar. The identity of the orderable would still be held by the UUID attribute.

Pros:

  • No changes need outside the endpoint during the initial implementation.
  • No changes needed at all to the entities that treat the orderable as mutable, the UUID would still point to the newest version.
  • All versions stored in one place (referencedata), no need for snapshots in any module - all they would need would be the UUID and version.
  • We’re not breaking backward combability, the currently implemented endpoint still returns the latest orderable version based on the UUID. To get the older version of the orderable we would have to pass an optional request parameter. This would lead to the endpoint being extended rather than modified.

Cons:

  • The need to identify and update all the places that use orderables and expect them to be immutable.

Open questions:

  • Should we use GUUID or int to track version?

    By using a simple int we could easily retrieve the latest version of the orderable from the database.

  • Where should we store the version in object referencing a specific version of the orderable?

    ReferencedataObject class is a perfect candidate for that.

  • Would this break the contract and lead to the major version being increased?

    The endpoint itself stays the same, but introduction of the versioning would probably lead to the behavior change meaning this would most likely break the contract and require a major version bump.

  • Which changes should spawn new version?

    Ideally, typo fixes should not lead to a version bump, but those things might be hard to detect. We could give the users ability to decide whether the update should spawn a new version of the orderable or update the newest one, but this feature would most likely confuse the end user.

Snapshooting

With this approach we would be snapshooting properties of the orderable which are expected to be immutable by the associating classes.

Pros:

  • Easy to implement.
  • This would break some dependencies leading to a lesser coupling of the fulfillment/requisition and referencedata modules at the cost of code duplication.
  • This would lead to a temporary performance improvement (less calls to the referencedata i.e. during requisition retrieval)…

Cons:

  • …which would become a hit to the performance in the long run (bigger database).
  • This would be a little WET as we would be duplicating some of the properties, but we could snapshot only those fields we want immutable and ignore orderable beyond the point of creation of the object that was snapshooting the orderable.

The Proposed Solution

I would like to propose Versioning as the go-to solution for this problem. It has the most pros with only one con, which is true for all the solutions - eventually, we will have to change something somewhere to make orderables editable flawlessly. This approach would also require least work outside the orderable endpoint. I’ve diagnosed only two places that expect the orderable to be immutable, which are requisition and order creation, rest of the associations rely on the identity only and don’t really care about the orderable details changing.

The Proposed Strategy

This is a proposed strategy for implementing the orderable edit feature. These steps should make the implementation as smooth as possible.

  1. Add the version property to the orderable and modify endpoint to respect it. Create a migration to set the version to 0 by default. Don’t implement orderable edit feature yet.
  2. Update the Requisition and Order creation and classes to actually use ReferencedataObject instead of an ID, the ReferencedataObject should be extended to include the version property (VersionedReferencedataObject class perhaps?). This step could also include removal of the snapshoot properties from the RequisitionLineItem, as the specific version of the orderable wouldn’t be editable (a little DRY is always good).
  3. Implement the orderable edit feature with automated version updating.
  4. Watch everything going smooth.

The Concern

Implementing edit feature on an entity that is treated both as mutable and immutable depending on the associating class will leave a gate for bugs to creep in that might be hard to deal with. My main concern is a situation in which user decides that editing a discontinued orderable and overriding it with a new one is a good idea. This means that the identity of the object has changed and I couldn’t come up with a way to prevent it other than making some of the fields immutable.

Kind regards,

Nikodem

On Friday, September 29, 2017 at 6:09:15 PM UTC+2, Nikodem Graczewski wrote:

Hi everyone,

as per our Sprint Q&A meeting I’m starting a discussion about how do we want to approach the Orderables edit as allowing this action can possibly result is some unforeseen consequences in some of the modules depending on them.

Here’s a list of modules that are somehow related to Orderables:

  • referencedata - holds the Orderables
  • requisition
  • fulfillment
  • stockmanagement
    I’ve done some research and I could only find one thing that would break after we implement Orderables edit. Requisition line item uses the ProgramOrderable to actually set the price per pack of itself. If we modify the ProgramsOrderable price per pack after the requisition initialization it won’t be reflected in the requisition line item.

Here’s a couple of ideas on how we could approach the problem:

  • We could create a new endpoint for updating the ProgramOrderables only, without touching the Orderable itself as proposed by Paweł Gesek here.

  • We could use the FHIR approach by versioning the Orderable. This approach would use two IDs, one for tracking the Orderable itself, so the ID of Orderable would never change and a second for tracking the Orderable version.

    This would allow us to:

  • Retrieve the newest version of Orderable simply by calling a GET endpoint with the ID. This would be used by the administration screens so they always display the newest Orderable. Modifying the Orderable would result in creation of a new version of it, and storing the previous one as history.

  • All the module that would require the older version of the Orderable would simply pass the version ID as a request parameter to the GET endpoint. This would result in retrieving a history version of an Orderable instead of the newest one.
    If you have any ideas, comments or concerns about this feature, please post them in this thread.

Best Regards,

Nikodem

**

SolDevelo** Sp. z o.o. [LLC] /
www.soldevelo.com

Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland

Phone: +48 58 782 45 40 / Fax: +48 58 782 45 41

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to
openlmis-dev@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/openlmis-dev/e423e6a1-d69c-4348-93de-fa5d8a619129%40googlegroups.com
.

For more options, visit https://groups.google.com/d/optout.

Sebastian Brudziński

Software Developer / Team Leader

sbrudzinski@soldevelo.com

**

SolDevelo** Sp. z o.o. [LLC] /
www.soldevelo.com

Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland

Phone: +48 58 782 45 40 / Fax: +48 58 782 45 41

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to
openlmis-dev@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/openlmis-dev/0fc3491d-3734-3bee-b1ac-6d852517ead3%40soldevelo.com
.

For more options, visit https://groups.google.com/d/optout.


(Łukasz Lewczyński) #14

I have small problem with versioning option. Let assume we have a productwith initial versionand create a requisition with that product and version. Now:

  1. If I modified the product name (because I found a typo) should I see this change on the product grid? In my opinion yes because this change does not modify my product.
  2. If I modified the product price should I see this change on the product grid? I think no because this change modifies my budget for the given period. Also if this change will be visible, the legacy data will be corrupted.

If we assume that each change creates a new version of product, how much new entries will be in database? In the worst scenario I can modify a single property, save the product, modify the next property, save it and so on.This could create a lot of rows in table.


SolDevelo
Sp. z o.o. [LLC] / www.soldevelo.com
Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland
Phone: +48 58 782 45 40 / Fax: +48 58 782 45 41

···

On Fri, Dec 8, 2017 at 10:33 PM, Josh Zamor josh.zamor@villagereach.org wrote:

I love where this thread is going and some of the issues that various folks have explored and brought to the fore. Thanks everyone for the effort.

Of course, I also think Versioning is our best option. Some quick responses:

  • I agree with Sebastian and Brandon that we shouldn’t worry (much) about immutable fields. I’ve been burned in an old system where an admin was updating user profiles to “create” new users. It was a learning experience, but mostly on the UI side of things. Once the UI directed the admin to what changes meant, that mistake was corrected.
  • I also agree with Brandon’s comments on what types of changes create new versions (always - and FHIR does this as well) and also on where the version should be stored in a referencing entity.
  • I disagree that we should use a UUID for the version, and instead recommend we use a serial integer starting at 1 (and make sure in the DB it has plenty of room to grow). Reasoning: FHIR STU 3 uses a serial integer starting at 1. The UUID is important for uniquely identifying things. The version of that uniquely identified thing is something that is bounded by ACID, and therefore should always increase in a predictable fashion. Note what this means is that when you change an entity (say in the UI), you never increment the version and send it back, that’s a function of the write operation on the server from which it was read.

A few responses to Brandon’s other points:

For GS1 Trade Item-based Orderables, what are the GS1 GDSN rules about mutability? If Tylenol ever changes its PackSize, its UPC Barcode or changes name to “Tylenol Zero Sugar” must they issue a whole new Trade Item?

Great question. The actual answer to this is complex. The simple explanation I got in the GTIN training was this: if you care about identifying yellow shirts, blue shirts and orange shirts in the supply chain, they each need a GTIN. If you just care about shirts, one GTIN will do. The document I linked is aimed at manufacturers / brand owners and when they should issue new GTINs. The key for us however is that we’re supposed to trust what they do. If they make a mistake, as recipients of their information our system can’t automatically decide the change is in error or not. GS1 I believe works to assist those creating GTINs do so correctly. Put simply: OpenLMIS should allow any attribute change which is defined in the GDSN and trust that the author understands the effect of that master data in the supply chain.

based Orderables, do those systems ever change the name of something (like how “Autism” was changed to “Autism Spectrum Disorder” in the DSM-5)?

For UNSPSC/GPC/Commodity-Type-

Another great question. This one I don’t know, and I suspect it might vary depending on the author of the classification system (WHO, GS1, country-specific). The flexibility to where these classification systems come from is intentional in our design, even though we’d hope a country chooses mature systems. Because of this flexibility though, we might err on the side of caution and version CommodityType. I’m not 100% sure and I’ve created a ticket to track it: https://openlmis.atlassian.net/browse/OLMIS-3790

Finally, for now I believe we are just talking about making an individual Orderable editable. As a separate issue (later) we will also talk about how the FTAP list should be editable. But that’s a separate topic for later. (Correct me if I’m wrong.)

I think we are/could include FTAP in this discussion. It might have been how all this got started. Either way I think we know the basics:

  • FTAP should reference non-versioned Orderable, Program, and Facility Type.
  • Changes to a FTAP should be versioned.
  • Entities such as Requisition would reference the versioned Orderable and FTAP.
  • Sound correct?

Lets keep this moving. Next steps I think are:

  • Java and REST patterns created/documented for versioned Resources. We should look at FHIR STU 3 for REST and perhaps http://hapifhir.io/ for Java
  • Pattern in Java and REST for Resources which reference versioned Resources. Again same sources as above for guidance and inspiration.
  • Choose an area to start incrementally applying this. The product model is likely it, however I’d ask Sebastian’s input as an implementer here to re-iterate what he thinks we should approach first in an incremental fashion.
    Best,

Josh

On Thursday, December 7, 2017 at 7:26:32 PM UTC-8, brandon.bowersox-johnson wrote:

Of the 3 proposed approaches, Versioning is probably also the one I would also lean towards if we want to make an Orderable editable. I agree with Sebastian that implementers should be allowed to change the code and name. In the case of a typo in the code, it is fine to change, and the rest of our software references the UUID and not the code so I believe it does not break anything. Same with changing an orderable name.

The 4 open questions about Versioning still need some proposals, so I have a few responses to those:

• “Should we use GUUID or int to track version?”

GUUID sounds more robust to me and just as easy. The newer version could store a reference to the previous version.

• “Which changes should spawn new version?”

I think any changes should always spawn a new version. That is a safer and simpler solution rather than having a complex logic where certain field edits do NOT change the version number while other field edits DO change the version number. We might as well always make each edit a new version. Any big downsides to that?

• “Where should we store the version in object referencing a specific version of the orderable?”

We need to be clear on whether the other parts of OpenLMIS that reference an Orderable are also storing a reference to a specific version of the Orderable. To take one specific example, I think the Requisition object might want to store a specific version reference for each orderable. At the time of requisition initiation, it builds the Requisition using the latest version of each Orderable from the FTAP list. It also saves that version used. Then, if an Orderable changes over time (maybe code, maybe name), if I view an old requisition from 5 years ago, the on-screen view will show me the name and code from the exact version that was used back then.

I also have a few other points:

• We will need guidance or hard rules on which Orderable properties are NOT allowed to be mutable. Can anyone name an example?

• For GS1 Trade Item-based Orderables, what are the GS1 GDSN rules about mutability? If Tylenol ever changes its PackSize, its UPC Barcode or changes name to “Tylenol Zero Sugar” must they issue a whole new Trade Item?

• For UNSPSC/GPC/Commodity-Type-based Orderables, do those systems ever change the name of something (like how “Autism” was changed to “Autism Spectrum Disorder” in the DSM-5)?

• Finally, for now I believe we are just talking about making an individual Orderable editable. As a separate issue (later) we will also talk about how the FTAP list should be editable. But that’s a separate topic for later. (Correct me if I’m wrong.)

From: openlmis-dev@googlegroups.com on behalf of Sebastian Brudziński sbrudzinski@soldevelo.com

Date: Wednesday, November 29, 2017 at 10:18 AM

To: "openlmis-dev@googlegroups.com" openlmis-dev@googlegroups.com

Subject: Re: [openlmis-dev] Re: Orderable edit design

Thanks for reviving this discussion and compiling the summary, Nikodem. I think it’s important that we start making progress with the orderables edit while we are still far from the next release, since the impact of this change will be huge.

My comment on the “identity”, under “what should be editable” and “the concern” would be that it is not something that we (as a core product) need or should worry about too much. No matter how hard we try, it’s ultimately up to the implementer to ensure that the edits they are making, to any instance in the system, are not causing that instance to represent a different thing. If we wanted to enforce that on our end, we would pretty much need to disable the ability to edit most of the properties for all our entities (eg. a name, a code…).

You are proposing that we make the product code immutable. I’d personally tend towards giving more, rather than less freedom in the edits. While the code is something that generally shouldn’t change, we already had an issue with an incorrect facility code assigned in Malawi. I believe this will/should be the only case where a code of any instance is edited, but it’s good to have that option if necessary, especially that editing the code doesn’t have any impact on the system as far as I know (references are by the id). It would be great to hear what other people think though.

+1 from me for “versioning” as our strategy to edit orderables. I think it’s the cleanest solution out of the proposed ones. It can also be implemented incrementally, without affecting other services (support for versioning first, migrating services one-by-one to use the orderable version second).

Best regards,

Sebastian.

On 28.11.2017 19:07, Nikodem Graczewski wrote:

Hello everyone!

I would like to restart the orderables edit discussion. In order to do it I’ve created the following recap on the problem and proposed a solution for dealing with it.

The Problem

Making Orderables editable to allow users to fix typos and pricing adjustments.

The Pain Points

Currently we have some classes that have the associations on the orderables. We can split them into two groups that:

  • expect orderables to be immutable. These classes care about the orderable details and don’t expect the orderable to change. Here’s a list of the classes that behave this way:
  • RequisitionLineItem
  • OrderLineItem
  • don’t care if orderable has changed. All they care that the object with the specific UUID still represents the same orderable, meaning the identity of the orderable hasn’t changed. Here’s a list of class that meet this condition:
  • FacilityTypeApprovedProduct
  • VmmApplicable
  • IdentifiableByOrderableLot
  • PhysicalInventoryLineItem
  • StockCard

What should be editable?

The main concern behind this question is that allowing the users to update the orderable could lead to an orderable object actually changing identity, meaning it would represent a different orderable than it did originally. The solution for this problem would be making the product code immutable. This would enforce that the orderable can not change its identity. If the user would like to change the code she/he would most likely be doing something she/he shouldn’t. As the problem we’re trying to solve here is rather about fixing the errors and updating the price this most likely won’t be a problem, but he possibility would still remain.

By the way, this concern is most likely also valid for any other entity that is mutable in the OpenLMIS.

The Solutions

I would like to propose three possible solutions I see from which I have chosen one. I will describe them in more details below

Archiving

With this approach we would be archiving the orderable before updating it and creating a new entry within the database giving it a new UUID.

Pros:

  • No changes would have to be made to the entities that treat the orderable as immutable, the UUID would point to the exact orderable version they want.

Cons:

  • We would have to update entities that treat orderable as mutable on every update as they don’t really care about orderable details but about its identity (meaning they should point to the newest possible version of the orderable). FTAP is a great example here. In Malawi implementation there is around 60 FTAPs for a single orderable, which would have to be updated/recreated every time we update the orderable. If we picked up the recreation path the orderables table in the database could grow really huge really fast. We would also have to deal with entities that associate the orderable and update them too, and this could propagate.
  • This would be hard to maintain (see the previous bullet point).
  • The database constraints would be non-trivial, we would have to restrict the situation when there is a couple of orderables that have the same product code and are not archived (this wouldn’t be true for archived orderables).

Open questions:

  • How would we track the orderable identity if the UUID could change?

    Product code is the natural candidate for that.

  • How would we track the newest orderable version?

    We would probably have to introduce the archived flag, which should be set to false for the newest version of the orderable.

Versioning

With this approach we would add a new field to the orderable named version/versionId/something similar. The identity of the orderable would still be held by the UUID attribute.

Pros:

  • No changes need outside the endpoint during the initial implementation.
  • No changes needed at all to the entities that treat the orderable as mutable, the UUID would still point to the newest version.
  • All versions stored in one place (referencedata), no need for snapshots in any module - all they would need would be the UUID and version.
  • We’re not breaking backward combability, the currently implemented endpoint still returns the latest orderable version based on the UUID. To get the older version of the orderable we would have to pass an optional request parameter. This would lead to the endpoint being extended rather than modified.

Cons:

  • The need to identify and update all the places that use orderables and expect them to be immutable.

Open questions:

  • Should we use GUUID or int to track version?

    By using a simple int we could easily retrieve the latest version of the orderable from the database.

  • Where should we store the version in object referencing a specific version of the orderable?

    ReferencedataObject class is a perfect candidate for that.

  • Would this break the contract and lead to the major version being increased?

    The endpoint itself stays the same, but introduction of the versioning would probably lead to the behavior change meaning this would most likely break the contract and require a major version bump.

  • Which changes should spawn new version?

    Ideally, typo fixes should not lead to a version bump, but those things might be hard to detect. We could give the users ability to decide whether the update should spawn a new version of the orderable or update the newest one, but this feature would most likely confuse the end user.

Snapshooting

With this approach we would be snapshooting properties of the orderable which are expected to be immutable by the associating classes.

Pros:

  • Easy to implement.
  • This would break some dependencies leading to a lesser coupling of the fulfillment/requisition and referencedata modules at the cost of code duplication.
  • This would lead to a temporary performance improvement (less calls to the referencedata i.e. during requisition retrieval)…

Cons:

  • …which would become a hit to the performance in the long run (bigger database).
  • This would be a little WET as we would be duplicating some of the properties, but we could snapshot only those fields we want immutable and ignore orderable beyond the point of creation of the object that was snapshooting the orderable.

The Proposed Solution

I would like to propose Versioning as the go-to solution for this problem. It has the most pros with only one con, which is true for all the solutions - eventually, we will have to change something somewhere to make orderables editable flawlessly. This approach would also require least work outside the orderable endpoint. I’ve diagnosed only two places that expect the orderable to be immutable, which are requisition and order creation, rest of the associations rely on the identity only and don’t really care about the orderable details changing.

The Proposed Strategy

This is a proposed strategy for implementing the orderable edit feature. These steps should make the implementation as smooth as possible.

  1. Add the version property to the orderable and modify endpoint to respect it. Create a migration to set the version to 0 by default. Don’t implement orderable edit feature yet.
  2. Update the Requisition and Order creation and classes to actually use ReferencedataObject instead of an ID, the ReferencedataObject should be extended to include the version property (VersionedReferencedataObject class perhaps?). This step could also include removal of the snapshoot properties from the RequisitionLineItem, as the specific version of the orderable wouldn’t be editable (a little DRY is always good).
  3. Implement the orderable edit feature with automated version updating.
  4. Watch everything going smooth.

The Concern

Implementing edit feature on an entity that is treated both as mutable and immutable depending on the associating class will leave a gate for bugs to creep in that might be hard to deal with. My main concern is a situation in which user decides that editing a discontinued orderable and overriding it with a new one is a good idea. This means that the identity of the object has changed and I couldn’t come up with a way to prevent it other than making some of the fields immutable.

Kind regards,

Nikodem

On Friday, September 29, 2017 at 6:09:15 PM UTC+2, Nikodem Graczewski wrote:

Hi everyone,

as per our Sprint Q&A meeting I’m starting a discussion about how do we want to approach the Orderables edit as allowing this action can possibly result is some unforeseen consequences in some of the modules depending on them.

Here’s a list of modules that are somehow related to Orderables:

  • referencedata - holds the Orderables
  • requisition
  • fulfillment
  • stockmanagement
    I’ve done some research and I could only find one thing that would break after we implement Orderables edit. Requisition line item uses the ProgramOrderable to actually set the price per pack of itself. If we modify the ProgramsOrderable price per pack after the requisition initialization it won’t be reflected in the requisition line item.

Here’s a couple of ideas on how we could approach the problem:

  • We could create a new endpoint for updating the ProgramOrderables only, without touching the Orderable itself as proposed by Paweł Gesek here.

  • We could use the FHIR approach by versioning the Orderable. This approach would use two IDs, one for tracking the Orderable itself, so the ID of Orderable would never change and a second for tracking the Orderable version.

    This would allow us to:

  • Retrieve the newest version of Orderable simply by calling a GET endpoint with the ID. This would be used by the administration screens so they always display the newest Orderable. Modifying the Orderable would result in creation of a new version of it, and storing the previous one as history.

  • All the module that would require the older version of the Orderable would simply pass the version ID as a request parameter to the GET endpoint. This would result in retrieving a history version of an Orderable instead of the newest one.
    If you have any ideas, comments or concerns about this feature, please post them in this thread.

Best Regards,

Nikodem

**

SolDevelo** Sp. z o.o. [LLC] /
www.soldevelo.com

Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland

Phone: +48 58 782 45 40 / Fax: +48 58 782 45 41

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to
openlmis-dev@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/openlmis-dev/e423e6a1-d69c-4348-93de-fa5d8a619129%40googlegroups.com
.

For more options, visit https://groups.google.com/d/optout.

Sebastian Brudziński

Software Developer / Team Leader

sbrudzinski@soldevelo.com

**

SolDevelo** Sp. z o.o. [LLC] /
www.soldevelo.com

Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland

Phone: +48 58 782 45 40 / Fax: +48 58 782 45 41

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to
openlmis-dev@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/openlmis-dev/0fc3491d-3734-3bee-b1ac-6d852517ead3%40soldevelo.com
.

For more options, visit https://groups.google.com/d/optout.

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to openlmis-dev@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openlmis-dev/4bec1537-73d8-4333-a9ac-55f71047b6f0%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Łukasz Lewczyński
Software Developer
llewczynski@soldevelo.com


(Josh Zamor) #15

Hi Lukasz, my thoughts:

  1. If I modified the product name (because I found a typo) should I see this change on the product grid? In my opinion yes because this change does not modify my product.

While I could see someone wanting this, there already is a simple way to get this: delete the requisition that was created and create it again. This is simple for a user to understand as it mimics what happens with paper based requisitions, i.e. it fits their mental model. Moreover doing this, instead of something fancier, keeps the separation of concerns: a typo fix to an Orderable name is the same as any other change: it creates a new version of the record. Any client which uses the Orderable now gets to decide how it wants to use the Orderable and its versions: use the latest, use a specific version, make explicit functions of it’s workflow in case the Orderable updates (e.g. while a Requisition is still in process). This is a nice separation of concerns which we should be striving for in our microservices.

  1. If I modified the product price should I see this change on the product grid? I think no because this change modifies my budget for the given period. Also if this change will be visible, the legacy data will be corrupted.

I think that’s what we’re saying with a versioned Orderable: the price wouldn’t change in the Requisition as it has a pointer to the version that was latest when it was created.

If we assume that each change creates a new version of product, how much new entries will be in database? In the worst scenario I can modify a single property, save the product, modify the next property, save it and so on.This could create a lot of rows in table.

This should be well within the capabilities of Postgres. While write operations here may be frequent, mostly we’re thinking in terms of read operations, where a UUID (primary key) and version may be put into a multi-column index. As a reference in the requisition line item table with 7+ million rows we have sub millisecond lookups in Postgres. Lets say we had 10,000 Orderables, each with 1000 changes, this would be roughly equivalent to only 10M rows in the Orderable table.

Good questions, keep them coming.

Best,

Josh

···

On Fri, Dec 8, 2017 at 10:33 PM, Josh Zamor
josh.zamor@villagereach.org wrote:

I love where this thread is going and some of the issues that various folks have explored and brought to the fore. Thanks everyone for the effort.

Of course, I also think Versioning is our best option. Some quick responses:

  • I agree with Sebastian and Brandon that we shouldn’t worry (much) about immutable fields. I’ve been burned in an old system where an admin was updating user profiles to “create” new users. It was a learning experience, but mostly on the UI side of things. Once the UI directed the admin to what changes meant, that mistake was corrected.
  • I also agree with Brandon’s comments on what types of changes create new versions (always - and FHIR does this as well) and also on where the version should be stored in a referencing entity.
  • I disagree that we should use a UUID for the version, and instead recommend we use a serial integer starting at 1 (and make sure in the DB it has plenty of room to grow). Reasoning: FHIR STU 3 uses a serial integer starting at 1. The UUID is important for uniquely identifying things. The version of that uniquely identified thing is something that is bounded by ACID, and therefore should always increase in a predictable fashion. Note what this means is that when you change an entity (say in the UI), you never increment the version and send it back, that’s a function of the write operation on the server from which it was read.

A few responses to Brandon’s other points:

For GS1 Trade Item-based Orderables, what are the GS1 GDSN rules about mutability? If Tylenol ever changes its PackSize, its UPC Barcode or changes name to “Tylenol Zero Sugar” must they issue a whole new Trade Item?

Great question. The actual answer to this is
complex
. The simple explanation I got in the GTIN training was this: if you care about identifying yellow shirts, blue shirts and orange shirts in the supply chain, they each need a GTIN. If you just care about shirts, one GTIN will do. The document I linked is aimed at manufacturers / brand owners and when they should issue new GTINs. The key for us however is that we’re supposed to trust what they do. If they make a mistake, as recipients of their information our system can’t automatically decide the change is in error or not. GS1 I believe works to assist those creating GTINs do so correctly. Put simply: OpenLMIS should allow any attribute change which is defined in the GDSN and trust that the author understands the effect of that master data in the supply chain.

based Orderables, do those systems ever change the name of something (like how “Autism” was changed to “Autism Spectrum Disorder” in the DSM-5)?

For UNSPSC/GPC/Commodity-Type-

Another great question. This one I don’t know, and I suspect it might vary depending on the author of the classification system (WHO, GS1, country-specific). The flexibility to where these classification systems come from is intentional in our design, even though we’d hope a country chooses mature systems. Because of this flexibility though, we might err on the side of caution and version CommodityType. I’m not 100% sure and I’ve created a ticket to track it:
https://openlmis.atlassian.net/browse/OLMIS-3790

Finally, for now I believe we are just talking about making an individual Orderable editable. As a separate issue (later) we will also talk about how the FTAP list should be editable. But that’s a separate topic for later. (Correct me if I’m wrong.)

I think we are/could include FTAP in this discussion. It might have been how all this got started. Either way I think we know the basics:

  • FTAP should reference non-versioned Orderable, Program, and Facility Type.
  • Changes to a FTAP should be versioned.
  • Entities such as Requisition would reference the versioned Orderable and FTAP.
  • Sound correct?

Lets keep this moving. Next steps I think are:

  • Java and REST patterns created/documented for versioned Resources. We should look at FHIR STU 3 for REST and perhaps http://hapifhir.io/ for Java
  • Pattern in Java and REST for Resources which reference versioned Resources. Again same sources as above for guidance and inspiration.
  • Choose an area to start incrementally applying this. The product model is likely it, however I’d ask Sebastian’s input as an implementer here to re-iterate what he thinks we should approach first in an incremental fashion.
    Best,

Josh

On Thursday, December 7, 2017 at 7:26:32 PM UTC-8, brandon.bowersox-johnson wrote:

Of the 3 proposed approaches, Versioning is probably also the one I would also lean towards if we want to make an Orderable editable. I agree with Sebastian that implementers should be allowed to change the code and name. In the case of a typo in the code, it is fine to change, and the rest of our software references the UUID and not the code so I believe it does not break anything. Same with changing an orderable name.

The 4 open questions about Versioning still need some proposals, so I have a few responses to those:

• “Should we use GUUID or int to track version?”

GUUID sounds more robust to me and just as easy. The newer version could store a reference to the previous version.

• “Which changes should spawn new version?”

I think any changes should always spawn a new version. That is a safer and simpler solution rather than having a complex logic where certain field edits do NOT change the version number while other field edits DO change the version number. We might as well always make each edit a new version. Any big downsides to that?

• “Where should we store the version in object referencing a specific version of the orderable?”

We need to be clear on whether the other parts of OpenLMIS that reference an Orderable are also storing a reference to a specific version of the Orderable. To take one specific example, I think the Requisition object might want to store a specific version reference for each orderable. At the time of requisition initiation, it builds the Requisition using the latest version of each Orderable from the FTAP list. It also saves that version used. Then, if an Orderable changes over time (maybe code, maybe name), if I view an old requisition from 5 years ago, the on-screen view will show me the name and code from the exact version that was used back then.

I also have a few other points:

• We will need guidance or hard rules on which Orderable properties are NOT allowed to be mutable. Can anyone name an example?

• For GS1 Trade Item-based Orderables, what are the GS1 GDSN rules about mutability? If Tylenol ever changes its PackSize, its UPC Barcode or changes name to “Tylenol Zero Sugar” must they issue a whole new Trade Item?

• For UNSPSC/GPC/Commodity-Type-based Orderables, do those systems ever change the name of something (like how “Autism” was changed to “Autism Spectrum Disorder” in the DSM-5)?

• Finally, for now I believe we are just talking about making an individual Orderable editable. As a separate issue (later) we will also talk about how the FTAP list should be editable. But that’s a separate topic for later. (Correct me if I’m wrong.)

From: openlmis-dev@googlegroups.com on behalf of Sebastian Brudziński sbrudzinski@soldevelo.com

Date: Wednesday, November 29, 2017 at 10:18 AM

To: "openlmis-dev@googlegroups.com" openlmis-dev@googlegroups.com

Subject: Re: [openlmis-dev] Re: Orderable edit design

Thanks for reviving this discussion and compiling the summary, Nikodem. I think it’s important that we start making progress with the orderables edit while we are still far from the next release, since the impact of this change will be huge.

My comment on the “identity”, under “what should be editable” and “the concern” would be that it is not something that we (as a core product) need or should worry about too much. No matter how hard we try, it’s ultimately up to the implementer to ensure that the edits they are making, to any instance in the system, are not causing that instance to represent a different thing. If we wanted to enforce that on our end, we would pretty much need to disable the ability to edit most of the properties for all our entities (eg. a name, a code…).

You are proposing that we make the product code immutable. I’d personally tend towards giving more, rather than less freedom in the edits. While the code is something that generally shouldn’t change, we already had an issue with an incorrect facility code assigned in Malawi. I believe this will/should be the only case where a code of any instance is edited, but it’s good to have that option if necessary, especially that editing the code doesn’t have any impact on the system as far as I know (references are by the id). It would be great to hear what other people think though.

+1 from me for “versioning” as our strategy to edit orderables. I think it’s the cleanest solution out of the proposed ones. It can also be implemented incrementally, without affecting other services (support for versioning first, migrating services one-by-one to use the orderable version second).

Best regards,

Sebastian.

On 28.11.2017 19:07, Nikodem Graczewski wrote:

Hello everyone!

I would like to restart the orderables edit discussion. In order to do it I’ve created the following recap on the problem and proposed a solution for dealing with it.

The Problem

Making Orderables editable to allow users to fix typos and pricing adjustments.

The Pain Points

Currently we have some classes that have the associations on the orderables. We can split them into two groups that:

  • expect orderables to be immutable. These classes care about the orderable details and don’t expect the orderable to change. Here’s a list of the classes that behave this way:
  • RequisitionLineItem
  • OrderLineItem
  • don’t care if orderable has changed. All they care that the object with the specific UUID still represents the same orderable, meaning the identity of the orderable hasn’t changed. Here’s a list of class that meet this condition:
  • FacilityTypeApprovedProduct
  • VmmApplicable
  • IdentifiableByOrderableLot
  • PhysicalInventoryLineItem
  • StockCard

What should be editable?

The main concern behind this question is that allowing the users to update the orderable could lead to an orderable object actually changing identity, meaning it would represent a different orderable than it did originally. The solution for this problem would be making the product code immutable. This would enforce that the orderable can not change its identity. If the user would like to change the code she/he would most likely be doing something she/he shouldn’t. As the problem we’re trying to solve here is rather about fixing the errors and updating the price this most likely won’t be a problem, but he possibility would still remain.

By the way, this concern is most likely also valid for any other entity that is mutable in the OpenLMIS.

The Solutions

I would like to propose three possible solutions I see from which I have chosen one. I will describe them in more details below

Archiving

With this approach we would be archiving the orderable before updating it and creating a new entry within the database giving it a new UUID.

Pros:

  • No changes would have to be made to the entities that treat the orderable as immutable, the UUID would point to the exact orderable version they want.

Cons:

  • We would have to update entities that treat orderable as mutable on every update as they don’t really care about orderable details but about its identity (meaning they should point to the newest possible version of the orderable). FTAP is a great example here. In Malawi implementation there is around 60 FTAPs for a single orderable, which would have to be updated/recreated every time we update the orderable. If we picked up the recreation path the orderables table in the database could grow really huge really fast. We would also have to deal with entities that associate the orderable and update them too, and this could propagate.
  • This would be hard to maintain (see the previous bullet point).
  • The database constraints would be non-trivial, we would have to restrict the situation when there is a couple of orderables that have the same product code and are not archived (this wouldn’t be true for archived orderables).

Open questions:

  • How would we track the orderable identity if the UUID could change?

    Product code is the natural candidate for that.

  • How would we track the newest orderable version?

    We would probably have to introduce the archived flag, which should be set to false for the newest version of the orderable.

Versioning

With this approach we would add a new field to the orderable named version/versionId/something similar. The identity of the orderable would still be held by the UUID attribute.

Pros:

  • No changes need outside the endpoint during the initial implementation.
  • No changes needed at all to the entities that treat the orderable as mutable, the UUID would still point to the newest version.
  • All versions stored in one place (referencedata), no need for snapshots in any module - all they would need would be the UUID and version.
  • We’re not breaking backward combability, the currently implemented endpoint still returns the latest orderable version based on the UUID. To get the older version of the orderable we would have to pass an optional request parameter. This would lead to the endpoint being extended rather than modified.

Cons:

  • The need to identify and update all the places that use orderables and expect them to be immutable.

Open questions:

  • Should we use GUUID or int to track version?

    By using a simple int we could easily retrieve the latest version of the orderable from the database.

  • Where should we store the version in object referencing a specific version of the orderable?

    ReferencedataObject class is a perfect candidate for that.

  • Would this break the contract and lead to the major version being increased?

    The endpoint itself stays the same, but introduction of the versioning would probably lead to the behavior change meaning this would most likely break the contract and require a major version bump.

  • Which changes should spawn new version?

    Ideally, typo fixes should not lead to a version bump, but those things might be hard to detect. We could give the users ability to decide whether the update should spawn a new version of the orderable or update the newest one, but this feature would most likely confuse the end user.

Snapshooting

With this approach we would be snapshooting properties of the orderable which are expected to be immutable by the associating classes.

Pros:

  • Easy to implement.
  • This would break some dependencies leading to a lesser coupling of the fulfillment/requisition and referencedata modules at the cost of code duplication.
  • This would lead to a temporary performance improvement (less calls to the referencedata i.e. during requisition retrieval)…

Cons:

  • …which would become a hit to the performance in the long run (bigger database).
  • This would be a little WET as we would be duplicating some of the properties, but we could snapshot only those fields we want immutable and ignore orderable beyond the point of creation of the object that was snapshooting the orderable.

The Proposed Solution

I would like to propose Versioning as the go-to solution for this problem. It has the most pros with only one con, which is true for all the solutions - eventually, we will have to change something somewhere to make orderables editable flawlessly. This approach would also require least work outside the orderable endpoint. I’ve diagnosed only two places that expect the orderable to be immutable, which are requisition and order creation, rest of the associations rely on the identity only and don’t really care about the orderable details changing.

The Proposed Strategy

This is a proposed strategy for implementing the orderable edit feature. These steps should make the implementation as smooth as possible.

  1. Add the version property to the orderable and modify endpoint to respect it. Create a migration to set the version to 0 by default. Don’t implement orderable edit feature yet.
  2. Update the Requisition and Order creation and classes to actually use ReferencedataObject instead of an ID, the ReferencedataObject should be extended to include the version property (VersionedReferencedataObject class perhaps?). This step could also include removal of the snapshoot properties from the RequisitionLineItem, as the specific version of the orderable wouldn’t be editable (a little DRY is always good).
  3. Implement the orderable edit feature with automated version updating.
  4. Watch everything going smooth.

The Concern

Implementing edit feature on an entity that is treated both as mutable and immutable depending on the associating class will leave a gate for bugs to creep in that might be hard to deal with. My main concern is a situation in which user decides that editing a discontinued orderable and overriding it with a new one is a good idea. This means that the identity of the object has changed and I couldn’t come up with a way to prevent it other than making some of the fields immutable.

Kind regards,

Nikodem

On Friday, September 29, 2017 at 6:09:15 PM UTC+2, Nikodem Graczewski wrote:

Hi everyone,

as per our Sprint Q&A meeting I’m starting a discussion about how do we want to approach the Orderables edit as allowing this action can possibly result is some unforeseen consequences in some of the modules depending on them.

Here’s a list of modules that are somehow related to Orderables:

  • referencedata - holds the Orderables
  • requisition
  • fulfillment
  • stockmanagement
    I’ve done some research and I could only find one thing that would break after we implement Orderables edit. Requisition line item uses the ProgramOrderable to actually set the price per pack of itself. If we modify the ProgramsOrderable price per pack after the requisition initialization it won’t be reflected in the requisition line item.

Here’s a couple of ideas on how we could approach the problem:

  • We could create a new endpoint for updating the ProgramOrderables only, without touching the Orderable itself as proposed by Paweł Gesek here.

  • We could use the FHIR approach by versioning the Orderable. This approach would use two IDs, one for tracking the Orderable itself, so the ID of Orderable would never change and a second for tracking the Orderable version.

    This would allow us to:

  • Retrieve the newest version of Orderable simply by calling a GET endpoint with the ID. This would be used by the administration screens so they always display the newest Orderable. Modifying the Orderable would result in creation of a new version of it, and storing the previous one as history.

  • All the module that would require the older version of the Orderable would simply pass the version ID as a request parameter to the GET endpoint. This would result in retrieving a history version of an Orderable instead of the newest one.
    If you have any ideas, comments or concerns about this feature, please post them in this thread.

Best Regards,

Nikodem

**

SolDevelo** Sp. z o.o. [LLC] /
www.soldevelo.com

Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland

Phone:
+48 58 782 45 40 / Fax:
+48 58 782 45 41

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to
openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to
openlmis-dev@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/openlmis-dev/e423e6a1-d69c-4348-93de-fa5d8a619129%40googlegroups.com
.

For more options, visit
https://groups.google.com/d/optout
.

Sebastian Brudziński

Software Developer / Team Leader

sbrudzinski@soldevelo.com

**

SolDevelo** Sp. z o.o. [LLC] /
www.soldevelo.com

Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland

Phone:
+48 58 782 45 40 / Fax:
+48 58 782 45 41

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to
openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to
openlmis-dev@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/openlmis-dev/0fc3491d-3734-3bee-b1ac-6d852517ead3%40soldevelo.com
.

For more options, visit
https://groups.google.com/d/optout
.

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to
openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to
openlmis-dev@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/openlmis-dev/4bec1537-73d8-4333-a9ac-55f71047b6f0%40googlegroups.com
.

For more options, visit
https://groups.google.com/d/optout
.

Łukasz Lewczyński

Software Developer

llewczynski@soldevelo.com


(Łukasz Lewczyński) #16

The example with paper requisition make it very clear for me because if I made a typo on paper I will had to get another paper. Thanks for this explanation.


SolDevelo
Sp. z o.o. [LLC] / www.soldevelo.com
Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland
Phone: +48 58 782 45 40 / Fax: +48 58 782 45 41

···

On Mon, Dec 11, 2017 at 7:17 PM, Josh Zamor josh.zamor@villagereach.org wrote:

Hi Lukasz, my thoughts:

  1. If I modified the product name (because I found a typo) should I see this change on the product grid? In my opinion yes because this change does not modify my product.

While I could see someone wanting this, there already is a simple way to get this: delete the requisition that was created and create it again. This is simple for a user to understand as it mimics what happens with paper based requisitions, i.e. it fits their mental model. Moreover doing this, instead of something fancier, keeps the separation of concerns: a typo fix to an Orderable name is the same as any other change: it creates a new version of the record. Any client which uses the Orderable now gets to decide how it wants to use the Orderable and its versions: use the latest, use a specific version, make explicit functions of it’s workflow in case the Orderable updates (e.g. while a Requisition is still in process). This is a nice separation of concerns which we should be striving for in our microservices.

  1. If I modified the product price should I see this change on the product grid? I think no because this change modifies my budget for the given period. Also if this change will be visible, the legacy data will be corrupted.

I think that’s what we’re saying with a versioned Orderable: the price wouldn’t change in the Requisition as it has a pointer to the version that was latest when it was created.

If we assume that each change creates a new version of product, how much new entries will be in database? In the worst scenario I can modify a single property, save the product, modify the next property, save it and so on.This could create a lot of rows in table.

This should be well within the capabilities of Postgres. While write operations here may be frequent, mostly we’re thinking in terms of read operations, where a UUID (primary key) and version may be put into a multi-column index. As a reference in the requisition line item table with 7+ million rows we have sub millisecond lookups in Postgres. Lets say we had 10,000 Orderables, each with 1000 changes, this would be roughly equivalent to only 10M rows in the Orderable table.

Good questions, keep them coming.

Best,

Josh

On Dec 11, 2017, at 6:22 AM, Łukasz Lewczyński llewczynski@soldevelo.com wrote:

I have small problem with versioning option. Let assume we have a productwith initial versionand create a requisition with that product and version. Now:

  1. If I modified the product name (because I found a typo) should I see this change on the product grid? In my opinion yes because this change does not modify my product.
  2. If I modified the product price should I see this change on the product grid? I think no because this change modifies my budget for the given period. Also if this change will be visible, the legacy data will be corrupted.

If we assume that each change creates a new version of product, how much new entries will be in database? In the worst scenario I can modify a single property, save the product, modify the next property, save it and so on.This could create a lot of rows in table.

**

SolDevelo** Sp. z o.o. [LLC] /
www.soldevelo.com

Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland

Phone: +48 58 782 45 40 / Fax: +48 58 782 45 41

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to
openlmis-dev@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/openlmis-dev/CAAdp53wP-4UJ4SbrOi0f0FZVs7zHm7OQPsd8QZQV4KDGOVTWSw%40mail.gmail.com
.

For more options, visit https://groups.google.com/d/optout.

Łukasz Lewczyński
Software Developer
llewczynski@soldevelo.com

Łukasz Lewczyński

Software Developer

llewczynski@soldevelo.com

On Fri, Dec 8, 2017 at 10:33 PM, Josh Zamor
josh.zamor@villagereach.org wrote:

I love where this thread is going and some of the issues that various folks have explored and brought to the fore. Thanks everyone for the effort.

Of course, I also think Versioning is our best option. Some quick responses:

  • I agree with Sebastian and Brandon that we shouldn’t worry (much) about immutable fields. I’ve been burned in an old system where an admin was updating user profiles to “create” new users. It was a learning experience, but mostly on the UI side of things. Once the UI directed the admin to what changes meant, that mistake was corrected.
  • I also agree with Brandon’s comments on what types of changes create new versions (always - and FHIR does this as well) and also on where the version should be stored in a referencing entity.
  • I disagree that we should use a UUID for the version, and instead recommend we use a serial integer starting at 1 (and make sure in the DB it has plenty of room to grow). Reasoning: FHIR STU 3 uses a serial integer starting at 1. The UUID is important for uniquely identifying things. The version of that uniquely identified thing is something that is bounded by ACID, and therefore should always increase in a predictable fashion. Note what this means is that when you change an entity (say in the UI), you never increment the version and send it back, that’s a function of the write operation on the server from which it was read.

A few responses to Brandon’s other points:

For GS1 Trade Item-based Orderables, what are the GS1 GDSN rules about mutability? If Tylenol ever changes its PackSize, its UPC Barcode or changes name to “Tylenol Zero Sugar” must they issue a whole new Trade Item?

Great question. The actual answer to this is
complex
. The simple explanation I got in the GTIN training was this: if you care about identifying yellow shirts, blue shirts and orange shirts in the supply chain, they each need a GTIN. If you just care about shirts, one GTIN will do. The document I linked is aimed at manufacturers / brand owners and when they should issue new GTINs. The key for us however is that we’re supposed to trust what they do. If they make a mistake, as recipients of their information our system can’t automatically decide the change is in error or not. GS1 I believe works to assist those creating GTINs do so correctly. Put simply: OpenLMIS should allow any attribute change which is defined in the GDSN and trust that the author understands the effect of that master data in the supply chain.

based Orderables, do those systems ever change the name of something (like how “Autism” was changed to “Autism Spectrum Disorder” in the DSM-5)?

For UNSPSC/GPC/Commodity-Type-

Another great question. This one I don’t know, and I suspect it might vary depending on the author of the classification system (WHO, GS1, country-specific). The flexibility to where these classification systems come from is intentional in our design, even though we’d hope a country chooses mature systems. Because of this flexibility though, we might err on the side of caution and version CommodityType. I’m not 100% sure and I’ve created a ticket to track it:
https://openlmis.atlassian.net/browse/OLMIS-3790

Finally, for now I believe we are just talking about making an individual Orderable editable. As a separate issue (later) we will also talk about how the FTAP list should be editable. But that’s a separate topic for later. (Correct me if I’m wrong.)

I think we are/could include FTAP in this discussion. It might have been how all this got started. Either way I think we know the basics:

  • FTAP should reference non-versioned Orderable, Program, and Facility Type.
  • Changes to a FTAP should be versioned.
  • Entities such as Requisition would reference the versioned Orderable and FTAP.
  • Sound correct?

Lets keep this moving. Next steps I think are:

  • Java and REST patterns created/documented for versioned Resources. We should look at FHIR STU 3 for REST and perhaps http://hapifhir.io/ for Java
  • Pattern in Java and REST for Resources which reference versioned Resources. Again same sources as above for guidance and inspiration.
  • Choose an area to start incrementally applying this. The product model is likely it, however I’d ask Sebastian’s input as an implementer here to re-iterate what he thinks we should approach first in an incremental fashion.
    Best,

Josh

On Thursday, December 7, 2017 at 7:26:32 PM UTC-8, brandon.bowersox-johnson wrote:

Of the 3 proposed approaches, Versioning is probably also the one I would also lean towards if we want to make an Orderable editable. I agree with Sebastian that implementers should be allowed to change the code and name. In the case of a typo in the code, it is fine to change, and the rest of our software references the UUID and not the code so I believe it does not break anything. Same with changing an orderable name.

The 4 open questions about Versioning still need some proposals, so I have a few responses to those:

• “Should we use GUUID or int to track version?”

GUUID sounds more robust to me and just as easy. The newer version could store a reference to the previous version.

• “Which changes should spawn new version?”

I think any changes should always spawn a new version. That is a safer and simpler solution rather than having a complex logic where certain field edits do NOT change the version number while other field edits DO change the version number. We might as well always make each edit a new version. Any big downsides to that?

• “Where should we store the version in object referencing a specific version of the orderable?”

We need to be clear on whether the other parts of OpenLMIS that reference an Orderable are also storing a reference to a specific version of the Orderable. To take one specific example, I think the Requisition object might want to store a specific version reference for each orderable. At the time of requisition initiation, it builds the Requisition using the latest version of each Orderable from the FTAP list. It also saves that version used. Then, if an Orderable changes over time (maybe code, maybe name), if I view an old requisition from 5 years ago, the on-screen view will show me the name and code from the exact version that was used back then.

I also have a few other points:

• We will need guidance or hard rules on which Orderable properties are NOT allowed to be mutable. Can anyone name an example?

• For GS1 Trade Item-based Orderables, what are the GS1 GDSN rules about mutability? If Tylenol ever changes its PackSize, its UPC Barcode or changes name to “Tylenol Zero Sugar” must they issue a whole new Trade Item?

• For UNSPSC/GPC/Commodity-Type-based Orderables, do those systems ever change the name of something (like how “Autism” was changed to “Autism Spectrum Disorder” in the DSM-5)?

• Finally, for now I believe we are just talking about making an individual Orderable editable. As a separate issue (later) we will also talk about how the FTAP list should be editable. But that’s a separate topic for later. (Correct me if I’m wrong.)

From: openlmis-dev@googlegroups.com on behalf of Sebastian Brudziński sbrudzinski@soldevelo.com

Date: Wednesday, November 29, 2017 at 10:18 AM

To: "openlmis-dev@googlegroups.com" openlmis-dev@googlegroups.com

Subject: Re: [openlmis-dev] Re: Orderable edit design

Thanks for reviving this discussion and compiling the summary, Nikodem. I think it’s important that we start making progress with the orderables edit while we are still far from the next release, since the impact of this change will be huge.

My comment on the “identity”, under “what should be editable” and “the concern” would be that it is not something that we (as a core product) need or should worry about too much. No matter how hard we try, it’s ultimately up to the implementer to ensure that the edits they are making, to any instance in the system, are not causing that instance to represent a different thing. If we wanted to enforce that on our end, we would pretty much need to disable the ability to edit most of the properties for all our entities (eg. a name, a code…).

You are proposing that we make the product code immutable. I’d personally tend towards giving more, rather than less freedom in the edits. While the code is something that generally shouldn’t change, we already had an issue with an incorrect facility code assigned in Malawi. I believe this will/should be the only case where a code of any instance is edited, but it’s good to have that option if necessary, especially that editing the code doesn’t have any impact on the system as far as I know (references are by the id). It would be great to hear what other people think though.

+1 from me for “versioning” as our strategy to edit orderables. I think it’s the cleanest solution out of the proposed ones. It can also be implemented incrementally, without affecting other services (support for versioning first, migrating services one-by-one to use the orderable version second).

Best regards,

Sebastian.

On 28.11.2017 19:07, Nikodem Graczewski wrote:

Hello everyone!

I would like to restart the orderables edit discussion. In order to do it I’ve created the following recap on the problem and proposed a solution for dealing with it.

The Problem

Making Orderables editable to allow users to fix typos and pricing adjustments.

The Pain Points

Currently we have some classes that have the associations on the orderables. We can split them into two groups that:

  • expect orderables to be immutable. These classes care about the orderable details and don’t expect the orderable to change. Here’s a list of the classes that behave this way:
  • RequisitionLineItem
  • OrderLineItem
  • don’t care if orderable has changed. All they care that the object with the specific UUID still represents the same orderable, meaning the identity of the orderable hasn’t changed. Here’s a list of class that meet this condition:
  • FacilityTypeApprovedProduct
  • VmmApplicable
  • IdentifiableByOrderableLot
  • PhysicalInventoryLineItem
  • StockCard

What should be editable?

The main concern behind this question is that allowing the users to update the orderable could lead to an orderable object actually changing identity, meaning it would represent a different orderable than it did originally. The solution for this problem would be making the product code immutable. This would enforce that the orderable can not change its identity. If the user would like to change the code she/he would most likely be doing something she/he shouldn’t. As the problem we’re trying to solve here is rather about fixing the errors and updating the price this most likely won’t be a problem, but he possibility would still remain.

By the way, this concern is most likely also valid for any other entity that is mutable in the OpenLMIS.

The Solutions

I would like to propose three possible solutions I see from which I have chosen one. I will describe them in more details below

Archiving

With this approach we would be archiving the orderable before updating it and creating a new entry within the database giving it a new UUID.

Pros:

  • No changes would have to be made to the entities that treat the orderable as immutable, the UUID would point to the exact orderable version they want.

Cons:

  • We would have to update entities that treat orderable as mutable on every update as they don’t really care about orderable details but about its identity (meaning they should point to the newest possible version of the orderable). FTAP is a great example here. In Malawi implementation there is around 60 FTAPs for a single orderable, which would have to be updated/recreated every time we update the orderable. If we picked up the recreation path the orderables table in the database could grow really huge really fast. We would also have to deal with entities that associate the orderable and update them too, and this could propagate.
  • This would be hard to maintain (see the previous bullet point).
  • The database constraints would be non-trivial, we would have to restrict the situation when there is a couple of orderables that have the same product code and are not archived (this wouldn’t be true for archived orderables).

Open questions:

  • How would we track the orderable identity if the UUID could change?

    Product code is the natural candidate for that.

  • How would we track the newest orderable version?

    We would probably have to introduce the archived flag, which should be set to false for the newest version of the orderable.

Versioning

With this approach we would add a new field to the orderable named version/versionId/something similar. The identity of the orderable would still be held by the UUID attribute.

Pros:

  • No changes need outside the endpoint during the initial implementation.
  • No changes needed at all to the entities that treat the orderable as mutable, the UUID would still point to the newest version.
  • All versions stored in one place (referencedata), no need for snapshots in any module - all they would need would be the UUID and version.
  • We’re not breaking backward combability, the currently implemented endpoint still returns the latest orderable version based on the UUID. To get the older version of the orderable we would have to pass an optional request parameter. This would lead to the endpoint being extended rather than modified.

Cons:

  • The need to identify and update all the places that use orderables and expect them to be immutable.

Open questions:

  • Should we use GUUID or int to track version?

    By using a simple int we could easily retrieve the latest version of the orderable from the database.

  • Where should we store the version in object referencing a specific version of the orderable?

    ReferencedataObject class is a perfect candidate for that.

  • Would this break the contract and lead to the major version being increased?

    The endpoint itself stays the same, but introduction of the versioning would probably lead to the behavior change meaning this would most likely break the contract and require a major version bump.

  • Which changes should spawn new version?

    Ideally, typo fixes should not lead to a version bump, but those things might be hard to detect. We could give the users ability to decide whether the update should spawn a new version of the orderable or update the newest one, but this feature would most likely confuse the end user.

Snapshooting

With this approach we would be snapshooting properties of the orderable which are expected to be immutable by the associating classes.

Pros:

  • Easy to implement.
  • This would break some dependencies leading to a lesser coupling of the fulfillment/requisition and referencedata modules at the cost of code duplication.
  • This would lead to a temporary performance improvement (less calls to the referencedata i.e. during requisition retrieval)…

Cons:

  • …which would become a hit to the performance in the long run (bigger database).
  • This would be a little WET as we would be duplicating some of the properties, but we could snapshot only those fields we want immutable and ignore orderable beyond the point of creation of the object that was snapshooting the orderable.

The Proposed Solution

I would like to propose Versioning as the go-to solution for this problem. It has the most pros with only one con, which is true for all the solutions - eventually, we will have to change something somewhere to make orderables editable flawlessly. This approach would also require least work outside the orderable endpoint. I’ve diagnosed only two places that expect the orderable to be immutable, which are requisition and order creation, rest of the associations rely on the identity only and don’t really care about the orderable details changing.

The Proposed Strategy

This is a proposed strategy for implementing the orderable edit feature. These steps should make the implementation as smooth as possible.

  1. Add the version property to the orderable and modify endpoint to respect it. Create a migration to set the version to 0 by default. Don’t implement orderable edit feature yet.
  2. Update the Requisition and Order creation and classes to actually use ReferencedataObject instead of an ID, the ReferencedataObject should be extended to include the version property (VersionedReferencedataObject class perhaps?). This step could also include removal of the snapshoot properties from the RequisitionLineItem, as the specific version of the orderable wouldn’t be editable (a little DRY is always good).
  3. Implement the orderable edit feature with automated version updating.
  4. Watch everything going smooth.

The Concern

Implementing edit feature on an entity that is treated both as mutable and immutable depending on the associating class will leave a gate for bugs to creep in that might be hard to deal with. My main concern is a situation in which user decides that editing a discontinued orderable and overriding it with a new one is a good idea. This means that the identity of the object has changed and I couldn’t come up with a way to prevent it other than making some of the fields immutable.

Kind regards,

Nikodem

On Friday, September 29, 2017 at 6:09:15 PM UTC+2, Nikodem Graczewski wrote:

Hi everyone,

as per our Sprint Q&A meeting I’m starting a discussion about how do we want to approach the Orderables edit as allowing this action can possibly result is some unforeseen consequences in some of the modules depending on them.

Here’s a list of modules that are somehow related to Orderables:

  • referencedata - holds the Orderables
  • requisition
  • fulfillment
  • stockmanagement
    I’ve done some research and I could only find one thing that would break after we implement Orderables edit. Requisition line item uses the ProgramOrderable to actually set the price per pack of itself. If we modify the ProgramsOrderable price per pack after the requisition initialization it won’t be reflected in the requisition line item.

Here’s a couple of ideas on how we could approach the problem:

  • We could create a new endpoint for updating the ProgramOrderables only, without touching the Orderable itself as proposed by Paweł Gesek here.

  • We could use the FHIR approach by versioning the Orderable. This approach would use two IDs, one for tracking the Orderable itself, so the ID of Orderable would never change and a second for tracking the Orderable version.

    This would allow us to:

  • Retrieve the newest version of Orderable simply by calling a GET endpoint with the ID. This would be used by the administration screens so they always display the newest Orderable. Modifying the Orderable would result in creation of a new version of it, and storing the previous one as history.

  • All the module that would require the older version of the Orderable would simply pass the version ID as a request parameter to the GET endpoint. This would result in retrieving a history version of an Orderable instead of the newest one.
    If you have any ideas, comments or concerns about this feature, please post them in this thread.

Best Regards,

Nikodem

**

SolDevelo** Sp. z o.o. [LLC] /
www.soldevelo.com

Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland

Phone:
+48 58 782 45 40 / Fax:
+48 58 782 45 41

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to
openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to
openlmis-dev@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/openlmis-dev/e423e6a1-d69c-4348-93de-fa5d8a619129%40googlegroups.com
.

For more options, visit
https://groups.google.com/d/optout
.

Sebastian Brudziński

Software Developer / Team Leader

sbrudzinski@soldevelo.com

**

SolDevelo** Sp. z o.o. [LLC] /
www.soldevelo.com

Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland

Phone:
+48 58 782 45 40 / Fax:
+48 58 782 45 41

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to
openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to
openlmis-dev@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/openlmis-dev/0fc3491d-3734-3bee-b1ac-6d852517ead3%40soldevelo.com
.

For more options, visit
https://groups.google.com/d/optout
.

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to
openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to
openlmis-dev@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/openlmis-dev/4bec1537-73d8-4333-a9ac-55f71047b6f0%40googlegroups.com
.

For more options, visit
https://groups.google.com/d/optout
.


(Nikodem Graczewski) #17

Hello everyone,

since the topic is slowly coming to a conclusion, I suggest that the next step should be creating tickets that will actually be responsible for starting the work on the implementation. I would split them as follows:

  1. Investigate and design versioning pattern.
  2. Implement the pattern on the Orderables (don’t add the edit endpoint yet).
  3. Update domain object that treat the Orderables as immutable by adding the version property.
  4. Implement the orderables edit endpoint, which should take care of versioning.
    However, this is something we can discuss during the next Tech Committee meeting.

Best regards,
Nikodem


SolDevelo
Sp. z o.o. [LLC] / www.soldevelo.com
Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland
Phone: +48 58 782 45 40 / Fax: +48 58 782 45 41

···

On Tue, Dec 12, 2017 at 9:18 AM, Łukasz Lewczyński llewczynski@soldevelo.com wrote:

The example with paper requisition make it very clear for me because if I made a typo on paper I will had to get another paper. Thanks for this explanation.


SolDevelo
Sp. z o.o. [LLC] / www.soldevelo.com
Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland
Phone: +48 58 782 45 40 / Fax: +48 58 782 45 41

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to openlmis-dev@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openlmis-dev/CAAdp53wdzsZdQ%2BHjAFBQgat-RPyv6CCXb1QCuc03r5GkCEtihQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

Nikodem Graczewski

Software Developer

ngraczewski@soldevelo.com

Łukasz Lewczyński
Software Developer
llewczynski@soldevelo.com

On Mon, Dec 11, 2017 at 7:17 PM, Josh Zamor josh.zamor@villagereach.org wrote:

Hi Lukasz, my thoughts:

  1. If I modified the product name (because I found a typo) should I see this change on the product grid? In my opinion yes because this change does not modify my product.

While I could see someone wanting this, there already is a simple way to get this: delete the requisition that was created and create it again. This is simple for a user to understand as it mimics what happens with paper based requisitions, i.e. it fits their mental model. Moreover doing this, instead of something fancier, keeps the separation of concerns: a typo fix to an Orderable name is the same as any other change: it creates a new version of the record. Any client which uses the Orderable now gets to decide how it wants to use the Orderable and its versions: use the latest, use a specific version, make explicit functions of it’s workflow in case the Orderable updates (e.g. while a Requisition is still in process). This is a nice separation of concerns which we should be striving for in our microservices.

  1. If I modified the product price should I see this change on the product grid? I think no because this change modifies my budget for the given period. Also if this change will be visible, the legacy data will be corrupted.

I think that’s what we’re saying with a versioned Orderable: the price wouldn’t change in the Requisition as it has a pointer to the version that was latest when it was created.

If we assume that each change creates a new version of product, how much new entries will be in database? In the worst scenario I can modify a single property, save the product, modify the next property, save it and so on.This could create a lot of rows in table.

This should be well within the capabilities of Postgres. While write operations here may be frequent, mostly we’re thinking in terms of read operations, where a UUID (primary key) and version may be put into a multi-column index. As a reference in the requisition line item table with 7+ million rows we have sub millisecond lookups in Postgres. Lets say we had 10,000 Orderables, each with 1000 changes, this would be roughly equivalent to only 10M rows in the Orderable table.

Good questions, keep them coming.

Best,

Josh

On Dec 11, 2017, at 6:22 AM, Łukasz Lewczyński llewczynski@soldevelo.com wrote:

I have small problem with versioning option. Let assume we have a productwith initial versionand create a requisition with that product and version. Now:

  1. If I modified the product name (because I found a typo) should I see this change on the product grid? In my opinion yes because this change does not modify my product.
  2. If I modified the product price should I see this change on the product grid? I think no because this change modifies my budget for the given period. Also if this change will be visible, the legacy data will be corrupted.

If we assume that each change creates a new version of product, how much new entries will be in database? In the worst scenario I can modify a single property, save the product, modify the next property, save it and so on.This could create a lot of rows in table.

**

SolDevelo** Sp. z o.o. [LLC] /
www.soldevelo.com

Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland

Phone: +48 58 782 45 40 / Fax: +48 58 782 45 41

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to
openlmis-dev@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/openlmis-dev/CAAdp53wP-4UJ4SbrOi0f0FZVs7zHm7OQPsd8QZQV4KDGOVTWSw%40mail.gmail.com
.

For more options, visit https://groups.google.com/d/optout.

Łukasz Lewczyński

Software Developer

llewczynski@soldevelo.com

On Fri, Dec 8, 2017 at 10:33 PM, Josh Zamor
josh.zamor@villagereach.org wrote:

I love where this thread is going and some of the issues that various folks have explored and brought to the fore. Thanks everyone for the effort.

Of course, I also think Versioning is our best option. Some quick responses:

  • I agree with Sebastian and Brandon that we shouldn’t worry (much) about immutable fields. I’ve been burned in an old system where an admin was updating user profiles to “create” new users. It was a learning experience, but mostly on the UI side of things. Once the UI directed the admin to what changes meant, that mistake was corrected.
  • I also agree with Brandon’s comments on what types of changes create new versions (always - and FHIR does this as well) and also on where the version should be stored in a referencing entity.
  • I disagree that we should use a UUID for the version, and instead recommend we use a serial integer starting at 1 (and make sure in the DB it has plenty of room to grow). Reasoning: FHIR STU 3 uses a serial integer starting at 1. The UUID is important for uniquely identifying things. The version of that uniquely identified thing is something that is bounded by ACID, and therefore should always increase in a predictable fashion. Note what this means is that when you change an entity (say in the UI), you never increment the version and send it back, that’s a function of the write operation on the server from which it was read.

A few responses to Brandon’s other points:

For GS1 Trade Item-based Orderables, what are the GS1 GDSN rules about mutability? If Tylenol ever changes its PackSize, its UPC Barcode or changes name to “Tylenol Zero Sugar” must they issue a whole new Trade Item?

Great question. The actual answer to this is
complex
. The simple explanation I got in the GTIN training was this: if you care about identifying yellow shirts, blue shirts and orange shirts in the supply chain, they each need a GTIN. If you just care about shirts, one GTIN will do. The document I linked is aimed at manufacturers / brand owners and when they should issue new GTINs. The key for us however is that we’re supposed to trust what they do. If they make a mistake, as recipients of their information our system can’t automatically decide the change is in error or not. GS1 I believe works to assist those creating GTINs do so correctly. Put simply: OpenLMIS should allow any attribute change which is defined in the GDSN and trust that the author understands the effect of that master data in the supply chain.

based Orderables, do those systems ever change the name of something (like how “Autism” was changed to “Autism Spectrum Disorder” in the DSM-5)?

For UNSPSC/GPC/Commodity-Type-

Another great question. This one I don’t know, and I suspect it might vary depending on the author of the classification system (WHO, GS1, country-specific). The flexibility to where these classification systems come from is intentional in our design, even though we’d hope a country chooses mature systems. Because of this flexibility though, we might err on the side of caution and version CommodityType. I’m not 100% sure and I’ve created a ticket to track it:
https://openlmis.atlassian.net/browse/OLMIS-3790

Finally, for now I believe we are just talking about making an individual Orderable editable. As a separate issue (later) we will also talk about how the FTAP list should be editable. But that’s a separate topic for later. (Correct me if I’m wrong.)

I think we are/could include FTAP in this discussion. It might have been how all this got started. Either way I think we know the basics:

  • FTAP should reference non-versioned Orderable, Program, and Facility Type.
  • Changes to a FTAP should be versioned.
  • Entities such as Requisition would reference the versioned Orderable and FTAP.
  • Sound correct?

Lets keep this moving. Next steps I think are:

  • Java and REST patterns created/documented for versioned Resources. We should look at FHIR STU 3 for REST and perhaps http://hapifhir.io/ for Java
  • Pattern in Java and REST for Resources which reference versioned Resources. Again same sources as above for guidance and inspiration.
  • Choose an area to start incrementally applying this. The product model is likely it, however I’d ask Sebastian’s input as an implementer here to re-iterate what he thinks we should approach first in an incremental fashion.
    Best,

Josh

On Thursday, December 7, 2017 at 7:26:32 PM UTC-8, brandon.bowersox-johnson wrote:

Of the 3 proposed approaches, Versioning is probably also the one I would also lean towards if we want to make an Orderable editable. I agree with Sebastian that implementers should be allowed to change the code and name. In the case of a typo in the code, it is fine to change, and the rest of our software references the UUID and not the code so I believe it does not break anything. Same with changing an orderable name.

The 4 open questions about Versioning still need some proposals, so I have a few responses to those:

• “Should we use GUUID or int to track version?”

GUUID sounds more robust to me and just as easy. The newer version could store a reference to the previous version.

• “Which changes should spawn new version?”

I think any changes should always spawn a new version. That is a safer and simpler solution rather than having a complex logic where certain field edits do NOT change the version number while other field edits DO change the version number. We might as well always make each edit a new version. Any big downsides to that?

• “Where should we store the version in object referencing a specific version of the orderable?”

We need to be clear on whether the other parts of OpenLMIS that reference an Orderable are also storing a reference to a specific version of the Orderable. To take one specific example, I think the Requisition object might want to store a specific version reference for each orderable. At the time of requisition initiation, it builds the Requisition using the latest version of each Orderable from the FTAP list. It also saves that version used. Then, if an Orderable changes over time (maybe code, maybe name), if I view an old requisition from 5 years ago, the on-screen view will show me the name and code from the exact version that was used back then.

I also have a few other points:

• We will need guidance or hard rules on which Orderable properties are NOT allowed to be mutable. Can anyone name an example?

• For GS1 Trade Item-based Orderables, what are the GS1 GDSN rules about mutability? If Tylenol ever changes its PackSize, its UPC Barcode or changes name to “Tylenol Zero Sugar” must they issue a whole new Trade Item?

• For UNSPSC/GPC/Commodity-Type-based Orderables, do those systems ever change the name of something (like how “Autism” was changed to “Autism Spectrum Disorder” in the DSM-5)?

• Finally, for now I believe we are just talking about making an individual Orderable editable. As a separate issue (later) we will also talk about how the FTAP list should be editable. But that’s a separate topic for later. (Correct me if I’m wrong.)

From: openlmis-dev@googlegroups.com on behalf of Sebastian Brudziński sbrudzinski@soldevelo.com

Date: Wednesday, November 29, 2017 at 10:18 AM

To: "openlmis-dev@googlegroups.com" openlmis-dev@googlegroups.com

Subject: Re: [openlmis-dev] Re: Orderable edit design

Thanks for reviving this discussion and compiling the summary, Nikodem. I think it’s important that we start making progress with the orderables edit while we are still far from the next release, since the impact of this change will be huge.

My comment on the “identity”, under “what should be editable” and “the concern” would be that it is not something that we (as a core product) need or should worry about too much. No matter how hard we try, it’s ultimately up to the implementer to ensure that the edits they are making, to any instance in the system, are not causing that instance to represent a different thing. If we wanted to enforce that on our end, we would pretty much need to disable the ability to edit most of the properties for all our entities (eg. a name, a code…).

You are proposing that we make the product code immutable. I’d personally tend towards giving more, rather than less freedom in the edits. While the code is something that generally shouldn’t change, we already had an issue with an incorrect facility code assigned in Malawi. I believe this will/should be the only case where a code of any instance is edited, but it’s good to have that option if necessary, especially that editing the code doesn’t have any impact on the system as far as I know (references are by the id). It would be great to hear what other people think though.

+1 from me for “versioning” as our strategy to edit orderables. I think it’s the cleanest solution out of the proposed ones. It can also be implemented incrementally, without affecting other services (support for versioning first, migrating services one-by-one to use the orderable version second).

Best regards,

Sebastian.

On 28.11.2017 19:07, Nikodem Graczewski wrote:

Hello everyone!

I would like to restart the orderables edit discussion. In order to do it I’ve created the following recap on the problem and proposed a solution for dealing with it.

The Problem

Making Orderables editable to allow users to fix typos and pricing adjustments.

The Pain Points

Currently we have some classes that have the associations on the orderables. We can split them into two groups that:

  • expect orderables to be immutable. These classes care about the orderable details and don’t expect the orderable to change. Here’s a list of the classes that behave this way:
  • RequisitionLineItem
  • OrderLineItem
  • don’t care if orderable has changed. All they care that the object with the specific UUID still represents the same orderable, meaning the identity of the orderable hasn’t changed. Here’s a list of class that meet this condition:
  • FacilityTypeApprovedProduct
  • VmmApplicable
  • IdentifiableByOrderableLot
  • PhysicalInventoryLineItem
  • StockCard

What should be editable?

The main concern behind this question is that allowing the users to update the orderable could lead to an orderable object actually changing identity, meaning it would represent a different orderable than it did originally. The solution for this problem would be making the product code immutable. This would enforce that the orderable can not change its identity. If the user would like to change the code she/he would most likely be doing something she/he shouldn’t. As the problem we’re trying to solve here is rather about fixing the errors and updating the price this most likely won’t be a problem, but he possibility would still remain.

By the way, this concern is most likely also valid for any other entity that is mutable in the OpenLMIS.

The Solutions

I would like to propose three possible solutions I see from which I have chosen one. I will describe them in more details below

Archiving

With this approach we would be archiving the orderable before updating it and creating a new entry within the database giving it a new UUID.

Pros:

  • No changes would have to be made to the entities that treat the orderable as immutable, the UUID would point to the exact orderable version they want.

Cons:

  • We would have to update entities that treat orderable as mutable on every update as they don’t really care about orderable details but about its identity (meaning they should point to the newest possible version of the orderable). FTAP is a great example here. In Malawi implementation there is around 60 FTAPs for a single orderable, which would have to be updated/recreated every time we update the orderable. If we picked up the recreation path the orderables table in the database could grow really huge really fast. We would also have to deal with entities that associate the orderable and update them too, and this could propagate.
  • This would be hard to maintain (see the previous bullet point).
  • The database constraints would be non-trivial, we would have to restrict the situation when there is a couple of orderables that have the same product code and are not archived (this wouldn’t be true for archived orderables).

Open questions:

  • How would we track the orderable identity if the UUID could change?

    Product code is the natural candidate for that.

  • How would we track the newest orderable version?

    We would probably have to introduce the archived flag, which should be set to false for the newest version of the orderable.

Versioning

With this approach we would add a new field to the orderable named version/versionId/something similar. The identity of the orderable would still be held by the UUID attribute.

Pros:

  • No changes need outside the endpoint during the initial implementation.
  • No changes needed at all to the entities that treat the orderable as mutable, the UUID would still point to the newest version.
  • All versions stored in one place (referencedata), no need for snapshots in any module - all they would need would be the UUID and version.
  • We’re not breaking backward combability, the currently implemented endpoint still returns the latest orderable version based on the UUID. To get the older version of the orderable we would have to pass an optional request parameter. This would lead to the endpoint being extended rather than modified.

Cons:

  • The need to identify and update all the places that use orderables and expect them to be immutable.

Open questions:

  • Should we use GUUID or int to track version?

    By using a simple int we could easily retrieve the latest version of the orderable from the database.

  • Where should we store the version in object referencing a specific version of the orderable?

    ReferencedataObject class is a perfect candidate for that.

  • Would this break the contract and lead to the major version being increased?

    The endpoint itself stays the same, but introduction of the versioning would probably lead to the behavior change meaning this would most likely break the contract and require a major version bump.

  • Which changes should spawn new version?

    Ideally, typo fixes should not lead to a version bump, but those things might be hard to detect. We could give the users ability to decide whether the update should spawn a new version of the orderable or update the newest one, but this feature would most likely confuse the end user.

Snapshooting

With this approach we would be snapshooting properties of the orderable which are expected to be immutable by the associating classes.

Pros:

  • Easy to implement.
  • This would break some dependencies leading to a lesser coupling of the fulfillment/requisition and referencedata modules at the cost of code duplication.
  • This would lead to a temporary performance improvement (less calls to the referencedata i.e. during requisition retrieval)…

Cons:

  • …which would become a hit to the performance in the long run (bigger database).
  • This would be a little WET as we would be duplicating some of the properties, but we could snapshot only those fields we want immutable and ignore orderable beyond the point of creation of the object that was snapshooting the orderable.

The Proposed Solution

I would like to propose Versioning as the go-to solution for this problem. It has the most pros with only one con, which is true for all the solutions - eventually, we will have to change something somewhere to make orderables editable flawlessly. This approach would also require least work outside the orderable endpoint. I’ve diagnosed only two places that expect the orderable to be immutable, which are requisition and order creation, rest of the associations rely on the identity only and don’t really care about the orderable details changing.

The Proposed Strategy

This is a proposed strategy for implementing the orderable edit feature. These steps should make the implementation as smooth as possible.

  1. Add the version property to the orderable and modify endpoint to respect it. Create a migration to set the version to 0 by default. Don’t implement orderable edit feature yet.
  2. Update the Requisition and Order creation and classes to actually use ReferencedataObject instead of an ID, the ReferencedataObject should be extended to include the version property (VersionedReferencedataObject class perhaps?). This step could also include removal of the snapshoot properties from the RequisitionLineItem, as the specific version of the orderable wouldn’t be editable (a little DRY is always good).
  3. Implement the orderable edit feature with automated version updating.
  4. Watch everything going smooth.

The Concern

Implementing edit feature on an entity that is treated both as mutable and immutable depending on the associating class will leave a gate for bugs to creep in that might be hard to deal with. My main concern is a situation in which user decides that editing a discontinued orderable and overriding it with a new one is a good idea. This means that the identity of the object has changed and I couldn’t come up with a way to prevent it other than making some of the fields immutable.

Kind regards,

Nikodem

On Friday, September 29, 2017 at 6:09:15 PM UTC+2, Nikodem Graczewski wrote:

Hi everyone,

as per our Sprint Q&A meeting I’m starting a discussion about how do we want to approach the Orderables edit as allowing this action can possibly result is some unforeseen consequences in some of the modules depending on them.

Here’s a list of modules that are somehow related to Orderables:

  • referencedata - holds the Orderables
  • requisition
  • fulfillment
  • stockmanagement
    I’ve done some research and I could only find one thing that would break after we implement Orderables edit. Requisition line item uses the ProgramOrderable to actually set the price per pack of itself. If we modify the ProgramsOrderable price per pack after the requisition initialization it won’t be reflected in the requisition line item.

Here’s a couple of ideas on how we could approach the problem:

  • We could create a new endpoint for updating the ProgramOrderables only, without touching the Orderable itself as proposed by Paweł Gesek here.

  • We could use the FHIR approach by versioning the Orderable. This approach would use two IDs, one for tracking the Orderable itself, so the ID of Orderable would never change and a second for tracking the Orderable version.

    This would allow us to:

  • Retrieve the newest version of Orderable simply by calling a GET endpoint with the ID. This would be used by the administration screens so they always display the newest Orderable. Modifying the Orderable would result in creation of a new version of it, and storing the previous one as history.

  • All the module that would require the older version of the Orderable would simply pass the version ID as a request parameter to the GET endpoint. This would result in retrieving a history version of an Orderable instead of the newest one.
    If you have any ideas, comments or concerns about this feature, please post them in this thread.

Best Regards,

Nikodem

**

SolDevelo** Sp. z o.o. [LLC] /
www.soldevelo.com

Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland

Phone:
+48 58 782 45 40 / Fax:
+48 58 782 45 41

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to
openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to
openlmis-dev@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/openlmis-dev/e423e6a1-d69c-4348-93de-fa5d8a619129%40googlegroups.com
.

For more options, visit
https://groups.google.com/d/optout
.

Sebastian Brudziński

Software Developer / Team Leader

sbrudzinski@soldevelo.com

**

SolDevelo** Sp. z o.o. [LLC] /
www.soldevelo.com

Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland

Phone:
+48 58 782 45 40 / Fax:
+48 58 782 45 41

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to
openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to
openlmis-dev@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/openlmis-dev/0fc3491d-3734-3bee-b1ac-6d852517ead3%40soldevelo.com
.

For more options, visit
https://groups.google.com/d/optout
.

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to
openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to
openlmis-dev@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/openlmis-dev/4bec1537-73d8-4333-a9ac-55f71047b6f0%40googlegroups.com
.

For more options, visit
https://groups.google.com/d/optout
.


(Chongsun Ahn) #18

Hey everyone,

So I have recently worked on implementing the first part of this design in OLMIS-3885. Some technical updates/details:

  • Since orderable ids would no longer be unique, a new composite primary key of both orderable id and versionId was made.
  • There were some challenges in getting JPA/Hibernate to work well with a composite primary key, especially because Hibernate usually auto-generates ids.
  • Because there are several foreign keys to the orderables table (including facility_type_approved_products), adding a versionId now has added versionIds to all of these child tables.
  • When an orderable response JSON is returned, the versionId is in a property called “meta”.

Next steps for this orderable edit work:

  • Facility type approved products (FTAPs) also need to have a version property (OLMIS-5308).
  • Orderables need to have their version incremented when edited/updated (OLMIS-3887 or OLMIS-3933?).
  • FTAPs need to have their version incremented when edited/updated (OLMIS-3933).

If you have any comments or questions, feel free to respond.

Shalom,

Chongsun

– ​

There are 10 kinds of people in this world; those who understand binary, and those who don’t.

Chongsun Ahn | chongsun.ahn@villagereach.org

Software Development Engineer

Village****Reach* ** Starting at the Last Mile*

2900 Eastlake Ave. E, Suite 230, Seattle, WA 98102, USA

DIRECT: 1.206.512.1536 **CELL: **1.206.910.0973 FAX: 1.206.860.6972

SKYPE: chongsun.ahn.vr

www.villagereach.org

Connect on Facebook****, Twitter** ** and our Blog

···

On Tue, Dec 12, 2017 at 9:18 AM, Łukasz Lewczyński
llewczynski@soldevelo.com wrote:

The example with paper requisition make it very clear for me because if I made a typo on paper I will had to get another paper. Thanks for this explanation.

**

SolDevelo** Sp. z o.o. [LLC] /
www.soldevelo.com

Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland

Phone: +48 58 782 45 40 / Fax: +48 58 782 45 41

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to
openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to
openlmis-dev@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/openlmis-dev/CAAdp53wdzsZdQ%2BHjAFBQgat-RPyv6CCXb1QCuc03r5GkCEtihQ%40mail.gmail.com
.

For more options, visit
https://groups.google.com/d/optout
.

**Nikodem Graczewski
**

Software Developer

ngraczewski@soldevelo.com

Łukasz Lewczyński

Software Developer

llewczynski@soldevelo.com

On Mon, Dec 11, 2017 at 7:17 PM, Josh Zamor
josh.zamor@villagereach.org wrote:

Hi Lukasz, my thoughts:

  1. If I modified the product name (because I found a typo) should I see this change on the product grid? In my opinion yes because this change does not modify my product.

While I could see someone wanting this, there already is a simple way to get this: delete the requisition that was created and create it again. This is simple for a user to understand as it mimics what happens with paper based requisitions, i.e. it fits their mental model. Moreover doing this, instead of something fancier, keeps the separation of concerns: a typo fix to an Orderable name is the same as any other change: it creates a new version of the record. Any client which uses the Orderable now gets to decide how it wants to use the Orderable and its versions: use the latest, use a specific version, make explicit functions of it’s workflow in case the Orderable updates (e.g. while a Requisition is still in process). This is a nice separation of concerns which we should be striving for in our microservices.

  1. If I modified the product price should I see this change on the product grid? I think no because this change modifies my budget for the given period. Also if this change will be visible, the legacy data will be corrupted.

I think that’s what we’re saying with a versioned Orderable: the price wouldn’t change in the Requisition as it has a pointer to the version that was latest when it was created.

If we assume that each change creates a new version of product, how much new entries will be in database? In the worst scenario I can modify a single property, save the product, modify the next property, save it and so on.This could create a lot of rows in table.

This should be well within the capabilities of Postgres. While write operations here may be frequent, mostly we’re thinking in terms of read operations, where a UUID (primary key) and version may be put into a multi-column index. As a reference in the requisition line item table with 7+ million rows we have sub millisecond lookups in Postgres. Lets say we had 10,000 Orderables, each with 1000 changes, this would be roughly equivalent to only 10M rows in the Orderable table.

Good questions, keep them coming.

Best,

Josh

On Dec 11, 2017, at 6:22 AM, Łukasz Lewczyński llewczynski@soldevelo.com wrote:

I have small problem with versioning option. Let assume we have a productwith initial versionand create a requisition with that product and version. Now:

  1. If I modified the product name (because I found a typo) should I see this change on the product grid? In my opinion yes because this change does not modify my product.
  2. If I modified the product price should I see this change on the product grid? I think no because this change modifies my budget for the given period. Also if this change will be visible, the legacy data will be corrupted.

If we assume that each change creates a new version of product, how much new entries will be in database? In the worst scenario I can modify a single property, save the product, modify the next property, save it and so on.This could create a lot of rows in table.

**

SolDevelo** Sp. z o.o. [LLC] /
www.soldevelo.com

Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland

Phone: +48 58 782 45 40 / Fax: +48 58 782 45 41

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to
openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to
openlmis-dev@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/openlmis-dev/CAAdp53wP-4UJ4SbrOi0f0FZVs7zHm7OQPsd8QZQV4KDGOVTWSw%40mail.gmail.com
.

For more options, visit
https://groups.google.com/d/optout
.

Łukasz Lewczyński

Software Developer

llewczynski@soldevelo.com

On Fri, Dec 8, 2017 at 10:33 PM, Josh Zamor
josh.zamor@villagereach.org wrote:

I love where this thread is going and some of the issues that various folks have explored and brought to the fore. Thanks everyone for the effort.

Of course, I also think Versioning is our best option. Some quick responses:

  • I agree with Sebastian and Brandon that we shouldn’t worry (much) about immutable fields. I’ve been burned in an old system where an admin was updating user profiles to “create” new users. It was a learning experience, but mostly on the UI side of things. Once the UI directed the admin to what changes meant, that mistake was corrected.
  • I also agree with Brandon’s comments on what types of changes create new versions (always - and FHIR does this as well) and also on where the version should be stored in a referencing entity.
  • I disagree that we should use a UUID for the version, and instead recommend we use a serial integer starting at 1 (and make sure in the DB it has plenty of room to grow). Reasoning: FHIR STU 3 uses a serial integer starting at 1. The UUID is important for uniquely identifying things. The version of that uniquely identified thing is something that is bounded by ACID, and therefore should always increase in a predictable fashion. Note what this means is that when you change an entity (say in the UI), you never increment the version and send it back, that’s a function of the write operation on the server from which it was read.

A few responses to Brandon’s other points:

For GS1 Trade Item-based Orderables, what are the GS1 GDSN rules about mutability? If Tylenol ever changes its PackSize, its UPC Barcode or changes name to “Tylenol Zero Sugar” must they issue a whole new Trade Item?

Great question. The actual answer to this is
complex
. The simple explanation I got in the GTIN training was this: if you care about identifying yellow shirts, blue shirts and orange shirts in the supply chain, they each need a GTIN. If you just care about shirts, one GTIN will do. The document I linked is aimed at manufacturers / brand owners and when they should issue new GTINs. The key for us however is that we’re supposed to trust what they do. If they make a mistake, as recipients of their information our system can’t automatically decide the change is in error or not. GS1 I believe works to assist those creating GTINs do so correctly. Put simply: OpenLMIS should allow any attribute change which is defined in the GDSN and trust that the author understands the effect of that master data in the supply chain.

based Orderables, do those systems ever change the name of something (like how “Autism” was changed to “Autism Spectrum Disorder” in the DSM-5)?

For UNSPSC/GPC/Commodity-Type-

Another great question. This one I don’t know, and I suspect it might vary depending on the author of the classification system (WHO, GS1, country-specific). The flexibility to where these classification systems come from is intentional in our design, even though we’d hope a country chooses mature systems. Because of this flexibility though, we might err on the side of caution and version CommodityType. I’m not 100% sure and I’ve created a ticket to track it:
https://openlmis.atlassian.net/browse/OLMIS-3790

Finally, for now I believe we are just talking about making an individual Orderable editable. As a separate issue (later) we will also talk about how the FTAP list should be editable. But that’s a separate topic for later. (Correct me if I’m wrong.)

I think we are/could include FTAP in this discussion. It might have been how all this got started. Either way I think we know the basics:

  • FTAP should reference non-versioned Orderable, Program, and Facility Type.
  • Changes to a FTAP should be versioned.
  • Entities such as Requisition would reference the versioned Orderable and FTAP.
  • Sound correct?

Lets keep this moving. Next steps I think are:

  • Java and REST patterns created/documented for versioned Resources. We should look at FHIR STU 3 for REST and perhaps http://hapifhir.io/ for Java
  • Pattern in Java and REST for Resources which reference versioned Resources. Again same sources as above for guidance and inspiration.
  • Choose an area to start incrementally applying this. The product model is likely it, however I’d ask Sebastian’s input as an implementer here to re-iterate what he thinks we should approach first in an incremental fashion.
    Best,

Josh

On Thursday, December 7, 2017 at 7:26:32 PM UTC-8, brandon.bowersox-johnson wrote:

Of the 3 proposed approaches, Versioning is probably also the one I would also lean towards if we want to make an Orderable editable. I agree with Sebastian that implementers should be allowed to change the code and name. In the case of a typo in the code, it is fine to change, and the rest of our software references the UUID and not the code so I believe it does not break anything. Same with changing an orderable name.

The 4 open questions about Versioning still need some proposals, so I have a few responses to those:

• “Should we use GUUID or int to track version?”

GUUID sounds more robust to me and just as easy. The newer version could store a reference to the previous version.

• “Which changes should spawn new version?”

I think any changes should always spawn a new version. That is a safer and simpler solution rather than having a complex logic where certain field edits do NOT change the version number while other field edits DO change the version number. We might as well always make each edit a new version. Any big downsides to that?

• “Where should we store the version in object referencing a specific version of the orderable?”

We need to be clear on whether the other parts of OpenLMIS that reference an Orderable are also storing a reference to a specific version of the Orderable. To take one specific example, I think the Requisition object might want to store a specific version reference for each orderable. At the time of requisition initiation, it builds the Requisition using the latest version of each Orderable from the FTAP list. It also saves that version used. Then, if an Orderable changes over time (maybe code, maybe name), if I view an old requisition from 5 years ago, the on-screen view will show me the name and code from the exact version that was used back then.

I also have a few other points:

• We will need guidance or hard rules on which Orderable properties are NOT allowed to be mutable. Can anyone name an example?

• For GS1 Trade Item-based Orderables, what are the GS1 GDSN rules about mutability? If Tylenol ever changes its PackSize, its UPC Barcode or changes name to “Tylenol Zero Sugar” must they issue a whole new Trade Item?

• For UNSPSC/GPC/Commodity-Type-based Orderables, do those systems ever change the name of something (like how “Autism” was changed to “Autism Spectrum Disorder” in the DSM-5)?

• Finally, for now I believe we are just talking about making an individual Orderable editable. As a separate issue (later) we will also talk about how the FTAP list should be editable. But that’s a separate topic for later. (Correct me if I’m wrong.)

From: openlmis-dev@googlegroups.com on behalf of Sebastian Brudziński sbrudzinski@soldevelo.com

Date: Wednesday, November 29, 2017 at 10:18 AM

To: "openlmis-dev@googlegroups.com" openlmis-dev@googlegroups.com

Subject: Re: [openlmis-dev] Re: Orderable edit design

Thanks for reviving this discussion and compiling the summary, Nikodem. I think it’s important that we start making progress with the orderables edit while we are still far from the next release, since the impact of this change will be huge.

My comment on the “identity”, under “what should be editable” and “the concern” would be that it is not something that we (as a core product) need or should worry about too much. No matter how hard we try, it’s ultimately up to the implementer to ensure that the edits they are making, to any instance in the system, are not causing that instance to represent a different thing. If we wanted to enforce that on our end, we would pretty much need to disable the ability to edit most of the properties for all our entities (eg. a name, a code…).

You are proposing that we make the product code immutable. I’d personally tend towards giving more, rather than less freedom in the edits. While the code is something that generally shouldn’t change, we already had an issue with an incorrect facility code assigned in Malawi. I believe this will/should be the only case where a code of any instance is edited, but it’s good to have that option if necessary, especially that editing the code doesn’t have any impact on the system as far as I know (references are by the id). It would be great to hear what other people think though.

+1 from me for “versioning” as our strategy to edit orderables. I think it’s the cleanest solution out of the proposed ones. It can also be implemented incrementally, without affecting other services (support for versioning first, migrating services one-by-one to use the orderable version second).

Best regards,

Sebastian.

On 28.11.2017 19:07, Nikodem Graczewski wrote:

Hello everyone!

I would like to restart the orderables edit discussion. In order to do it I’ve created the following recap on the problem and proposed a solution for dealing with it.

The Problem

Making Orderables editable to allow users to fix typos and pricing adjustments.

The Pain Points

Currently we have some classes that have the associations on the orderables. We can split them into two groups that:

  • expect orderables to be immutable. These classes care about the orderable details and don’t expect the orderable to change. Here’s a list of the classes that behave this way:
  • RequisitionLineItem
  • OrderLineItem
  • don’t care if orderable has changed. All they care that the object with the specific UUID still represents the same orderable, meaning the identity of the orderable hasn’t changed. Here’s a list of class that meet this condition:
  • FacilityTypeApprovedProduct
  • VmmApplicable
  • IdentifiableByOrderableLot
  • PhysicalInventoryLineItem
  • StockCard

What should be editable?

The main concern behind this question is that allowing the users to update the orderable could lead to an orderable object actually changing identity, meaning it would represent a different orderable than it did originally. The solution for this problem would be making the product code immutable. This would enforce that the orderable can not change its identity. If the user would like to change the code she/he would most likely be doing something she/he shouldn’t. As the problem we’re trying to solve here is rather about fixing the errors and updating the price this most likely won’t be a problem, but he possibility would still remain.

By the way, this concern is most likely also valid for any other entity that is mutable in the OpenLMIS.

The Solutions

I would like to propose three possible solutions I see from which I have chosen one. I will describe them in more details below

Archiving

With this approach we would be archiving the orderable before updating it and creating a new entry within the database giving it a new UUID.

Pros:

  • No changes would have to be made to the entities that treat the orderable as immutable, the UUID would point to the exact orderable version they want.

Cons:

  • We would have to update entities that treat orderable as mutable on every update as they don’t really care about orderable details but about its identity (meaning they should point to the newest possible version of the orderable). FTAP is a great example here. In Malawi implementation there is around 60 FTAPs for a single orderable, which would have to be updated/recreated every time we update the orderable. If we picked up the recreation path the orderables table in the database could grow really huge really fast. We would also have to deal with entities that associate the orderable and update them too, and this could propagate.
  • This would be hard to maintain (see the previous bullet point).
  • The database constraints would be non-trivial, we would have to restrict the situation when there is a couple of orderables that have the same product code and are not archived (this wouldn’t be true for archived orderables).

Open questions:

  • How would we track the orderable identity if the UUID could change?

    Product code is the natural candidate for that.

  • How would we track the newest orderable version?

    We would probably have to introduce the archived flag, which should be set to false for the newest version of the orderable.

Versioning

With this approach we would add a new field to the orderable named version/versionId/something similar. The identity of the orderable would still be held by the UUID attribute.

Pros:

  • No changes need outside the endpoint during the initial implementation.
  • No changes needed at all to the entities that treat the orderable as mutable, the UUID would still point to the newest version.
  • All versions stored in one place (referencedata), no need for snapshots in any module - all they would need would be the UUID and version.
  • We’re not breaking backward combability, the currently implemented endpoint still returns the latest orderable version based on the UUID. To get the older version of the orderable we would have to pass an optional request parameter. This would lead to the endpoint being extended rather than modified.

Cons:

  • The need to identify and update all the places that use orderables and expect them to be immutable.

Open questions:

  • Should we use GUUID or int to track version?

    By using a simple int we could easily retrieve the latest version of the orderable from the database.

  • Where should we store the version in object referencing a specific version of the orderable?

    ReferencedataObject class is a perfect candidate for that.

  • Would this break the contract and lead to the major version being increased?

    The endpoint itself stays the same, but introduction of the versioning would probably lead to the behavior change meaning this would most likely break the contract and require a major version bump.

  • Which changes should spawn new version?

    Ideally, typo fixes should not lead to a version bump, but those things might be hard to detect. We could give the users ability to decide whether the update should spawn a new version of the orderable or update the newest one, but this feature would most likely confuse the end user.

Snapshooting

With this approach we would be snapshooting properties of the orderable which are expected to be immutable by the associating classes.

Pros:

  • Easy to implement.
  • This would break some dependencies leading to a lesser coupling of the fulfillment/requisition and referencedata modules at the cost of code duplication.
  • This would lead to a temporary performance improvement (less calls to the referencedata i.e. during requisition retrieval)…

Cons:

  • …which would become a hit to the performance in the long run (bigger database).
  • This would be a little WET as we would be duplicating some of the properties, but we could snapshot only those fields we want immutable and ignore orderable beyond the point of creation of the object that was snapshooting the orderable.

The Proposed Solution

I would like to propose Versioning as the go-to solution for this problem. It has the most pros with only one con, which is true for all the solutions - eventually, we will have to change something somewhere to make orderables editable flawlessly. This approach would also require least work outside the orderable endpoint. I’ve diagnosed only two places that expect the orderable to be immutable, which are requisition and order creation, rest of the associations rely on the identity only and don’t really care about the orderable details changing.

The Proposed Strategy

This is a proposed strategy for implementing the orderable edit feature. These steps should make the implementation as smooth as possible.

  1. Add the version property to the orderable and modify endpoint to respect it. Create a migration to set the version to 0 by default. Don’t implement orderable edit feature yet.
  2. Update the Requisition and Order creation and classes to actually use ReferencedataObject instead of an ID, the ReferencedataObject should be extended to include the version property (VersionedReferencedataObject class perhaps?). This step could also include removal of the snapshoot properties from the RequisitionLineItem, as the specific version of the orderable wouldn’t be editable (a little DRY is always good).
  3. Implement the orderable edit feature with automated version updating.
  4. Watch everything going smooth.

The Concern

Implementing edit feature on an entity that is treated both as mutable and immutable depending on the associating class will leave a gate for bugs to creep in that might be hard to deal with. My main concern is a situation in which user decides that editing a discontinued orderable and overriding it with a new one is a good idea. This means that the identity of the object has changed and I couldn’t come up with a way to prevent it other than making some of the fields immutable.

Kind regards,

Nikodem

On Friday, September 29, 2017 at 6:09:15 PM UTC+2, Nikodem Graczewski wrote:

Hi everyone,

as per our Sprint Q&A meeting I’m starting a discussion about how do we want to approach the Orderables edit as allowing this action can possibly result is some unforeseen consequences in some of the modules depending on them.

Here’s a list of modules that are somehow related to Orderables:

  • referencedata - holds the Orderables
  • requisition
  • fulfillment
  • stockmanagement
    I’ve done some research and I could only find one thing that would break after we implement Orderables edit. Requisition line item uses the ProgramOrderable to actually set the price per pack of itself. If we modify the ProgramsOrderable price per pack after the requisition initialization it won’t be reflected in the requisition line item.

Here’s a couple of ideas on how we could approach the problem:

  • We could create a new endpoint for updating the ProgramOrderables only, without touching the Orderable itself as proposed by Paweł Gesek here.

  • We could use the FHIR approach by versioning the Orderable. This approach would use two IDs, one for tracking the Orderable itself, so the ID of Orderable would never change and a second for tracking the Orderable version.

    This would allow us to:

  • Retrieve the newest version of Orderable simply by calling a GET endpoint with the ID. This would be used by the administration screens so they always display the newest Orderable. Modifying the Orderable would result in creation of a new version of it, and storing the previous one as history.

  • All the module that would require the older version of the Orderable would simply pass the version ID as a request parameter to the GET endpoint. This would result in retrieving a history version of an Orderable instead of the newest one.
    If you have any ideas, comments or concerns about this feature, please post them in this thread.

Best Regards,

Nikodem

**

SolDevelo** Sp. z o.o. [LLC] /
www.soldevelo.com

Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland

Phone:
+48 58 782 45 40 / Fax:
+48 58 782 45 41

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to
openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to
openlmis-dev@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/openlmis-dev/e423e6a1-d69c-4348-93de-fa5d8a619129%40googlegroups.com
.

For more options, visit
https://groups.google.com/d/optout
.

Sebastian Brudziński

Software Developer / Team Leader

sbrudzinski@soldevelo.com

**

SolDevelo** Sp. z o.o. [LLC] /
www.soldevelo.com

Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland

Phone:
+48 58 782 45 40 / Fax:
+48 58 782 45 41

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to
openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to
openlmis-dev@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/openlmis-dev/0fc3491d-3734-3bee-b1ac-6d852517ead3%40soldevelo.com
.

For more options, visit
https://groups.google.com/d/optout
.

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to
openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to
openlmis-dev@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/openlmis-dev/4bec1537-73d8-4333-a9ac-55f71047b6f0%40googlegroups.com
.

For more options, visit
https://groups.google.com/d/optout
.


(Łukasz Lewczyński) #19

Hi,

I have one question that I asked in OLMIS-3885 but I did not get a response yet. In many places, we retrieve orderables and FTAPs by using the batch fetch so we call an endpoint only one time. Those endpoints are GET and handle id field that can be used multiple time. Now when we have the additional field - versionId - how could we handle that situation? I am not sure if a list of ids and list of versionIds would be correct here:

/api/orderables?id=<>&id=<>&versionId=<>&versionId=<>

Mainly because there is no way to said which id and versionId should be used to find an orderable. Another option is to add a POST search endpoint. What is your thought about this problem?

···

Łukasz Lewczyński
Software Developer
llewczynski@soldevelo.com

**Nikodem Graczewski
**

Software Developer

ngraczewski@soldevelo.com

On Tue, Dec 12, 2017 at 9:18 AM, Łukasz Lewczyński
llewczynski@soldevelo.com wrote:

The example with paper requisition make it very clear for me because if I made a typo on paper I will had to get another paper. Thanks for this explanation.

**

SolDevelo** Sp. z o.o. [LLC] /
www.soldevelo.com

Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland

Phone: +48 58 782 45 40 / Fax: +48 58 782 45 41

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to
openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to
openlmis-dev@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/openlmis-dev/CAAdp53wdzsZdQ%2BHjAFBQgat-RPyv6CCXb1QCuc03r5GkCEtihQ%40mail.gmail.com
.

For more options, visit
https://groups.google.com/d/optout
.

Łukasz Lewczyński

Software Developer

llewczynski@soldevelo.com

On Mon, Dec 11, 2017 at 7:17 PM, Josh Zamor
josh.zamor@villagereach.org wrote:

Hi Lukasz, my thoughts:

  1. If I modified the product name (because I found a typo) should I see this change on the product grid? In my opinion yes because this change does not modify my product.

While I could see someone wanting this, there already is a simple way to get this: delete the requisition that was created and create it again. This is simple for a user to understand as it mimics what happens with paper based requisitions, i.e. it fits their mental model. Moreover doing this, instead of something fancier, keeps the separation of concerns: a typo fix to an Orderable name is the same as any other change: it creates a new version of the record. Any client which uses the Orderable now gets to decide how it wants to use the Orderable and its versions: use the latest, use a specific version, make explicit functions of it’s workflow in case the Orderable updates (e.g. while a Requisition is still in process). This is a nice separation of concerns which we should be striving for in our microservices.

  1. If I modified the product price should I see this change on the product grid? I think no because this change modifies my budget for the given period. Also if this change will be visible, the legacy data will be corrupted.

I think that’s what we’re saying with a versioned Orderable: the price wouldn’t change in the Requisition as it has a pointer to the version that was latest when it was created.

If we assume that each change creates a new version of product, how much new entries will be in database? In the worst scenario I can modify a single property, save the product, modify the next property, save it and so on.This could create a lot of rows in table.

This should be well within the capabilities of Postgres. While write operations here may be frequent, mostly we’re thinking in terms of read operations, where a UUID (primary key) and version may be put into a multi-column index. As a reference in the requisition line item table with 7+ million rows we have sub millisecond lookups in Postgres. Lets say we had 10,000 Orderables, each with 1000 changes, this would be roughly equivalent to only 10M rows in the Orderable table.

Good questions, keep them coming.

Best,

Josh

On Dec 11, 2017, at 6:22 AM, Łukasz Lewczyński llewczynski@soldevelo.com wrote:

I have small problem with versioning option. Let assume we have a productwith initial versionand create a requisition with that product and version. Now:

  1. If I modified the product name (because I found a typo) should I see this change on the product grid? In my opinion yes because this change does not modify my product.
  2. If I modified the product price should I see this change on the product grid? I think no because this change modifies my budget for the given period. Also if this change will be visible, the legacy data will be corrupted.

If we assume that each change creates a new version of product, how much new entries will be in database? In the worst scenario I can modify a single property, save the product, modify the next property, save it and so on.This could create a lot of rows in table.

**

SolDevelo** Sp. z o.o. [LLC] /
www.soldevelo.com

Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland

Phone: +48 58 782 45 40 / Fax: +48 58 782 45 41

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to
openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to
openlmis-dev@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/openlmis-dev/CAAdp53wP-4UJ4SbrOi0f0FZVs7zHm7OQPsd8QZQV4KDGOVTWSw%40mail.gmail.com
.

For more options, visit
https://groups.google.com/d/optout
.

Łukasz Lewczyński

Software Developer

llewczynski@soldevelo.com

On Fri, Dec 8, 2017 at 10:33 PM, Josh Zamor
josh.zamor@villagereach.org wrote:

I love where this thread is going and some of the issues that various folks have explored and brought to the fore. Thanks everyone for the effort.

Of course, I also think Versioning is our best option. Some quick responses:

  • I agree with Sebastian and Brandon that we shouldn’t worry (much) about immutable fields. I’ve been burned in an old system where an admin was updating user profiles to “create” new users. It was a learning experience, but mostly on the UI side of things. Once the UI directed the admin to what changes meant, that mistake was corrected.
  • I also agree with Brandon’s comments on what types of changes create new versions (always - and FHIR does this as well) and also on where the version should be stored in a referencing entity.
  • I disagree that we should use a UUID for the version, and instead recommend we use a serial integer starting at 1 (and make sure in the DB it has plenty of room to grow). Reasoning: FHIR STU 3 uses a serial integer starting at 1. The UUID is important for uniquely identifying things. The version of that uniquely identified thing is something that is bounded by ACID, and therefore should always increase in a predictable fashion. Note what this means is that when you change an entity (say in the UI), you never increment the version and send it back, that’s a function of the write operation on the server from which it was read.

A few responses to Brandon’s other points:

For GS1 Trade Item-based Orderables, what are the GS1 GDSN rules about mutability? If Tylenol ever changes its PackSize, its UPC Barcode or changes name to “Tylenol Zero Sugar” must they issue a whole new Trade Item?

Great question. The actual answer to this is
complex
. The simple explanation I got in the GTIN training was this: if you care about identifying yellow shirts, blue shirts and orange shirts in the supply chain, they each need a GTIN. If you just care about shirts, one GTIN will do. The document I linked is aimed at manufacturers / brand owners and when they should issue new GTINs. The key for us however is that we’re supposed to trust what they do. If they make a mistake, as recipients of their information our system can’t automatically decide the change is in error or not. GS1 I believe works to assist those creating GTINs do so correctly. Put simply: OpenLMIS should allow any attribute change which is defined in the GDSN and trust that the author understands the effect of that master data in the supply chain.

based Orderables, do those systems ever change the name of something (like how “Autism” was changed to “Autism Spectrum Disorder” in the DSM-5)?

For UNSPSC/GPC/Commodity-Type-

Another great question. This one I don’t know, and I suspect it might vary depending on the author of the classification system (WHO, GS1, country-specific). The flexibility to where these classification systems come from is intentional in our design, even though we’d hope a country chooses mature systems. Because of this flexibility though, we might err on the side of caution and version CommodityType. I’m not 100% sure and I’ve created a ticket to track it:
https://openlmis.atlassian.net/browse/OLMIS-3790

Finally, for now I believe we are just talking about making an individual Orderable editable. As a separate issue (later) we will also talk about how the FTAP list should be editable. But that’s a separate topic for later. (Correct me if I’m wrong.)

I think we are/could include FTAP in this discussion. It might have been how all this got started. Either way I think we know the basics:

  • FTAP should reference non-versioned Orderable, Program, and Facility Type.
  • Changes to a FTAP should be versioned.
  • Entities such as Requisition would reference the versioned Orderable and FTAP.
  • Sound correct?

Lets keep this moving. Next steps I think are:

  • Java and REST patterns created/documented for versioned Resources. We should look at FHIR STU 3 for REST and perhaps http://hapifhir.io/ for Java
  • Pattern in Java and REST for Resources which reference versioned Resources. Again same sources as above for guidance and inspiration.
  • Choose an area to start incrementally applying this. The product model is likely it, however I’d ask Sebastian’s input as an implementer here to re-iterate what he thinks we should approach first in an incremental fashion.
    Best,

Josh

On Thursday, December 7, 2017 at 7:26:32 PM UTC-8, brandon.bowersox-johnson wrote:

Of the 3 proposed approaches, Versioning is probably also the one I would also lean towards if we want to make an Orderable editable. I agree with Sebastian that implementers should be allowed to change the code and name. In the case of a typo in the code, it is fine to change, and the rest of our software references the UUID and not the code so I believe it does not break anything. Same with changing an orderable name.

The 4 open questions about Versioning still need some proposals, so I have a few responses to those:

• “Should we use GUUID or int to track version?”

GUUID sounds more robust to me and just as easy. The newer version could store a reference to the previous version.

• “Which changes should spawn new version?”

I think any changes should always spawn a new version. That is a safer and simpler solution rather than having a complex logic where certain field edits do NOT change the version number while other field edits DO change the version number. We might as well always make each edit a new version. Any big downsides to that?

• “Where should we store the version in object referencing a specific version of the orderable?”

We need to be clear on whether the other parts of OpenLMIS that reference an Orderable are also storing a reference to a specific version of the Orderable. To take one specific example, I think the Requisition object might want to store a specific version reference for each orderable. At the time of requisition initiation, it builds the Requisition using the latest version of each Orderable from the FTAP list. It also saves that version used. Then, if an Orderable changes over time (maybe code, maybe name), if I view an old requisition from 5 years ago, the on-screen view will show me the name and code from the exact version that was used back then.

I also have a few other points:

• We will need guidance or hard rules on which Orderable properties are NOT allowed to be mutable. Can anyone name an example?

• For GS1 Trade Item-based Orderables, what are the GS1 GDSN rules about mutability? If Tylenol ever changes its PackSize, its UPC Barcode or changes name to “Tylenol Zero Sugar” must they issue a whole new Trade Item?

• For UNSPSC/GPC/Commodity-Type-based Orderables, do those systems ever change the name of something (like how “Autism” was changed to “Autism Spectrum Disorder” in the DSM-5)?

• Finally, for now I believe we are just talking about making an individual Orderable editable. As a separate issue (later) we will also talk about how the FTAP list should be editable. But that’s a separate topic for later. (Correct me if I’m wrong.)

From: openlmis-dev@googlegroups.com on behalf of Sebastian Brudziński sbrudzinski@soldevelo.com

Date: Wednesday, November 29, 2017 at 10:18 AM

To: "openlmis-dev@googlegroups.com" openlmis-dev@googlegroups.com

Subject: Re: [openlmis-dev] Re: Orderable edit design

Thanks for reviving this discussion and compiling the summary, Nikodem. I think it’s important that we start making progress with the orderables edit while we are still far from the next release, since the impact of this change will be huge.

My comment on the “identity”, under “what should be editable” and “the concern” would be that it is not something that we (as a core product) need or should worry about too much. No matter how hard we try, it’s ultimately up to the implementer to ensure that the edits they are making, to any instance in the system, are not causing that instance to represent a different thing. If we wanted to enforce that on our end, we would pretty much need to disable the ability to edit most of the properties for all our entities (eg. a name, a code…).

You are proposing that we make the product code immutable. I’d personally tend towards giving more, rather than less freedom in the edits. While the code is something that generally shouldn’t change, we already had an issue with an incorrect facility code assigned in Malawi. I believe this will/should be the only case where a code of any instance is edited, but it’s good to have that option if necessary, especially that editing the code doesn’t have any impact on the system as far as I know (references are by the id). It would be great to hear what other people think though.

+1 from me for “versioning” as our strategy to edit orderables. I think it’s the cleanest solution out of the proposed ones. It can also be implemented incrementally, without affecting other services (support for versioning first, migrating services one-by-one to use the orderable version second).

Best regards,

Sebastian.

On 28.11.2017 19:07, Nikodem Graczewski wrote:

Hello everyone!

I would like to restart the orderables edit discussion. In order to do it I’ve created the following recap on the problem and proposed a solution for dealing with it.

The Problem

Making Orderables editable to allow users to fix typos and pricing adjustments.

The Pain Points

Currently we have some classes that have the associations on the orderables. We can split them into two groups that:

  • expect orderables to be immutable. These classes care about the orderable details and don’t expect the orderable to change. Here’s a list of the classes that behave this way:
  • RequisitionLineItem
  • OrderLineItem
  • don’t care if orderable has changed. All they care that the object with the specific UUID still represents the same orderable, meaning the identity of the orderable hasn’t changed. Here’s a list of class that meet this condition:
  • FacilityTypeApprovedProduct
  • VmmApplicable
  • IdentifiableByOrderableLot
  • PhysicalInventoryLineItem
  • StockCard

What should be editable?

The main concern behind this question is that allowing the users to update the orderable could lead to an orderable object actually changing identity, meaning it would represent a different orderable than it did originally. The solution for this problem would be making the product code immutable. This would enforce that the orderable can not change its identity. If the user would like to change the code she/he would most likely be doing something she/he shouldn’t. As the problem we’re trying to solve here is rather about fixing the errors and updating the price this most likely won’t be a problem, but he possibility would still remain.

By the way, this concern is most likely also valid for any other entity that is mutable in the OpenLMIS.

The Solutions

I would like to propose three possible solutions I see from which I have chosen one. I will describe them in more details below

Archiving

With this approach we would be archiving the orderable before updating it and creating a new entry within the database giving it a new UUID.

Pros:

  • No changes would have to be made to the entities that treat the orderable as immutable, the UUID would point to the exact orderable version they want.

Cons:

  • We would have to update entities that treat orderable as mutable on every update as they don’t really care about orderable details but about its identity (meaning they should point to the newest possible version of the orderable). FTAP is a great example here. In Malawi implementation there is around 60 FTAPs for a single orderable, which would have to be updated/recreated every time we update the orderable. If we picked up the recreation path the orderables table in the database could grow really huge really fast. We would also have to deal with entities that associate the orderable and update them too, and this could propagate.
  • This would be hard to maintain (see the previous bullet point).
  • The database constraints would be non-trivial, we would have to restrict the situation when there is a couple of orderables that have the same product code and are not archived (this wouldn’t be true for archived orderables).

Open questions:

  • How would we track the orderable identity if the UUID could change?

    Product code is the natural candidate for that.

  • How would we track the newest orderable version?

    We would probably have to introduce the archived flag, which should be set to false for the newest version of the orderable.

Versioning

With this approach we would add a new field to the orderable named version/versionId/something similar. The identity of the orderable would still be held by the UUID attribute.

Pros:

  • No changes need outside the endpoint during the initial implementation.
  • No changes needed at all to the entities that treat the orderable as mutable, the UUID would still point to the newest version.
  • All versions stored in one place (referencedata), no need for snapshots in any module - all they would need would be the UUID and version.
  • We’re not breaking backward combability, the currently implemented endpoint still returns the latest orderable version based on the UUID. To get the older version of the orderable we would have to pass an optional request parameter. This would lead to the endpoint being extended rather than modified.

Cons:

  • The need to identify and update all the places that use orderables and expect them to be immutable.

Open questions:

  • Should we use GUUID or int to track version?

    By using a simple int we could easily retrieve the latest version of the orderable from the database.

  • Where should we store the version in object referencing a specific version of the orderable?

    ReferencedataObject class is a perfect candidate for that.

  • Would this break the contract and lead to the major version being increased?

    The endpoint itself stays the same, but introduction of the versioning would probably lead to the behavior change meaning this would most likely break the contract and require a major version bump.

  • Which changes should spawn new version?

    Ideally, typo fixes should not lead to a version bump, but those things might be hard to detect. We could give the users ability to decide whether the update should spawn a new version of the orderable or update the newest one, but this feature would most likely confuse the end user.

Snapshooting

With this approach we would be snapshooting properties of the orderable which are expected to be immutable by the associating classes.

Pros:

  • Easy to implement.
  • This would break some dependencies leading to a lesser coupling of the fulfillment/requisition and referencedata modules at the cost of code duplication.
  • This would lead to a temporary performance improvement (less calls to the referencedata i.e. during requisition retrieval)…

Cons:

  • …which would become a hit to the performance in the long run (bigger database).
  • This would be a little WET as we would be duplicating some of the properties, but we could snapshot only those fields we want immutable and ignore orderable beyond the point of creation of the object that was snapshooting the orderable.

The Proposed Solution

I would like to propose Versioning as the go-to solution for this problem. It has the most pros with only one con, which is true for all the solutions - eventually, we will have to change something somewhere to make orderables editable flawlessly. This approach would also require least work outside the orderable endpoint. I’ve diagnosed only two places that expect the orderable to be immutable, which are requisition and order creation, rest of the associations rely on the identity only and don’t really care about the orderable details changing.

The Proposed Strategy

This is a proposed strategy for implementing the orderable edit feature. These steps should make the implementation as smooth as possible.

  1. Add the version property to the orderable and modify endpoint to respect it. Create a migration to set the version to 0 by default. Don’t implement orderable edit feature yet.
  2. Update the Requisition and Order creation and classes to actually use ReferencedataObject instead of an ID, the ReferencedataObject should be extended to include the version property (VersionedReferencedataObject class perhaps?). This step could also include removal of the snapshoot properties from the RequisitionLineItem, as the specific version of the orderable wouldn’t be editable (a little DRY is always good).
  3. Implement the orderable edit feature with automated version updating.
  4. Watch everything going smooth.

The Concern

Implementing edit feature on an entity that is treated both as mutable and immutable depending on the associating class will leave a gate for bugs to creep in that might be hard to deal with. My main concern is a situation in which user decides that editing a discontinued orderable and overriding it with a new one is a good idea. This means that the identity of the object has changed and I couldn’t come up with a way to prevent it other than making some of the fields immutable.

Kind regards,

Nikodem

On Friday, September 29, 2017 at 6:09:15 PM UTC+2, Nikodem Graczewski wrote:

Hi everyone,

as per our Sprint Q&A meeting I’m starting a discussion about how do we want to approach the Orderables edit as allowing this action can possibly result is some unforeseen consequences in some of the modules depending on them.

Here’s a list of modules that are somehow related to Orderables:

  • referencedata - holds the Orderables
  • requisition
  • fulfillment
  • stockmanagement
    I’ve done some research and I could only find one thing that would break after we implement Orderables edit. Requisition line item uses the ProgramOrderable to actually set the price per pack of itself. If we modify the ProgramsOrderable price per pack after the requisition initialization it won’t be reflected in the requisition line item.

Here’s a couple of ideas on how we could approach the problem:

  • We could create a new endpoint for updating the ProgramOrderables only, without touching the Orderable itself as proposed by Paweł Gesek here.

  • We could use the FHIR approach by versioning the Orderable. This approach would use two IDs, one for tracking the Orderable itself, so the ID of Orderable would never change and a second for tracking the Orderable version.

    This would allow us to:

  • Retrieve the newest version of Orderable simply by calling a GET endpoint with the ID. This would be used by the administration screens so they always display the newest Orderable. Modifying the Orderable would result in creation of a new version of it, and storing the previous one as history.

  • All the module that would require the older version of the Orderable would simply pass the version ID as a request parameter to the GET endpoint. This would result in retrieving a history version of an Orderable instead of the newest one.
    If you have any ideas, comments or concerns about this feature, please post them in this thread.

Best Regards,

Nikodem

**

SolDevelo** Sp. z o.o. [LLC] /
www.soldevelo.com

Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland

Phone:
+48 58 782 45 40 / Fax:
+48 58 782 45 41

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to
openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to
openlmis-dev@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/openlmis-dev/e423e6a1-d69c-4348-93de-fa5d8a619129%40googlegroups.com
.

For more options, visit
https://groups.google.com/d/optout
.

Sebastian Brudziński

Software Developer / Team Leader

sbrudzinski@soldevelo.com

**

SolDevelo** Sp. z o.o. [LLC] /
www.soldevelo.com

Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland

Phone:
+48 58 782 45 40 / Fax:
+48 58 782 45 41

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to
openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to
openlmis-dev@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/openlmis-dev/0fc3491d-3734-3bee-b1ac-6d852517ead3%40soldevelo.com
.

For more options, visit
https://groups.google.com/d/optout
.

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to
openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to
openlmis-dev@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/openlmis-dev/4bec1537-73d8-4333-a9ac-55f71047b6f0%40googlegroups.com
.

For more options, visit
https://groups.google.com/d/optout
.


(Chongsun Ahn) #20

Hey Łukasz,

Apologies for the late reply. I keep forgetting about this thread. You are correct that passing a list of ids AND versionIds in the URI would not work here, as we cannot be certain which ids match which versionIds.

What we probably want to do here is to support a search sub-resource under orderables to which we can POST (/api/orderables/search). That API is where we could POST a request body as a JSON object with a list of id/versionId JSON objects. This API’s would respond with the corresponding orderables.

I know we’ve discussed previously about not having search APIs, and this seems to backtrack from that position. The reason was that we were creating search APIs for simple filters, which could easily be handled by the GET resources API. However, it seems some of our APIs do require complex searches that cannot be handled through GET and request parameters only. Therefore, we should support both GET /api/resources and POST /api/resources/search when necessary. From the FHIR search spec, it seems like FHIR also supports both.

An alternative would be to create a search API that returns a “search ID” and use that ID to GET the results (/api/orderables/search/id). We would essentially be creating a new “orderable search”, with an ID. The results could be cached for a time and invalidated when necessary (during an update, or when the search is considered “expired”).

I prefer the first option, as the second option seems to be a more complex pattern. Thoughts?

Shalom,

Chongsun

– ​

There are 10 kinds of people in this world; those who understand binary, and those who don’t.

Chongsun Ahn | chongsun.ahn@villagereach.org

Software Development Engineer

Village****Reach* ** Starting at the Last Mile*

2900 Eastlake Ave. E, Suite 230, Seattle, WA 98102, USA

DIRECT: 1.206.512.1536 **CELL: **1.206.910.0973 FAX: 1.206.860.6972

SKYPE: chongsun.ahn.vr

www.villagereach.org

Connect on Facebook****, Twitter** ** and our Blog

···

Łukasz Lewczyński

Software Developer

llewczynski@soldevelo.com

**Nikodem Graczewski
**

Software Developer

ngraczewski@soldevelo.com

On Tue, Dec 12, 2017 at 9:18 AM, Łukasz Lewczyński
llewczynski@soldevelo.com wrote:

The example with paper requisition make it very clear for me because if I made a typo on paper I will had to get another paper. Thanks for this explanation.

**

SolDevelo** Sp. z o.o. [LLC] /
www.soldevelo.com

Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland

Phone: +48 58 782 45 40 / Fax: +48 58 782 45 41

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to
openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to
openlmis-dev@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/openlmis-dev/CAAdp53wdzsZdQ%2BHjAFBQgat-RPyv6CCXb1QCuc03r5GkCEtihQ%40mail.gmail.com
.

For more options, visit
https://groups.google.com/d/optout
.

Łukasz Lewczyński

Software Developer

llewczynski@soldevelo.com

On Mon, Dec 11, 2017 at 7:17 PM, Josh Zamor
josh.zamor@villagereach.org wrote:

Hi Lukasz, my thoughts:

  1. If I modified the product name (because I found a typo) should I see this change on the product grid? In my opinion yes because this change does not modify my product.

While I could see someone wanting this, there already is a simple way to get this: delete the requisition that was created and create it again. This is simple for a user to understand as it mimics what happens with paper based requisitions, i.e. it fits their mental model. Moreover doing this, instead of something fancier, keeps the separation of concerns: a typo fix to an Orderable name is the same as any other change: it creates a new version of the record. Any client which uses the Orderable now gets to decide how it wants to use the Orderable and its versions: use the latest, use a specific version, make explicit functions of it’s workflow in case the Orderable updates (e.g. while a Requisition is still in process). This is a nice separation of concerns which we should be striving for in our microservices.

  1. If I modified the product price should I see this change on the product grid? I think no because this change modifies my budget for the given period. Also if this change will be visible, the legacy data will be corrupted.

I think that’s what we’re saying with a versioned Orderable: the price wouldn’t change in the Requisition as it has a pointer to the version that was latest when it was created.

If we assume that each change creates a new version of product, how much new entries will be in database? In the worst scenario I can modify a single property, save the product, modify the next property, save it and so on.This could create a lot of rows in table.

This should be well within the capabilities of Postgres. While write operations here may be frequent, mostly we’re thinking in terms of read operations, where a UUID (primary key) and version may be put into a multi-column index. As a reference in the requisition line item table with 7+ million rows we have sub millisecond lookups in Postgres. Lets say we had 10,000 Orderables, each with 1000 changes, this would be roughly equivalent to only 10M rows in the Orderable table.

Good questions, keep them coming.

Best,

Josh

On Dec 11, 2017, at 6:22 AM, Łukasz Lewczyński llewczynski@soldevelo.com wrote:

I have small problem with versioning option. Let assume we have a productwith initial versionand create a requisition with that product and version. Now:

  1. If I modified the product name (because I found a typo) should I see this change on the product grid? In my opinion yes because this change does not modify my product.
  2. If I modified the product price should I see this change on the product grid? I think no because this change modifies my budget for the given period. Also if this change will be visible, the legacy data will be corrupted.

If we assume that each change creates a new version of product, how much new entries will be in database? In the worst scenario I can modify a single property, save the product, modify the next property, save it and so on.This could create a lot of rows in table.

**

SolDevelo** Sp. z o.o. [LLC] /
www.soldevelo.com

Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland

Phone: +48 58 782 45 40 / Fax: +48 58 782 45 41

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to
openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to
openlmis-dev@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/openlmis-dev/CAAdp53wP-4UJ4SbrOi0f0FZVs7zHm7OQPsd8QZQV4KDGOVTWSw%40mail.gmail.com
.

For more options, visit
https://groups.google.com/d/optout
.

Łukasz Lewczyński

Software Developer

llewczynski@soldevelo.com

On Fri, Dec 8, 2017 at 10:33 PM, Josh Zamor
josh.zamor@villagereach.org wrote:

I love where this thread is going and some of the issues that various folks have explored and brought to the fore. Thanks everyone for the effort.

Of course, I also think Versioning is our best option. Some quick responses:

  • I agree with Sebastian and Brandon that we shouldn’t worry (much) about immutable fields. I’ve been burned in an old system where an admin was updating user profiles to “create” new users. It was a learning experience, but mostly on the UI side of things. Once the UI directed the admin to what changes meant, that mistake was corrected.
  • I also agree with Brandon’s comments on what types of changes create new versions (always - and FHIR does this as well) and also on where the version should be stored in a referencing entity.
  • I disagree that we should use a UUID for the version, and instead recommend we use a serial integer starting at 1 (and make sure in the DB it has plenty of room to grow). Reasoning: FHIR STU 3 uses a serial integer starting at 1. The UUID is important for uniquely identifying things. The version of that uniquely identified thing is something that is bounded by ACID, and therefore should always increase in a predictable fashion. Note what this means is that when you change an entity (say in the UI), you never increment the version and send it back, that’s a function of the write operation on the server from which it was read.

A few responses to Brandon’s other points:

For GS1 Trade Item-based Orderables, what are the GS1 GDSN rules about mutability? If Tylenol ever changes its PackSize, its UPC Barcode or changes name to “Tylenol Zero Sugar” must they issue a whole new Trade Item?

Great question. The actual answer to this is
complex
. The simple explanation I got in the GTIN training was this: if you care about identifying yellow shirts, blue shirts and orange shirts in the supply chain, they each need a GTIN. If you just care about shirts, one GTIN will do. The document I linked is aimed at manufacturers / brand owners and when they should issue new GTINs. The key for us however is that we’re supposed to trust what they do. If they make a mistake, as recipients of their information our system can’t automatically decide the change is in error or not. GS1 I believe works to assist those creating GTINs do so correctly. Put simply: OpenLMIS should allow any attribute change which is defined in the GDSN and trust that the author understands the effect of that master data in the supply chain.

based Orderables, do those systems ever change the name of something (like how “Autism” was changed to “Autism Spectrum Disorder” in the DSM-5)?

For UNSPSC/GPC/Commodity-Type-

Another great question. This one I don’t know, and I suspect it might vary depending on the author of the classification system (WHO, GS1, country-specific). The flexibility to where these classification systems come from is intentional in our design, even though we’d hope a country chooses mature systems. Because of this flexibility though, we might err on the side of caution and version CommodityType. I’m not 100% sure and I’ve created a ticket to track it:
https://openlmis.atlassian.net/browse/OLMIS-3790

Finally, for now I believe we are just talking about making an individual Orderable editable. As a separate issue (later) we will also talk about how the FTAP list should be editable. But that’s a separate topic for later. (Correct me if I’m wrong.)

I think we are/could include FTAP in this discussion. It might have been how all this got started. Either way I think we know the basics:

  • FTAP should reference non-versioned Orderable, Program, and Facility Type.
  • Changes to a FTAP should be versioned.
  • Entities such as Requisition would reference the versioned Orderable and FTAP.
  • Sound correct?

Lets keep this moving. Next steps I think are:

  • Java and REST patterns created/documented for versioned Resources. We should look at FHIR STU 3 for REST and perhaps http://hapifhir.io/ for Java
  • Pattern in Java and REST for Resources which reference versioned Resources. Again same sources as above for guidance and inspiration.
  • Choose an area to start incrementally applying this. The product model is likely it, however I’d ask Sebastian’s input as an implementer here to re-iterate what he thinks we should approach first in an incremental fashion.
    Best,

Josh

On Thursday, December 7, 2017 at 7:26:32 PM UTC-8, brandon.bowersox-johnson wrote:

Of the 3 proposed approaches, Versioning is probably also the one I would also lean towards if we want to make an Orderable editable. I agree with Sebastian that implementers should be allowed to change the code and name. In the case of a typo in the code, it is fine to change, and the rest of our software references the UUID and not the code so I believe it does not break anything. Same with changing an orderable name.

The 4 open questions about Versioning still need some proposals, so I have a few responses to those:

• “Should we use GUUID or int to track version?”

GUUID sounds more robust to me and just as easy. The newer version could store a reference to the previous version.

• “Which changes should spawn new version?”

I think any changes should always spawn a new version. That is a safer and simpler solution rather than having a complex logic where certain field edits do NOT change the version number while other field edits DO change the version number. We might as well always make each edit a new version. Any big downsides to that?

• “Where should we store the version in object referencing a specific version of the orderable?”

We need to be clear on whether the other parts of OpenLMIS that reference an Orderable are also storing a reference to a specific version of the Orderable. To take one specific example, I think the Requisition object might want to store a specific version reference for each orderable. At the time of requisition initiation, it builds the Requisition using the latest version of each Orderable from the FTAP list. It also saves that version used. Then, if an Orderable changes over time (maybe code, maybe name), if I view an old requisition from 5 years ago, the on-screen view will show me the name and code from the exact version that was used back then.

I also have a few other points:

• We will need guidance or hard rules on which Orderable properties are NOT allowed to be mutable. Can anyone name an example?

• For GS1 Trade Item-based Orderables, what are the GS1 GDSN rules about mutability? If Tylenol ever changes its PackSize, its UPC Barcode or changes name to “Tylenol Zero Sugar” must they issue a whole new Trade Item?

• For UNSPSC/GPC/Commodity-Type-based Orderables, do those systems ever change the name of something (like how “Autism” was changed to “Autism Spectrum Disorder” in the DSM-5)?

• Finally, for now I believe we are just talking about making an individual Orderable editable. As a separate issue (later) we will also talk about how the FTAP list should be editable. But that’s a separate topic for later. (Correct me if I’m wrong.)

From: openlmis-dev@googlegroups.com on behalf of Sebastian Brudziński sbrudzinski@soldevelo.com

Date: Wednesday, November 29, 2017 at 10:18 AM

To: "openlmis-dev@googlegroups.com" openlmis-dev@googlegroups.com

Subject: Re: [openlmis-dev] Re: Orderable edit design

Thanks for reviving this discussion and compiling the summary, Nikodem. I think it’s important that we start making progress with the orderables edit while we are still far from the next release, since the impact of this change will be huge.

My comment on the “identity”, under “what should be editable” and “the concern” would be that it is not something that we (as a core product) need or should worry about too much. No matter how hard we try, it’s ultimately up to the implementer to ensure that the edits they are making, to any instance in the system, are not causing that instance to represent a different thing. If we wanted to enforce that on our end, we would pretty much need to disable the ability to edit most of the properties for all our entities (eg. a name, a code…).

You are proposing that we make the product code immutable. I’d personally tend towards giving more, rather than less freedom in the edits. While the code is something that generally shouldn’t change, we already had an issue with an incorrect facility code assigned in Malawi. I believe this will/should be the only case where a code of any instance is edited, but it’s good to have that option if necessary, especially that editing the code doesn’t have any impact on the system as far as I know (references are by the id). It would be great to hear what other people think though.

+1 from me for “versioning” as our strategy to edit orderables. I think it’s the cleanest solution out of the proposed ones. It can also be implemented incrementally, without affecting other services (support for versioning first, migrating services one-by-one to use the orderable version second).

Best regards,

Sebastian.

On 28.11.2017 19:07, Nikodem Graczewski wrote:

Hello everyone!

I would like to restart the orderables edit discussion. In order to do it I’ve created the following recap on the problem and proposed a solution for dealing with it.

The Problem

Making Orderables editable to allow users to fix typos and pricing adjustments.

The Pain Points

Currently we have some classes that have the associations on the orderables. We can split them into two groups that:

  • expect orderables to be immutable. These classes care about the orderable details and don’t expect the orderable to change. Here’s a list of the classes that behave this way:
  • RequisitionLineItem
  • OrderLineItem
  • don’t care if orderable has changed. All they care that the object with the specific UUID still represents the same orderable, meaning the identity of the orderable hasn’t changed. Here’s a list of class that meet this condition:
  • FacilityTypeApprovedProduct
  • VmmApplicable
  • IdentifiableByOrderableLot
  • PhysicalInventoryLineItem
  • StockCard

What should be editable?

The main concern behind this question is that allowing the users to update the orderable could lead to an orderable object actually changing identity, meaning it would represent a different orderable than it did originally. The solution for this problem would be making the product code immutable. This would enforce that the orderable can not change its identity. If the user would like to change the code she/he would most likely be doing something she/he shouldn’t. As the problem we’re trying to solve here is rather about fixing the errors and updating the price this most likely won’t be a problem, but he possibility would still remain.

By the way, this concern is most likely also valid for any other entity that is mutable in the OpenLMIS.

The Solutions

I would like to propose three possible solutions I see from which I have chosen one. I will describe them in more details below

Archiving

With this approach we would be archiving the orderable before updating it and creating a new entry within the database giving it a new UUID.

Pros:

  • No changes would have to be made to the entities that treat the orderable as immutable, the UUID would point to the exact orderable version they want.

Cons:

  • We would have to update entities that treat orderable as mutable on every update as they don’t really care about orderable details but about its identity (meaning they should point to the newest possible version of the orderable). FTAP is a great example here. In Malawi implementation there is around 60 FTAPs for a single orderable, which would have to be updated/recreated every time we update the orderable. If we picked up the recreation path the orderables table in the database could grow really huge really fast. We would also have to deal with entities that associate the orderable and update them too, and this could propagate.
  • This would be hard to maintain (see the previous bullet point).
  • The database constraints would be non-trivial, we would have to restrict the situation when there is a couple of orderables that have the same product code and are not archived (this wouldn’t be true for archived orderables).

Open questions:

  • How would we track the orderable identity if the UUID could change?

    Product code is the natural candidate for that.

  • How would we track the newest orderable version?

    We would probably have to introduce the archived flag, which should be set to false for the newest version of the orderable.

Versioning

With this approach we would add a new field to the orderable named version/versionId/something similar. The identity of the orderable would still be held by the UUID attribute.

Pros:

  • No changes need outside the endpoint during the initial implementation.
  • No changes needed at all to the entities that treat the orderable as mutable, the UUID would still point to the newest version.
  • All versions stored in one place (referencedata), no need for snapshots in any module - all they would need would be the UUID and version.
  • We’re not breaking backward combability, the currently implemented endpoint still returns the latest orderable version based on the UUID. To get the older version of the orderable we would have to pass an optional request parameter. This would lead to the endpoint being extended rather than modified.

Cons:

  • The need to identify and update all the places that use orderables and expect them to be immutable.

Open questions:

  • Should we use GUUID or int to track version?

    By using a simple int we could easily retrieve the latest version of the orderable from the database.

  • Where should we store the version in object referencing a specific version of the orderable?

    ReferencedataObject class is a perfect candidate for that.

  • Would this break the contract and lead to the major version being increased?

    The endpoint itself stays the same, but introduction of the versioning would probably lead to the behavior change meaning this would most likely break the contract and require a major version bump.

  • Which changes should spawn new version?

    Ideally, typo fixes should not lead to a version bump, but those things might be hard to detect. We could give the users ability to decide whether the update should spawn a new version of the orderable or update the newest one, but this feature would most likely confuse the end user.

Snapshooting

With this approach we would be snapshooting properties of the orderable which are expected to be immutable by the associating classes.

Pros:

  • Easy to implement.
  • This would break some dependencies leading to a lesser coupling of the fulfillment/requisition and referencedata modules at the cost of code duplication.
  • This would lead to a temporary performance improvement (less calls to the referencedata i.e. during requisition retrieval)…

Cons:

  • …which would become a hit to the performance in the long run (bigger database).
  • This would be a little WET as we would be duplicating some of the properties, but we could snapshot only those fields we want immutable and ignore orderable beyond the point of creation of the object that was snapshooting the orderable.

The Proposed Solution

I would like to propose Versioning as the go-to solution for this problem. It has the most pros with only one con, which is true for all the solutions - eventually, we will have to change something somewhere to make orderables editable flawlessly. This approach would also require least work outside the orderable endpoint. I’ve diagnosed only two places that expect the orderable to be immutable, which are requisition and order creation, rest of the associations rely on the identity only and don’t really care about the orderable details changing.

The Proposed Strategy

This is a proposed strategy for implementing the orderable edit feature. These steps should make the implementation as smooth as possible.

  1. Add the version property to the orderable and modify endpoint to respect it. Create a migration to set the version to 0 by default. Don’t implement orderable edit feature yet.
  2. Update the Requisition and Order creation and classes to actually use ReferencedataObject instead of an ID, the ReferencedataObject should be extended to include the version property (VersionedReferencedataObject class perhaps?). This step could also include removal of the snapshoot properties from the RequisitionLineItem, as the specific version of the orderable wouldn’t be editable (a little DRY is always good).
  3. Implement the orderable edit feature with automated version updating.
  4. Watch everything going smooth.

The Concern

Implementing edit feature on an entity that is treated both as mutable and immutable depending on the associating class will leave a gate for bugs to creep in that might be hard to deal with. My main concern is a situation in which user decides that editing a discontinued orderable and overriding it with a new one is a good idea. This means that the identity of the object has changed and I couldn’t come up with a way to prevent it other than making some of the fields immutable.

Kind regards,

Nikodem

On Friday, September 29, 2017 at 6:09:15 PM UTC+2, Nikodem Graczewski wrote:

Hi everyone,

as per our Sprint Q&A meeting I’m starting a discussion about how do we want to approach the Orderables edit as allowing this action can possibly result is some unforeseen consequences in some of the modules depending on them.

Here’s a list of modules that are somehow related to Orderables:

  • referencedata - holds the Orderables
  • requisition
  • fulfillment
  • stockmanagement
    I’ve done some research and I could only find one thing that would break after we implement Orderables edit. Requisition line item uses the ProgramOrderable to actually set the price per pack of itself. If we modify the ProgramsOrderable price per pack after the requisition initialization it won’t be reflected in the requisition line item.

Here’s a couple of ideas on how we could approach the problem:

  • We could create a new endpoint for updating the ProgramOrderables only, without touching the Orderable itself as proposed by Paweł Gesek here.

  • We could use the FHIR approach by versioning the Orderable. This approach would use two IDs, one for tracking the Orderable itself, so the ID of Orderable would never change and a second for tracking the Orderable version.

    This would allow us to:

  • Retrieve the newest version of Orderable simply by calling a GET endpoint with the ID. This would be used by the administration screens so they always display the newest Orderable. Modifying the Orderable would result in creation of a new version of it, and storing the previous one as history.

  • All the module that would require the older version of the Orderable would simply pass the version ID as a request parameter to the GET endpoint. This would result in retrieving a history version of an Orderable instead of the newest one.
    If you have any ideas, comments or concerns about this feature, please post them in this thread.

Best Regards,

Nikodem

**

SolDevelo** Sp. z o.o. [LLC] /
www.soldevelo.com

Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland

Phone:
+48 58 782 45 40 / Fax:
+48 58 782 45 41

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to
openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to
openlmis-dev@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/openlmis-dev/e423e6a1-d69c-4348-93de-fa5d8a619129%40googlegroups.com
.

For more options, visit
https://groups.google.com/d/optout
.

Sebastian Brudziński

Software Developer / Team Leader

sbrudzinski@soldevelo.com

**

SolDevelo** Sp. z o.o. [LLC] /
www.soldevelo.com

Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland

Phone:
+48 58 782 45 40 / Fax:
+48 58 782 45 41

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to
openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to
openlmis-dev@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/openlmis-dev/0fc3491d-3734-3bee-b1ac-6d852517ead3%40soldevelo.com
.

For more options, visit
https://groups.google.com/d/optout
.

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to
openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to
openlmis-dev@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/openlmis-dev/4bec1537-73d8-4333-a9ac-55f71047b6f0%40googlegroups.com
.

For more options, visit
https://groups.google.com/d/optout
.