Orderable edit design


(Łukasz Lewczyński) #21

Hi Chongsun,

I also prefer the first option. I remember that in some cases we allow having the search endpoint if there is a need and I think in this case we have good arguments to use it.

···

Łukasz Lewczyński
Software Developer
llewczynski@soldevelo.com

**Nikodem Graczewski
**

Software Developer

ngraczewski@soldevelo.com

On Tue, Dec 12, 2017 at 9:18 AM, Łukasz Lewczyński
llewczynski@soldevelo.com wrote:

The example with paper requisition make it very clear for me because if I made a typo on paper I will had to get another paper. Thanks for this explanation.

**

SolDevelo** Sp. z o.o. [LLC] /
www.soldevelo.com

Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland

Phone: +48 58 782 45 40 / Fax: +48 58 782 45 41

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to
openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to
openlmis-dev@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/openlmis-dev/CAAdp53wdzsZdQ%2BHjAFBQgat-RPyv6CCXb1QCuc03r5GkCEtihQ%40mail.gmail.com
.

For more options, visit
https://groups.google.com/d/optout
.

Łukasz Lewczyński

Software Developer

llewczynski@soldevelo.com

On Mon, Dec 11, 2017 at 7:17 PM, Josh Zamor
josh.zamor@villagereach.org wrote:

Hi Lukasz, my thoughts:

  1. If I modified the product name (because I found a typo) should I see this change on the product grid? In my opinion yes because this change does not modify my product.

While I could see someone wanting this, there already is a simple way to get this: delete the requisition that was created and create it again. This is simple for a user to understand as it mimics what happens with paper based requisitions, i.e. it fits their mental model. Moreover doing this, instead of something fancier, keeps the separation of concerns: a typo fix to an Orderable name is the same as any other change: it creates a new version of the record. Any client which uses the Orderable now gets to decide how it wants to use the Orderable and its versions: use the latest, use a specific version, make explicit functions of it’s workflow in case the Orderable updates (e.g. while a Requisition is still in process). This is a nice separation of concerns which we should be striving for in our microservices.

  1. If I modified the product price should I see this change on the product grid? I think no because this change modifies my budget for the given period. Also if this change will be visible, the legacy data will be corrupted.

I think that’s what we’re saying with a versioned Orderable: the price wouldn’t change in the Requisition as it has a pointer to the version that was latest when it was created.

If we assume that each change creates a new version of product, how much new entries will be in database? In the worst scenario I can modify a single property, save the product, modify the next property, save it and so on.This could create a lot of rows in table.

This should be well within the capabilities of Postgres. While write operations here may be frequent, mostly we’re thinking in terms of read operations, where a UUID (primary key) and version may be put into a multi-column index. As a reference in the requisition line item table with 7+ million rows we have sub millisecond lookups in Postgres. Lets say we had 10,000 Orderables, each with 1000 changes, this would be roughly equivalent to only 10M rows in the Orderable table.

Good questions, keep them coming.

Best,

Josh

On Dec 11, 2017, at 6:22 AM, Łukasz Lewczyński llewczynski@soldevelo.com wrote:

I have small problem with versioning option. Let assume we have a productwith initial versionand create a requisition with that product and version. Now:

  1. If I modified the product name (because I found a typo) should I see this change on the product grid? In my opinion yes because this change does not modify my product.
  2. If I modified the product price should I see this change on the product grid? I think no because this change modifies my budget for the given period. Also if this change will be visible, the legacy data will be corrupted.

If we assume that each change creates a new version of product, how much new entries will be in database? In the worst scenario I can modify a single property, save the product, modify the next property, save it and so on.This could create a lot of rows in table.

**

SolDevelo** Sp. z o.o. [LLC] /
www.soldevelo.com

Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland

Phone: +48 58 782 45 40 / Fax: +48 58 782 45 41

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to
openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to
openlmis-dev@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/openlmis-dev/CAAdp53wP-4UJ4SbrOi0f0FZVs7zHm7OQPsd8QZQV4KDGOVTWSw%40mail.gmail.com
.

For more options, visit
https://groups.google.com/d/optout
.

Łukasz Lewczyński

Software Developer

llewczynski@soldevelo.com

On Fri, Dec 8, 2017 at 10:33 PM, Josh Zamor
josh.zamor@villagereach.org wrote:

I love where this thread is going and some of the issues that various folks have explored and brought to the fore. Thanks everyone for the effort.

Of course, I also think Versioning is our best option. Some quick responses:

  • I agree with Sebastian and Brandon that we shouldn’t worry (much) about immutable fields. I’ve been burned in an old system where an admin was updating user profiles to “create” new users. It was a learning experience, but mostly on the UI side of things. Once the UI directed the admin to what changes meant, that mistake was corrected.
  • I also agree with Brandon’s comments on what types of changes create new versions (always - and FHIR does this as well) and also on where the version should be stored in a referencing entity.
  • I disagree that we should use a UUID for the version, and instead recommend we use a serial integer starting at 1 (and make sure in the DB it has plenty of room to grow). Reasoning: FHIR STU 3 uses a serial integer starting at 1. The UUID is important for uniquely identifying things. The version of that uniquely identified thing is something that is bounded by ACID, and therefore should always increase in a predictable fashion. Note what this means is that when you change an entity (say in the UI), you never increment the version and send it back, that’s a function of the write operation on the server from which it was read.

A few responses to Brandon’s other points:

For GS1 Trade Item-based Orderables, what are the GS1 GDSN rules about mutability? If Tylenol ever changes its PackSize, its UPC Barcode or changes name to “Tylenol Zero Sugar” must they issue a whole new Trade Item?

Great question. The actual answer to this is
complex
. The simple explanation I got in the GTIN training was this: if you care about identifying yellow shirts, blue shirts and orange shirts in the supply chain, they each need a GTIN. If you just care about shirts, one GTIN will do. The document I linked is aimed at manufacturers / brand owners and when they should issue new GTINs. The key for us however is that we’re supposed to trust what they do. If they make a mistake, as recipients of their information our system can’t automatically decide the change is in error or not. GS1 I believe works to assist those creating GTINs do so correctly. Put simply: OpenLMIS should allow any attribute change which is defined in the GDSN and trust that the author understands the effect of that master data in the supply chain.

based Orderables, do those systems ever change the name of something (like how “Autism” was changed to “Autism Spectrum Disorder” in the DSM-5)?

For UNSPSC/GPC/Commodity-Type-

Another great question. This one I don’t know, and I suspect it might vary depending on the author of the classification system (WHO, GS1, country-specific). The flexibility to where these classification systems come from is intentional in our design, even though we’d hope a country chooses mature systems. Because of this flexibility though, we might err on the side of caution and version CommodityType. I’m not 100% sure and I’ve created a ticket to track it:
https://openlmis.atlassian.net/browse/OLMIS-3790

Finally, for now I believe we are just talking about making an individual Orderable editable. As a separate issue (later) we will also talk about how the FTAP list should be editable. But that’s a separate topic for later. (Correct me if I’m wrong.)

I think we are/could include FTAP in this discussion. It might have been how all this got started. Either way I think we know the basics:

  • FTAP should reference non-versioned Orderable, Program, and Facility Type.
  • Changes to a FTAP should be versioned.
  • Entities such as Requisition would reference the versioned Orderable and FTAP.
  • Sound correct?

Lets keep this moving. Next steps I think are:

  • Java and REST patterns created/documented for versioned Resources. We should look at FHIR STU 3 for REST and perhaps http://hapifhir.io/ for Java
  • Pattern in Java and REST for Resources which reference versioned Resources. Again same sources as above for guidance and inspiration.
  • Choose an area to start incrementally applying this. The product model is likely it, however I’d ask Sebastian’s input as an implementer here to re-iterate what he thinks we should approach first in an incremental fashion.
    Best,

Josh

On Thursday, December 7, 2017 at 7:26:32 PM UTC-8, brandon.bowersox-johnson wrote:

Of the 3 proposed approaches, Versioning is probably also the one I would also lean towards if we want to make an Orderable editable. I agree with Sebastian that implementers should be allowed to change the code and name. In the case of a typo in the code, it is fine to change, and the rest of our software references the UUID and not the code so I believe it does not break anything. Same with changing an orderable name.

The 4 open questions about Versioning still need some proposals, so I have a few responses to those:

• “Should we use GUUID or int to track version?”

GUUID sounds more robust to me and just as easy. The newer version could store a reference to the previous version.

• “Which changes should spawn new version?”

I think any changes should always spawn a new version. That is a safer and simpler solution rather than having a complex logic where certain field edits do NOT change the version number while other field edits DO change the version number. We might as well always make each edit a new version. Any big downsides to that?

• “Where should we store the version in object referencing a specific version of the orderable?”

We need to be clear on whether the other parts of OpenLMIS that reference an Orderable are also storing a reference to a specific version of the Orderable. To take one specific example, I think the Requisition object might want to store a specific version reference for each orderable. At the time of requisition initiation, it builds the Requisition using the latest version of each Orderable from the FTAP list. It also saves that version used. Then, if an Orderable changes over time (maybe code, maybe name), if I view an old requisition from 5 years ago, the on-screen view will show me the name and code from the exact version that was used back then.

I also have a few other points:

• We will need guidance or hard rules on which Orderable properties are NOT allowed to be mutable. Can anyone name an example?

• For GS1 Trade Item-based Orderables, what are the GS1 GDSN rules about mutability? If Tylenol ever changes its PackSize, its UPC Barcode or changes name to “Tylenol Zero Sugar” must they issue a whole new Trade Item?

• For UNSPSC/GPC/Commodity-Type-based Orderables, do those systems ever change the name of something (like how “Autism” was changed to “Autism Spectrum Disorder” in the DSM-5)?

• Finally, for now I believe we are just talking about making an individual Orderable editable. As a separate issue (later) we will also talk about how the FTAP list should be editable. But that’s a separate topic for later. (Correct me if I’m wrong.)

From: openlmis-dev@googlegroups.com on behalf of Sebastian Brudziński sbrudzinski@soldevelo.com

Date: Wednesday, November 29, 2017 at 10:18 AM

To: "openlmis-dev@googlegroups.com" openlmis-dev@googlegroups.com

Subject: Re: [openlmis-dev] Re: Orderable edit design

Thanks for reviving this discussion and compiling the summary, Nikodem. I think it’s important that we start making progress with the orderables edit while we are still far from the next release, since the impact of this change will be huge.

My comment on the “identity”, under “what should be editable” and “the concern” would be that it is not something that we (as a core product) need or should worry about too much. No matter how hard we try, it’s ultimately up to the implementer to ensure that the edits they are making, to any instance in the system, are not causing that instance to represent a different thing. If we wanted to enforce that on our end, we would pretty much need to disable the ability to edit most of the properties for all our entities (eg. a name, a code…).

You are proposing that we make the product code immutable. I’d personally tend towards giving more, rather than less freedom in the edits. While the code is something that generally shouldn’t change, we already had an issue with an incorrect facility code assigned in Malawi. I believe this will/should be the only case where a code of any instance is edited, but it’s good to have that option if necessary, especially that editing the code doesn’t have any impact on the system as far as I know (references are by the id). It would be great to hear what other people think though.

+1 from me for “versioning” as our strategy to edit orderables. I think it’s the cleanest solution out of the proposed ones. It can also be implemented incrementally, without affecting other services (support for versioning first, migrating services one-by-one to use the orderable version second).

Best regards,

Sebastian.

On 28.11.2017 19:07, Nikodem Graczewski wrote:

Hello everyone!

I would like to restart the orderables edit discussion. In order to do it I’ve created the following recap on the problem and proposed a solution for dealing with it.

The Problem

Making Orderables editable to allow users to fix typos and pricing adjustments.

The Pain Points

Currently we have some classes that have the associations on the orderables. We can split them into two groups that:

  • expect orderables to be immutable. These classes care about the orderable details and don’t expect the orderable to change. Here’s a list of the classes that behave this way:
  • RequisitionLineItem
  • OrderLineItem
  • don’t care if orderable has changed. All they care that the object with the specific UUID still represents the same orderable, meaning the identity of the orderable hasn’t changed. Here’s a list of class that meet this condition:
  • FacilityTypeApprovedProduct
  • VmmApplicable
  • IdentifiableByOrderableLot
  • PhysicalInventoryLineItem
  • StockCard

What should be editable?

The main concern behind this question is that allowing the users to update the orderable could lead to an orderable object actually changing identity, meaning it would represent a different orderable than it did originally. The solution for this problem would be making the product code immutable. This would enforce that the orderable can not change its identity. If the user would like to change the code she/he would most likely be doing something she/he shouldn’t. As the problem we’re trying to solve here is rather about fixing the errors and updating the price this most likely won’t be a problem, but he possibility would still remain.

By the way, this concern is most likely also valid for any other entity that is mutable in the OpenLMIS.

The Solutions

I would like to propose three possible solutions I see from which I have chosen one. I will describe them in more details below

Archiving

With this approach we would be archiving the orderable before updating it and creating a new entry within the database giving it a new UUID.

Pros:

  • No changes would have to be made to the entities that treat the orderable as immutable, the UUID would point to the exact orderable version they want.

Cons:

  • We would have to update entities that treat orderable as mutable on every update as they don’t really care about orderable details but about its identity (meaning they should point to the newest possible version of the orderable). FTAP is a great example here. In Malawi implementation there is around 60 FTAPs for a single orderable, which would have to be updated/recreated every time we update the orderable. If we picked up the recreation path the orderables table in the database could grow really huge really fast. We would also have to deal with entities that associate the orderable and update them too, and this could propagate.
  • This would be hard to maintain (see the previous bullet point).
  • The database constraints would be non-trivial, we would have to restrict the situation when there is a couple of orderables that have the same product code and are not archived (this wouldn’t be true for archived orderables).

Open questions:

  • How would we track the orderable identity if the UUID could change?

    Product code is the natural candidate for that.

  • How would we track the newest orderable version?

    We would probably have to introduce the archived flag, which should be set to false for the newest version of the orderable.

Versioning

With this approach we would add a new field to the orderable named version/versionId/something similar. The identity of the orderable would still be held by the UUID attribute.

Pros:

  • No changes need outside the endpoint during the initial implementation.
  • No changes needed at all to the entities that treat the orderable as mutable, the UUID would still point to the newest version.
  • All versions stored in one place (referencedata), no need for snapshots in any module - all they would need would be the UUID and version.
  • We’re not breaking backward combability, the currently implemented endpoint still returns the latest orderable version based on the UUID. To get the older version of the orderable we would have to pass an optional request parameter. This would lead to the endpoint being extended rather than modified.

Cons:

  • The need to identify and update all the places that use orderables and expect them to be immutable.

Open questions:

  • Should we use GUUID or int to track version?

    By using a simple int we could easily retrieve the latest version of the orderable from the database.

  • Where should we store the version in object referencing a specific version of the orderable?

    ReferencedataObject class is a perfect candidate for that.

  • Would this break the contract and lead to the major version being increased?

    The endpoint itself stays the same, but introduction of the versioning would probably lead to the behavior change meaning this would most likely break the contract and require a major version bump.

  • Which changes should spawn new version?

    Ideally, typo fixes should not lead to a version bump, but those things might be hard to detect. We could give the users ability to decide whether the update should spawn a new version of the orderable or update the newest one, but this feature would most likely confuse the end user.

Snapshooting

With this approach we would be snapshooting properties of the orderable which are expected to be immutable by the associating classes.

Pros:

  • Easy to implement.
  • This would break some dependencies leading to a lesser coupling of the fulfillment/requisition and referencedata modules at the cost of code duplication.
  • This would lead to a temporary performance improvement (less calls to the referencedata i.e. during requisition retrieval)…

Cons:

  • …which would become a hit to the performance in the long run (bigger database).
  • This would be a little WET as we would be duplicating some of the properties, but we could snapshot only those fields we want immutable and ignore orderable beyond the point of creation of the object that was snapshooting the orderable.

The Proposed Solution

I would like to propose Versioning as the go-to solution for this problem. It has the most pros with only one con, which is true for all the solutions - eventually, we will have to change something somewhere to make orderables editable flawlessly. This approach would also require least work outside the orderable endpoint. I’ve diagnosed only two places that expect the orderable to be immutable, which are requisition and order creation, rest of the associations rely on the identity only and don’t really care about the orderable details changing.

The Proposed Strategy

This is a proposed strategy for implementing the orderable edit feature. These steps should make the implementation as smooth as possible.

  1. Add the version property to the orderable and modify endpoint to respect it. Create a migration to set the version to 0 by default. Don’t implement orderable edit feature yet.
  2. Update the Requisition and Order creation and classes to actually use ReferencedataObject instead of an ID, the ReferencedataObject should be extended to include the version property (VersionedReferencedataObject class perhaps?). This step could also include removal of the snapshoot properties from the RequisitionLineItem, as the specific version of the orderable wouldn’t be editable (a little DRY is always good).
  3. Implement the orderable edit feature with automated version updating.
  4. Watch everything going smooth.

The Concern

Implementing edit feature on an entity that is treated both as mutable and immutable depending on the associating class will leave a gate for bugs to creep in that might be hard to deal with. My main concern is a situation in which user decides that editing a discontinued orderable and overriding it with a new one is a good idea. This means that the identity of the object has changed and I couldn’t come up with a way to prevent it other than making some of the fields immutable.

Kind regards,

Nikodem

On Friday, September 29, 2017 at 6:09:15 PM UTC+2, Nikodem Graczewski wrote:

Hi everyone,

as per our Sprint Q&A meeting I’m starting a discussion about how do we want to approach the Orderables edit as allowing this action can possibly result is some unforeseen consequences in some of the modules depending on them.

Here’s a list of modules that are somehow related to Orderables:

  • referencedata - holds the Orderables
  • requisition
  • fulfillment
  • stockmanagement
    I’ve done some research and I could only find one thing that would break after we implement Orderables edit. Requisition line item uses the ProgramOrderable to actually set the price per pack of itself. If we modify the ProgramsOrderable price per pack after the requisition initialization it won’t be reflected in the requisition line item.

Here’s a couple of ideas on how we could approach the problem:

  • We could create a new endpoint for updating the ProgramOrderables only, without touching the Orderable itself as proposed by Paweł Gesek here.

  • We could use the FHIR approach by versioning the Orderable. This approach would use two IDs, one for tracking the Orderable itself, so the ID of Orderable would never change and a second for tracking the Orderable version.

    This would allow us to:

  • Retrieve the newest version of Orderable simply by calling a GET endpoint with the ID. This would be used by the administration screens so they always display the newest Orderable. Modifying the Orderable would result in creation of a new version of it, and storing the previous one as history.

  • All the module that would require the older version of the Orderable would simply pass the version ID as a request parameter to the GET endpoint. This would result in retrieving a history version of an Orderable instead of the newest one.
    If you have any ideas, comments or concerns about this feature, please post them in this thread.

Best Regards,

Nikodem

**

SolDevelo** Sp. z o.o. [LLC] /
www.soldevelo.com

Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland

Phone:
+48 58 782 45 40 / Fax:
+48 58 782 45 41

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to
openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to
openlmis-dev@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/openlmis-dev/e423e6a1-d69c-4348-93de-fa5d8a619129%40googlegroups.com
.

For more options, visit
https://groups.google.com/d/optout
.

Sebastian Brudziński

Software Developer / Team Leader

sbrudzinski@soldevelo.com

**

SolDevelo** Sp. z o.o. [LLC] /
www.soldevelo.com

Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland

Phone:
+48 58 782 45 40 / Fax:
+48 58 782 45 41

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to
openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to
openlmis-dev@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/openlmis-dev/0fc3491d-3734-3bee-b1ac-6d852517ead3%40soldevelo.com
.

For more options, visit
https://groups.google.com/d/optout
.

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to
openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to
openlmis-dev@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/openlmis-dev/4bec1537-73d8-4333-a9ac-55f71047b6f0%40googlegroups.com
.

For more options, visit
https://groups.google.com/d/optout
.