Program Orderable synchronization using PCMT

Hello Everyone,

I’m working on COV-107 and I found some issues in our current solution which provide product information to OpenLMIS form PCMT.

Issues:

  1. Orderable.csv, uploaded via openlmis-refdata-seed, cannot tell if it has an older or newer version of the product.
  2. The current solution does not support many programs
  3. Currently, the price is not updating due to the association of the price with a program.
  4. It is required to manually synchronize CSVs related to Orderables (Orderables.csv, ProgramOrderables.csv, FTAP.csv)

Current workflow:

  1. Run integration with PCMT
  2. Upload Orderables.csv
  3. Upload ProgramOrderables.csv
  4. Upload FTAP.csv

Pros:

  • Orderables will be updated.
  • Orderables will keep the association with programs.
  • Orderables will keep the association with FTAPS.
  • FTAPs and program orderables stays and are not duplicated

Cons:

  • All CSV files have to be synced with PCMT.
  • If Oderables will be provided by PCMT, but an admin will skip the seed tool part, orderables will be useless due to missing program and facility associations.

Solution #1

Workflow:

  1. Run integration with PCMT
  2. Upload ProgramOrderables.csv
  3. Upload FTAP.csv

ToDo:

  • The seed tool, responsible for uploading CSV config, has to be extended with fetching orderables and updating them with program orderables.
  • PCMT Integration will get orderable with program orderables, find COVID one, and update the prize. In this scenario, PCMT has to be run twice (after and before CSV data upload).

Assumptions:

  • ProgramOrderables.csv and Upload FTAP.csv still have to be synced with PCMT manually.
  • PCMT integration will support the COVID program and supporting more programs will require code changes.

Resolves: 1. and 3. issues.

Solution #2

Workflow:

  1. Run integration with PCMT
    1.1. The integration will not affect program orderables not related to COVID.
    1.2. The integration will update COVID program orderable or fetch COVID program and create a new COVID program orderable for that orderable.
    1.3. PCMT integration will create a connection between every facility type and program (FTAP) for every orderable.
    1.4. FTAPs and program orderables can be removed manually by OLMIS admin (default FTAPs and program orderables will not be created again)

ToDo:

  • PCMT integration service will have to implement steps 1.1. - 1.4.

Assumptions:

  • PCMT integration will support the COVID program and supporting more programs will require code changes.

Resolves: 1., 3., and 4. issues.

Solution #3

Workflow:

  1. All data, including programs and FTAPs, are provided via PCMT
  2. Run integration with PCMT

ToDo:

  • Change the structure of PCMT data
  • Extend PCMT integration to support new fields

Assumptions:

  • PCMT integration will support as many programs as needed without code change at PCMT integration.
  • We are able to extend product data at PCMT.

Resolves: 1., 2., 3., and 4. issues.

CSV Data collection – CSV files provided by the client/users.

So we would need to add an array of objects to every PCMT product. Every object would consist of: program code, category code, and price.
[ { "programCode": "COVID", "categoryCode": "Vaccines", "priceReference": 0.35 }, { "programCode": "HIV", "categoryCode": "Category #1", "priceReference": 2.00 } ]
@joshzamor can we extend product data at PCMT as above?

In my opinion 3. solution would be most beneficial, but I need to confirm that we can extend PCMT data.
Please, let me know which solution would be best.

Best,
Wojciech Buława

Thanks @wbulawa,

You’re right in that the PCMT Integration doesn’t support programs nor OpenLMIS prices. So far that’s been by design.

If we stick with the design that Programs are not owned by PCMT (i.e. a country’s National Product Catalog), then solution #1 looks good:

However I don’t see why that workflow leads to

Doesn’t the seed tool already fetch the Orderable before adding the ProgramOrderable? Why would the integration need to be run twice? Why would it only be limited to the COVID program’s products?

The workflow when using spreadsheets/CSVs looks right, just not with those limitations. Don’t we have a way to decouple the upload of ProgramOrderable and FTAPs from the PCMT Integration schedule using the product’s code?

Best,
Josh

Thanks @joshzamor,

OpenLMIS API allows us to update Program Orderable only by including it in Orderable JSON. The seed tool combines Orderables.csv and ProgramOrderables.csv to create such JSON and update the Orderable. Adding fetching Orderable by the seed tool would eliminate the need to maintain Orderable.csv.

A product, to be available to the user, has to be connected to the program. Since PCMT does not own such information, we have to upload them after the PCMT integration process.
That’s because the product’s price is stored as a part of program information.
price_as_a_part_of_program
So the price won’t be synchronized with OLMIS for the first time. We have to run the PCMT integration process once again after uploading CSVs.

I guess we could assume that the product is associated with all programs or only with COVID one. The category would be updated via CSV. That would eliminate the second PCMT integration mentioned above, but I think that is not the right solution.

Workflow:

  1. Run integration with PCMT
  2. Upload ProgramOrderables.csv
  3. Upload FTAP.csv
  4. Run integration with PCMT to upload prices.

Assumptions:

  • ProgramOrderables.csv and Upload FTAP.csv still have to be synced with PCMT manually.
  • PCMT integration will support any number of programs but will require the second sync. The third sync will not require resync. Additional sync is needed once as a step 4. in the workflow.

ToDo:

  1. Fix bug with updating the prize and notify the team that the second PCMT integration run has to be conducted.
  2. The seed tool, responsible for uploading CSV config, has to be extended with fetching orderables and updating them with program orderables.

Justification of 2. task:
Currently, the client or user sends us the CSV Data collection. Such data is used to update the PCMT catalog by one team and CSVs by the dev team. It is common to lose the update and desynchronize Orderables.csv and PCMT catalog. If the seed tool will be able to update Program Orderables without Orderables.csv, then there is no desynchronization. PCMT team is responsible for Orderables and OLMIS team for Program Orderables.

I will handle task 1. in COV-107. For the latter, I will create a ticket since it is not a blocker (we can update Orderable.csv). It could be handled as a process improvement of updating the data for COVID instances.

@joshzamor
Also, can I assume that the price of a product is the same for every OpenLMIS program?

Ah, I see. Another way around this would be to support PATCH requests in the API, yeah? I don’t think OpenLMIS has a convention yet for using PATCH requests, however I think the only purpose so far to leaving that out was to keep it simpler.

I see, my presumption here was that PCMT’s reference price wasn’t going to be used to set the program-oriented price in OpenLMIS. I think unless we reverse course on PCMT’s agnostic approach to Programs, then we should ignore the reference price in PCMT.

Do you mean it’s common for the PCMT data to be overwritten by a subsequent use of the seed tool? I’m not sure what you meant here.

Best,
Josh

So I guess the Product price is not synchronized with PCMT is obsolete because the price should not be uploaded via PCMT at the first place.

Yes. I think that it can be the root of Duplicate products because CSV updates using product code and we changed these codes e.g. by removing the row in a spreadsheet (codes can be generated based on =ROW() number).
I’ve created a ticket to handle that Extend the seed tool to update program orderables without Orderables.csv. For now we have to ba aware that, despite unique UUID of every product, we should always update Orderables.csv and do not reuse product codes.

Right, so far the reference price in PCMT is a reference for the product, likely for procuring it, however we’re not confident that this is a price or cost that should be presented to a public health facility trying to request it through their public health supply chain. tl;dr ignore the price in PCMT.

That’s great. I can’t help but wonder though if supporting a PATCH request might be a similar amount of work though and possibly be more flexible. What do you think?

Thanks, I’ll add it as a suggestion to the Extend the seed tool to update program orderables without Orderables.csv ticket.