Demo Data Loading Approach

Hello everyone,

  I wanted to follow up on the demo data loading approach that was brought up during last technical committee call, given that one of the proposed solutions () is to re-use the reference data seeding tool that the Malawi team has contributed to core.

  I feel that one of the biggest advantages that has been missed in the research doc is the ability to create references between instances, based on the *chosen* property of the referenced entity. What's more, since the refdata seed tool uses APIs, it is possible to create references, that are based on the object properties, even between the services. Referencing by specific fields is possible due to mapping file and is documented in the refdata seed tool readme:

  All of this could lead to a more human readable demo data and way easier creating/editing of the demo data objects. Instead of UUIDs in every possible relation, any other, unique (within test data) property of an entity could be used (like code, name, displayOrder). A very short example of how it would look for the GeographicZones is also present in the README (see references to geographicLevel and parent, by code).

  On the other hand, one additional drawback that might have been missed is that it may be problematic to create demo data for entities that don't expose strictly RESTful APIs - like requisitions. The API to create (initiate) a requisition sets most of the requisition properties by itself. Also, the validation in the update endpoint prevents specific fields from being changed, based on the requisition status. Creating demo data object for requisition, and other entities without RESTful APIs, would therefore potentially require several API calls. On the good side though, this would prevent the creation of demo data that is in an illegal state (and this was a case in the past already).

  I'll be happy to answer any questions you may have about the ref data seed tool in general or in the context of demo data loading.

Best regards,

  Sebastian.
···

https://openlmis.atlassian.net/wiki/spaces/OP/pages/124387388/Demo+Data+Loading+Approachhttps://github.com/OpenLMIS/openlmis-refdata-seed/blob/master/README.md

      Sebastian Brudziński

    Software Developer / Team Leader

sbrudzinski@soldevelo.com

SolDevelo
Sp. z o.o. [LLC] / www.soldevelo.com
Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland
Phone: +48 58 782 45 40 / Fax: +48 58 782 45 41

Nobody has responded to Sebastian’s additional feedback. Should we perhaps revisit this at our next Tech Committee meeting on the 28th?

Also, based on discussion in a Fisheye review (https://review.openlmis.org/cru/FEOLMIS-2152)) I also suggest we revisit the topic about how to handle multiple errors in an API JSON error response. When we started discussing the details, additional considerations were raised.

-Brandon

···

From: openlmis-dev@googlegroups.com on behalf of Sebastian Brudziński sbrudzinski@soldevelo.com

Date: Friday, November 17, 2017 at 6:17 AM

To:openlmis-dev@googlegroups.comopenlmis-dev@googlegroups.com

Subject: [openlmis-dev] Demo Data Loading Approach

Hello everyone,

I wanted to follow up on the demo data loading approach that was brought up during last technical committee call, given that one of the proposed solutions (https://openlmis.atlassian.net/wiki/spaces/OP/pages/124387388/Demo+Data+Loading+Approach ) is to re-use the reference data seeding tool that the Malawi team has contributed to core.

I feel that one of the biggest advantages that has been missed in the research doc is the ability to create references between instances, based on the chosen property of the referenced entity. What’s more, since the refdata seed tool uses APIs, it is possible to create references, that are based on the object properties, even between the services. Referencing by specific fields is possible due to mapping file and is documented in the refdata seed tool readme:
https://github.com/OpenLMIS/openlmis-refdata-seed/blob/master/README.md

All of this could lead to a more human readable demo data and way easier creating/editing of the demo data objects. Instead of UUIDs in every possible relation, any other, unique (within test data) property of an entity could be used (like code, name, displayOrder). A very short example of how it would look for the GeographicZones is also present in the README (see references to geographicLevel and parent, by code).

On the other hand, one additional drawback that might have been missed is that it may be problematic to create demo data for entities that don’t expose strictly RESTful APIs - like requisitions. The API to create (initiate) a requisition sets most of the requisition properties by itself. Also, the validation in the update endpoint prevents specific fields from being changed, based on the requisition status. Creating demo data object for requisition, and other entities without RESTful APIs, would therefore potentially require several API calls. On the good side though, this would prevent the creation of demo data that is in an illegal state (and this was a case in the past already).

I’ll be happy to answer any questions you may have about the ref data seed tool in general or in the context of demo data loading.

Best regards,

Sebastian.

Sebastian Brudziński

Software Developer / Team Leader

sbrudzinski@soldevelo.com

**

SolDevelo** Sp. z o.o. [LLC] /
www.soldevelo.com

Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland

Phone: +48 58 782 45 40 / Fax: +48 58 782 45 41

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to
openlmis-dev@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/openlmis-dev/989dd2c8-f46b-8106-e6cf-6047b25141a2%40soldevelo.com
.

For more options, visit https://groups.google.com/d/optout.

I would say that tying instances together by something like codes or names, not UUIDs, would make demo data much more readable and easier to create. Do we have an idea of how much effort is required to migrate our existing demo data to this approach?

Regards,

Paweł


SolDevelo
Sp. z o.o. [LLC] / www.soldevelo.com
Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland
Phone: +48 58 782 45 40 / Fax: +48 58 782 45 41

···

On Wed, Nov 22, 2017 at 1:12 AM, Brandon Bowersox-Johnson brandon.bowersox-johnson@villagereach.org wrote:

Nobody has responded to Sebastian’s additional feedback. Should we perhaps revisit this at our next Tech Committee meeting on the 28th?

Also, based on discussion in a Fisheye review (https://review.openlmis.org/cru/FEOLMIS-2152)) I also suggest we revisit the topic about how to handle multiple errors in an API JSON error response. When we started discussing the details, additional considerations were raised.

-Brandon

From: openlmis-dev@googlegroups.com on behalf of Sebastian Brudziński sbrudzinski@soldevelo.com

Date: Friday, November 17, 2017 at 6:17 AM

To:openlmis-dev@googlegroups.comopenlmis-dev@googlegroups.com

Subject: [openlmis-dev] Demo Data Loading Approach

Hello everyone,

I wanted to follow up on the demo data loading approach that was brought up during last technical committee call, given that one of the proposed solutions (https://openlmis.atlassian.net/wiki/spaces/OP/pages/124387388/Demo+Data+Loading+Approach ) is to re-use the reference data seeding tool that the Malawi team has contributed to core.

I feel that one of the biggest advantages that has been missed in the research doc is the ability to create references between instances, based on the chosen property of the referenced entity. What’s more, since the refdata seed tool uses APIs, it is possible to create references, that are based on the object properties, even between the services. Referencing by specific fields is possible due to mapping file and is documented in the refdata seed tool readme:
https://github.com/OpenLMIS/openlmis-refdata-seed/blob/master/README.md

All of this could lead to a more human readable demo data and way easier creating/editing of the demo data objects. Instead of UUIDs in every possible relation, any other, unique (within test data) property of an entity could be used (like code, name, displayOrder). A very short example of how it would look for the GeographicZones is also present in the README (see references to geographicLevel and parent, by code).

On the other hand, one additional drawback that might have been missed is that it may be problematic to create demo data for entities that don’t expose strictly RESTful APIs - like requisitions. The API to create (initiate) a requisition sets most of the requisition properties by itself. Also, the validation in the update endpoint prevents specific fields from being changed, based on the requisition status. Creating demo data object for requisition, and other entities without RESTful APIs, would therefore potentially require several API calls. On the good side though, this would prevent the creation of demo data that is in an illegal state (and this was a case in the past already).

I’ll be happy to answer any questions you may have about the ref data seed tool in general or in the context of demo data loading.

Best regards,

Sebastian.

Sebastian Brudziński

Software Developer / Team Leader

sbrudzinski@soldevelo.com

**

SolDevelo** Sp. z o.o. [LLC] /
www.soldevelo.com

Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland

Phone: +48 58 782 45 40 / Fax: +48 58 782 45 41

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to
openlmis-dev@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/openlmis-dev/989dd2c8-f46b-8106-e6cf-6047b25141a2%40soldevelo.com
.

For more options, visit https://groups.google.com/d/optout.

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to openlmis-dev@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openlmis-dev/9175FF75-0D3C-48C3-9B89-1C59FE7F72E5%40villagereach.org.

For more options, visit https://groups.google.com/d/optout.

Paweł Gesek

    Technical Project Manager

     pgesek@soldevelo.com / +48 690 020 875

But how to connect resources that does not have code field for example how to connect requisition with line items? Also how to create requisition in the past?

  • Lukasz


SolDevelo
Sp. z o.o. [LLC] / www.soldevelo.com
Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland
Phone: +48 58 782 45 40 / Fax: +48 58 782 45 41

···

On Wed, Nov 22, 2017 at 10:29 AM, Paweł Gesek pgesek@soldevelo.com wrote:

I would say that tying instances together by something like codes or names, not UUIDs, would make demo data much more readable and easier to create. Do we have an idea of how much effort is required to migrate our existing demo data to this approach?

Regards,

Paweł

**
SolDevelo** Sp. z o.o. [LLC] / www.soldevelo.com
Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland
Phone: +48 58 782 45 40 / Fax: +48 58 782 45 41

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to openlmis-dev@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openlmis-dev/CADt-Nu1oDMFSkUftkhdEC8d7%2BXySrfpafnN_HapAH63QeRdKXw%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

Łukasz Lewczyński
Software Developer
llewczynski@soldevelo.com

On Wed, Nov 22, 2017 at 1:12 AM, Brandon Bowersox-Johnson brandon.bowersox-johnson@villagereach.org wrote:

Nobody has responded to Sebastian’s additional feedback. Should we perhaps revisit this at our next Tech Committee meeting on the 28th?

Also, based on discussion in a Fisheye review (https://review.openlmis.org/cru/FEOLMIS-2152)) I also suggest we revisit the topic about how to handle multiple errors in an API JSON error response. When we started discussing the details, additional considerations were raised.

-Brandon

From: openlmis-dev@googlegroups.com on behalf of Sebastian Brudziński sbrudzinski@soldevelo.com

Date: Friday, November 17, 2017 at 6:17 AM

To:openlmis-dev@googlegroups.comopenlmis-dev@googlegroups.com

Subject: [openlmis-dev] Demo Data Loading Approach

Hello everyone,

I wanted to follow up on the demo data loading approach that was brought up during last technical committee call, given that one of the proposed solutions (https://openlmis.atlassian.net/wiki/spaces/OP/pages/124387388/Demo+Data+Loading+Approach ) is to re-use the reference data seeding tool that the Malawi team has contributed to core.

I feel that one of the biggest advantages that has been missed in the research doc is the ability to create references between instances, based on the chosen property of the referenced entity. What’s more, since the refdata seed tool uses APIs, it is possible to create references, that are based on the object properties, even between the services. Referencing by specific fields is possible due to mapping file and is documented in the refdata seed tool readme:
https://github.com/OpenLMIS/openlmis-refdata-seed/blob/master/README.md

All of this could lead to a more human readable demo data and way easier creating/editing of the demo data objects. Instead of UUIDs in every possible relation, any other, unique (within test data) property of an entity could be used (like code, name, displayOrder). A very short example of how it would look for the GeographicZones is also present in the README (see references to geographicLevel and parent, by code).

On the other hand, one additional drawback that might have been missed is that it may be problematic to create demo data for entities that don’t expose strictly RESTful APIs - like requisitions. The API to create (initiate) a requisition sets most of the requisition properties by itself. Also, the validation in the update endpoint prevents specific fields from being changed, based on the requisition status. Creating demo data object for requisition, and other entities without RESTful APIs, would therefore potentially require several API calls. On the good side though, this would prevent the creation of demo data that is in an illegal state (and this was a case in the past already).

I’ll be happy to answer any questions you may have about the ref data seed tool in general or in the context of demo data loading.

Best regards,

Sebastian.

Sebastian Brudziński

Software Developer / Team Leader

sbrudzinski@soldevelo.com

**

SolDevelo** Sp. z o.o. [LLC] /
www.soldevelo.com

Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland

Phone: +48 58 782 45 40 / Fax: +48 58 782 45 41

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to
openlmis-dev@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/openlmis-dev/989dd2c8-f46b-8106-e6cf-6047b25141a2%40soldevelo.com
.

For more options, visit https://groups.google.com/d/optout.

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to openlmis-dev@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openlmis-dev/9175FF75-0D3C-48C3-9B89-1C59FE7F72E5%40villagereach.org.

For more options, visit https://groups.google.com/d/optout.


Paweł Gesek

    Technical Project Manager

     pgesek@soldevelo.com / +48 690 020 875

Can’t we just fallback to IDs in such cases? Ideally it could use composition - in this case it would be facility code, period name and program code. However for the first take on it, I’m fine with just using ids in this case.

Regards,

Paweł


SolDevelo
Sp. z o.o. [LLC] / www.soldevelo.com
Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland
Phone: +48 58 782 45 40 / Fax: +48 58 782 45 41

···

On Wed, Nov 22, 2017 at 10:32 AM, Łukasz Lewczyński llewczynski@soldevelo.com wrote:

But how to connect resources that does not have code field for example how to connect requisition with line items? Also how to create requisition in the past?

  • Lukasz


SolDevelo
Sp. z o.o. [LLC] / www.soldevelo.com
Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland
Phone: +48 58 782 45 40 / Fax: +48 58 782 45 41

Łukasz Lewczyński
Software Developer
llewczynski@soldevelo.com

On Wed, Nov 22, 2017 at 10:29 AM, Paweł Gesek pgesek@soldevelo.com wrote:

I would say that tying instances together by something like codes or names, not UUIDs, would make demo data much more readable and easier to create. Do we have an idea of how much effort is required to migrate our existing demo data to this approach?

Regards,

Paweł

**
SolDevelo** Sp. z o.o. [LLC] / www.soldevelo.com
Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland
Phone: +48 58 782 45 40 / Fax: +48 58 782 45 41

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to openlmis-dev@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openlmis-dev/CADt-Nu1oDMFSkUftkhdEC8d7%2BXySrfpafnN_HapAH63QeRdKXw%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

On Wed, Nov 22, 2017 at 1:12 AM, Brandon Bowersox-Johnson brandon.bowersox-johnson@villagereach.org wrote:

Nobody has responded to Sebastian’s additional feedback. Should we perhaps revisit this at our next Tech Committee meeting on the 28th?

Also, based on discussion in a Fisheye review (https://review.openlmis.org/cru/FEOLMIS-2152)) I also suggest we revisit the topic about how to handle multiple errors in an API JSON error response. When we started discussing the details, additional considerations were raised.

-Brandon

From: openlmis-dev@googlegroups.com on behalf of Sebastian Brudziński sbrudzinski@soldevelo.com

Date: Friday, November 17, 2017 at 6:17 AM

To:openlmis-dev@googlegroups.comopenlmis-dev@googlegroups.com

Subject: [openlmis-dev] Demo Data Loading Approach

Hello everyone,

I wanted to follow up on the demo data loading approach that was brought up during last technical committee call, given that one of the proposed solutions (https://openlmis.atlassian.net/wiki/spaces/OP/pages/124387388/Demo+Data+Loading+Approach ) is to re-use the reference data seeding tool that the Malawi team has contributed to core.

I feel that one of the biggest advantages that has been missed in the research doc is the ability to create references between instances, based on the chosen property of the referenced entity. What’s more, since the refdata seed tool uses APIs, it is possible to create references, that are based on the object properties, even between the services. Referencing by specific fields is possible due to mapping file and is documented in the refdata seed tool readme:
https://github.com/OpenLMIS/openlmis-refdata-seed/blob/master/README.md

All of this could lead to a more human readable demo data and way easier creating/editing of the demo data objects. Instead of UUIDs in every possible relation, any other, unique (within test data) property of an entity could be used (like code, name, displayOrder). A very short example of how it would look for the GeographicZones is also present in the README (see references to geographicLevel and parent, by code).

On the other hand, one additional drawback that might have been missed is that it may be problematic to create demo data for entities that don’t expose strictly RESTful APIs - like requisitions. The API to create (initiate) a requisition sets most of the requisition properties by itself. Also, the validation in the update endpoint prevents specific fields from being changed, based on the requisition status. Creating demo data object for requisition, and other entities without RESTful APIs, would therefore potentially require several API calls. On the good side though, this would prevent the creation of demo data that is in an illegal state (and this was a case in the past already).

I’ll be happy to answer any questions you may have about the ref data seed tool in general or in the context of demo data loading.

Best regards,

Sebastian.

Sebastian Brudziński

Software Developer / Team Leader

sbrudzinski@soldevelo.com

**

SolDevelo** Sp. z o.o. [LLC] /
www.soldevelo.com

Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland

Phone: +48 58 782 45 40 / Fax: +48 58 782 45 41

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to
openlmis-dev@googlegroups.com.

To view this discussion on the web visit
https://groups.google.com/d/msgid/openlmis-dev/989dd2c8-f46b-8106-e6cf-6047b25141a2%40soldevelo.com
.

For more options, visit https://groups.google.com/d/optout.

You received this message because you are subscribed to the Google Groups “OpenLMIS Dev” group.

To unsubscribe from this group and stop receiving emails from it, send an email to openlmis-dev+unsubscribe@googlegroups.com.

To post to this group, send email to openlmis-dev@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openlmis-dev/9175FF75-0D3C-48C3-9B89-1C59FE7F72E5%40villagereach.org.

For more options, visit https://groups.google.com/d/optout.


Paweł Gesek

    Technical Project Manager

     pgesek@soldevelo.com / +48 690 020 875

Paweł Gesek

    Technical Project Manager

     pgesek@soldevelo.com / +48 690 020 875