Reporting Pages to Build into OLMIS Toolbar Item

(Clay Crosby) #1

Hi team, please review the report groupings listed here and give feedback on this page either below each section or in a comment https://openlmis.atlassian.net/wiki/spaces/OP/pages/468058260/Reporting+Page+Groupings

(Wes Brown) #2

Hey Clay,

Thanks for putting this together.

I’ve added my thoughts to each page below:

General

  • I assume that all the column labels will be updated to more natural text; is that correct?
  • Why are we showing the timeliness information on each dashboard? This seems more likely to confuse the user and distract from the main purpose for each report dashboard, aside from the Reporting Rate and Timeliness dashboard.
  • In most (all?) of these reports I would want the current or maybe the last completed period to be filtered by default. Including all information is too much and makes it difficult to find whatever data should be considered important.

Reporting Rate and Timeliness

I would guess that this report would primarily be used to find Late and/or Not Reporting facilities for the current period. Right now the report shows all facilities for all periods which seems like too much information.

  • The pie chart is not very useful, especially once the data is filtered by timeliness.
  • Can you change the pie chart to use something other than timeliness when timeliness is filtered? Would this be helpful?
  • Is there an easy way to see the count of records found with a given filter?
  • Can you order the results in the filter dropdowns? The Period Name filter seems to be in a random order.

Stock Status

This seems to be mixing two different and only somewhat related reports. The timeliness of reporting applies to an overall facility (similar to the above report) while the stock status applies to individual item stock. Are these two reports likely to be needed by the same person? I ask because this Stock Status report for Malawi has this defined as the use case:

As a pharmacist I need to be able to see the stock status at each facility in the district so that I can provide supervision if/when necessary and know when to redistribute stock, and use data to inform ordering.

Is that the intended use case for this report dashboard? If so, I think that the timeliness should be moved to another dashboard; perhaps added to the above Reporting Rate and Timeliness dashboard? Regarding the stock status portion of this dashboard:

  • I like the Stock Status over Time graph, it looks great and provides helpful information
  • Given the use case, I would want to have the records that the user should care about be the default filter than expecting the user to properly set the filters.
    • The filter set for the table should not affect the Stock Status over Time. That graph should likely show results for the full dataset, and perhaps be pre-calculated so that this doesn’t have too large an impact on performance

Stockouts

I’ve got the same comment as above regarding including the timeliness information here; it doesn’t seem helpful here. It is pretty difficult to assess this dashboard with the current data. However, if this dashboard is supposed to address this report then I think that a lot of work will need to be done match the current functionality. Specifically:

  • Filtering the data appropriately so that the important data is highlighted. The eLMIS reports appear to mostly focus on the current and previous reporting periods which helps limit the data and visualizations significantly.
  • Effective use of color to highlight/differentiate information
  • Separating the elements that focus on a specific facility from those that focus on multiple facilities or districts. Perhaps trying to create a single dashboard that can do both is ending up with a dashboard that does both poorly.
  • Better use of graphs. The Facility (District) Stockout Rates Over Time graph is difficult to gauge with the current data for facilities but even when applied to districts I am not sure if this is a useful way to present this information. I don’t know what graph would be better but I think that we need to narrow the focus of this dashboard to a specific role or roles and then try to understand exactly how they would use this data.

Consumption

Same comment about timeliness info as above. Once again, I am not entirely sure how this data is supposed to be utilized so it is difficult to know how it should be presented.

  • Product name list is unsorted
  • Is data in the Logistics Summary Report section seems off. When I select the product Carbamazepine 200mg the table has multiple records for the same period but with different values. Is this showing the data for multiple facilities but not aggregating the data?
  • Is there a column missing from Consumption Report section? When I select the product Carbamazepine 200mg the table has a large number of records with different total_consumed_quantity and adjusted_consumption values but identical product_code and full_product_name values. I am not sure what this is supposed to show me but this seems wrong.

Adjustments

Same comment about timeliness info as above.

  • I assume that supplying_facility will be replaced with the facility name rather than the uuid
  • I assume that reasonid will be replaced with the reason text rather than the uuid
  • What are the bars in the SUM(quantity) and SUM(total_losses_and_adjustments) columns supposed to indicate?

Orders

Same comment about timeliness info as above.

  • In the Order & Item Fill Rate table, the data seems off. When I select the Mar2014 period there are multiple records for each product. Should this data be aggregated?
  • The Emergency v Regular Order Rate isn’t working properly. It just shows false: 100%

Admin

  • Would it be helpful to add a default sorting to Facility List table?

Thanks,
-Wes

(Clay Crosby) #3

Thanks for the feedback Wes :grinning: I’ve replied to your points in italics below:

General

  • I assume that all the column labels will be updated to more natural text; is that correct?
    ** I’ve updated that now. In some cases, to save space, instead of saying “Facility Name” I just said “Facility.” For things that display a “code” or “ID,” I left that as a part of the label. Note in filters, the filter field title appears with this formatting but the filter label within the box retains the database column label: image

  • Why are we showing the timeliness information on each dashboard? This seems more likely to confuse the user and distract from the main purpose for each report dashboard, aside from the Reporting Rate and Timeliness dashboard.
    ** I mentioned this on the wiki page I wrote, but my thinking was this is helpful to understanding the denominator in whatever data you’re looking at. The team in TZ definitely feels this way as well. Although I definitely see the other side, and perhaps it would be better to just include the pie chart on each page next to a hyperlink to the more detailed Reporting Rate and Timeliness page if they want to investigate further.

  • In most (all?) of these reports I would want the current or maybe the last completed period to be filtered by default. Including all information is too much and makes it difficult to find whatever data should be considered important.
    ** I’ve updated it to sort all tables by reporting_period_end_date in descending order by default, but because “Reporting Period” is a string and not a date, we cannot fetch the most recent one by default. I think this should work but let me know if you disagree

Reporting Rate and Timeliness

I would guess that this report would primarily be used to find Late and/or Not Reporting facilities for the current period. Right now the report shows all facilities for all periods which seems like too much information.
** This one was already sorted to show most recent period, and what you’re presented with is just data from the most recent period unless you navigate to other pages.

  • The pie chart is not very useful, especially once the data is filtered by timeliness.
    ** The pie chart was included as a carry over from eLMIS. What do you think about a stacked bar chart that shows it over time? At least this way, although you do lose seeing the aggregate, all-time rate, so maybe we use both? Note in this screenshot we’re always having 31 expected facilities although if that number changes over time (as it does to the rightmost end) then the bars won’t all be of a uniform height
    _

    _

  • Can you change the pie chart to use something other than timeliness when timeliness is filtered? Would this be helpful?
    ** Unfortunately we can’t have the chart reference a different indicator if the dashboard is filtered to that indicator. I agree though that the pie chart is not useful when filtering to only one status, although I’m not sure how to get around that with the current functionality. We could consider this as an enhancement, although technically I imagine it could be tricky to implement…

  • Is there an easy way to see the count of records found with a given filter?
    ** _Yep. I can add this as a “big number,” but note that it will be there as well _

  • Can you order the results in the filter dropdowns? The Period Name filter seems to be in a random order.
    ** Oh cool :smiley: I was able to do this using the same sorting by processing_period_end_date DESC like with other charts:

Stock Status

This seems to be mixing two different and only somewhat related reports. The timeliness of reporting applies to an overall facility (similar to the above report) while the stock status applies to individual item stock. Are these two reports likely to be needed by the same person? I ask because this Stock Status report for Malawi has this defined as the use case:

As a pharmacist I need to be able to see the stock status at each facility in the district so that I can provide supervision if/when necessary and know when to redistribute stock, and use data to inform ordering.

Is that the intended use case for this report dashboard? If so, I think that the timeliness should be moved to another dashboard; perhaps added to the above Reporting Rate and Timeliness dashboard? Regarding the stock status portion of this dashboard:

  • I like the Stock Status over Time graph, it looks great and provides helpful information
  • Given the use case, I would want to have the records that the user should care about be the default filter than expecting the user to properly set the filters.
    ** Yeah, I’m hearing the same thing here in TZ. We should set tracer products as default filters. Note that row-level filtering will take care of the facilities that aren’t pertinent to a user
    • The filter set for the table should not affect the Stock Status over Time. That graph should likely show results for the full dataset, and perhaps be pre-calculated so that this doesn’t have too large an impact on performance
      ** I’m not sure I agree. If I’m working at the national level and I want to see performance for a particular region, I would want this chart to be updated

Stockouts

I’ve got the same comment as above regarding including the timeliness information here; it doesn’t seem helpful here. It is pretty difficult to assess this dashboard with the current data. However, if this dashboard is supposed to address this report then I think that a lot of work will need to be done match the current functionality. Specifically:

  • Filtering the data appropriately so that the important data is highlighted. The eLMIS reports appear to mostly focus on the current and previous reporting periods which helps limit the data and visualizations significantly.
    ** _I think this should be addressed by my point of default sorting. I can also restrict the time period in the map to just show the most recent month. The map does need a lot of enhancements but those are not in scope for 3.5. Specifically: generating a popup that shows other indicator values aside from the one driving the map’s coloring, and displaying a legend. Also, Superset does not currently have a visualization with a bar chart + line chart
  • Effective use of color to highlight/differentiate information
    ** _We only have one district in our demo data, so we can’t see how the colors scale with a district-level shapefile. That’s a limitation of the dataset at this point. On a similar note, we can add a benchmark to timeseries charts (so the line chart) like this (the size, coloring, opacity, and dash style are configurable):

I need to research other types of annotations that Superset supports. Documentation is very sparse, so I just need some time to experiment_

  • Separating the elements that focus on a specific facility from those that focus on multiple facilities or districts. Perhaps trying to create a single dashboard that can do both is ending up with a dashboard that does both poorly.
    ** OK, as discussed above we can save custom, pre-filtered views with different groupings of commodities, so that should address this
  • Better use of graphs. The Facility (District) Stockout Rates Over Time graph is difficult to gauge with the current data for facilities but even when applied to districts I am not sure if this is a useful way to present this information. I don’t know what graph would be better but I think that we need to narrow the focus of this dashboard to a specific role or roles and then try to understand exactly how they would use this data.
    ** This was something the TZ team received quite well, with the caveat that for them the lines would be restricted to requisition groups (of which there are 3). It would then be complemented by a table that shows the specific stockout rate for each district within that requisition group. They also liked the idea of having a bar chart that shows the rolling 12-month average stockout rate among requisition groups. If you could be more specific about why you don’t think it’s a useful way to present information, that would be helpful. Again, remember that row-level filtering will make a lot of these less noisy

Consumption

Same comment about timeliness info as above. Once again, I am not entirely sure how this data is supposed to be utilized so it is difficult to know how it should be presented.

  • Product name list is unsorted
    ** I’ll need to investigate how to make each list individually sortable. Right now I have the filter boxes sorted by processing_eriod_end, and I’m not able to discern how the sorting goes after that
  • Is data in the Logistics Summary Report section seems off. When I select the product Carbamazepine 200mg the table has multiple records for the same period but with different values. Is this showing the data for multiple facilities but not aggregating the data?
    ** Correct. I added Facility as a column and now it’s less confusing. That column was originally omitted per the eLMIS version of that report
  • Is there a column missing from Consumption Report section? When I select the product Carbamazepine 200mg the table has a large number of records with different total_consumed_quantity and adjusted_consumption values but identical product_code and full_product_name values. I am not sure what this is supposed to show me but this seems wrong.
    ** Same reply as above I think_

Adjustments

Same comment about timeliness info as above.

  • I assume that supplying_facility will be replaced with the facility name rather than the uuid
    **Yes, working on other items
  • I assume that reasonid will be replaced with the reason text rather than the uuid
    **Yes, working on other items
  • What are the bars in the SUM(quantity) and SUM(total_losses_and_adjustments) columns supposed to indicate?
    **Fixed labelling

Orders

Same comment about timeliness info as above.

  • In the Order & Item Fill Rate table, the data seems off. When I select the Mar2014 period there are multiple records for each product. Should this data be aggregated?
    ** This is because the facility column was not included per the eLMIS spec. I’ve added it:
  • The Emergency v Regular Order Rate isn’t working properly. It just shows false: 100%
    ** That is an accurate representation of the data we have!
    image
    image

Admin

  • Would it be helpful to add a default sorting to Facility List table?
    ** We can only default sort by an aggregation currently. This is disappointing. However, you can sort it yourself by clicking on a column header. Alternatively we could sort by the dates available in the facilities table, MIN or MAX of golivestartdate or goliveenddate