Reporting Stack improvment - create Superset client automatically

Adding the Superset client automatically to the auth.oauth_client_details table is one of the improvements we can make for subsequent implementations of the Reporting Stack instance. There are probably a few ways to do this. In this post I would like to present some suggestions and I hope you will help me choose the most optimal option.

Suggestion 1.
Adding a script with the appropriate insert into the auth.oauth_client_details table that will be run during redeploy Reporting Stack. The advantage of this solution is that the script will only be called if the specific implementation actually has a reporting stack instance. Unfortunately, connecting to the open_lmis database from the Reporting instance is not possible at this time. This solution requires appropriate configuration on AWS to enable connection to the open_lmis database from the Reporting Stack instance. In addition, the person who would create a Reporting Stack for subsequent implementations would have to remember to add analogous AWS configurations.

Suggestion 2.
An analogous script that adds an appropriate entry to the auth.oauth_client_details table could be placed in the shared directory and could be called during redeploy an OLMIS instance. In this case, there would be no problem connecting to the database. Unfortunately, we do not know if each subsequent implementation will have a Reporting Stack instance - this means that the person implementing the Reporting Stack will have to remember to add the appropriate entry calling this script.

Suggestion 3.
Create docker image for running initial scripts for Reporting Stack. An entry for the new docker image would be placed in docker-compose.yml file for OLMIS deployment. The advantage of this option is that scripts called for Reporting Stack will be stored in a separate place. There will be no problem with connecting to the database but during implementation you will have to remember to add the appropriate entry in the docker-compose.yml file.

What do you think about the above options? Do you have any objections to the proposed ideas?

Best regards,
Aleksandra

2 Likes

Thanks for this @Aleksandra_Soltys

What about option 4: allow dynamic client registration for superset as a client? It looks like we’ve started down a path for dynamically registering resources by having those resources (services mostly) register themselves in Consul. We could extend that for client registration, or do something via environment variable, or via a protected API call of Auth service. Key point is that Auth should have some simple way to add clients dynamically. Have we considered something like that yet?

Thank you for your reply @joshzamor

I have a few doubts about this solution. Are we able to register resources in Consul while we’re in the reporting stack instance (I mean outside the OLMIS docker network) ? If yes, will we be able to make a direct query to the database after the connection? To INSERT or UPDATE specific record in auth.oauth_client_details table. Unfortunately we can’t do it via the API because there is no support for adding new or updating data in the oauth_client_details table.

That’s true @Aleksandra_Soltys, to use Consul you’d have to secure it before opening it up. Consul wasn’t the only option here though if we don’t think that’d LOE would fit our current timeline: how about registration through environment variable? Our solution here can be more of a hack as we rarely have had this need so far - which is why we haven’t built out the best solution yet either.

For reference and as an aside, at some-point we should evaluate if it continues to make sense to invest in our Auth service, or if we could adopt a batteries included one such as ORY Hydra or Keycloak.

Hi @joshzamor
Unfortunately I don’t quite understand what you mean by registration through environment variable. Could you write something more about it?

Hi @Aleksandra_Soltys,

All I mean is that the credentials are passed to the Auth service through the environment (e.g. settings.env) and then the Auth service is responsible for registering superset as a client using those credentials. It’s not perfect, but it should work as a temporary work-around. More ideally, Auth would allow clients to register themselves (properly authenticated of course).

Best,
Josh

Hi @joshzamor

OK, I understand, it’s actually a better solution than the ones I proposed. In this case, the person who will implement the Reporting Stack will only need to properly define the variables in the env file. Thank you for this idea.

Best,
Aleksandra

Great, glad I could help.