feat: Add CDA Environment Manager

[msweier]

This cross-platform feature would add a CDA environment manager to cwms-cli that would:

1. Help users build and store .env files in a standard format and that provides an environment name. Use the environment variables batch and cwms-cli expects (i.e. CDA_API_ROOT and CDA_API_KEY)
2. Install shell script helpers to provide a short hand for loading the environment variables into the current terminal (e.g. cwms-env cwbi-dev).
3. Allow cwms-cli to show current environment.
4. Allows for future development in cwms-cli to have --source-env and --target-env arguments for commands like cwms-cli load

See https://github.com/HydrologicEngineeringCenter/cwms-cli/blob/feature/env-man/docs/ENV_MANAGER.md

[Enovotny]

@msweier just a note. the pull request names should be feat, fix or feat!. not PATCH, MINOR, MAJOR.

[krowvin]

First round of comments, just visual inspection. Going to take another pass on it if I can pull it down and run it myself.

[krowvin]

Followup comments

cwms-cli login is also storing the tokens in plaintext on disk.

We may turn this into a Credentials Manager conversation and not just an env conversation.

If we can figure out the way we want to properly store credentials we can use that for both env api keys and for the tokens returned by keycloak.

I suggest we look into keyring it's cross platform and it will make it trivial to store credentials.

may want to just go ahead and ship this with the regular deps and not as a per method credential (i.e. only needed if you do cwms-cli login/env?)

We will want to consider what this means for headless apps too where the secrets are expected to already be in an env and NEVER user defined.

Would need to decide if we are trying to connect anything else to these env files.

We could go to using the credentials manger lib for sensitive things like token and API key but use the host env for regular data.

We talked about how to edit the environment for Solaris, I think we could append an environment file to the users .profile and then do a source after we do that for them.

[msweier]

I implemented most of the changes and it is a much better UX, but I didn't touch the token management. Also to be able to use a loaded env with VSCode is pretty difficult because it can't inherit the properties from the spawned terminal. But it does make the cli commands easier if you need to do something. If we upgraded the load commands to take --target-env and source-env that would help to.

My main question is this still a useful feature if you still need an env file use with VSCode and debugging? Maybe one of you know a better way to integrate it.

[krowvin]

Spoke with @msweier

We agreed we can come back to

token management

For now we can store the API key and ROOT in the Windows environment. We are doing this now even if that is plaintext. It is per user environment and requires the enduser to put the key it.

is this still a useful feature if you still need an env file use with VSCode and debugging?

I think so, because it gives you the ability to activate without having to know any of the critical URLs but you can also opt into those.

It has been a pain point of setting up the envs on windows for endusers and this allows you to do it all without leaving the terminal.

Tag i'm it

Spoke with Mitch about me taking over and trying to figure out how best to store and talk with the windows registry/env to store these env values.
Use the context manager for winreg -> https://docs.python.org/3/library/winreg.html#winreg.PyHKEY.__enter__

Take out the additional terminal spawing if possible to avoid terminal nesting. Will need to refreshenv

Mitch has a branch that refreshes, can go to him if I can't figure it out

[DanielTOsborne]

Not to be a downer, but what's the point of this?
Managing environment variables aren't difficult. It's kind of the bare minimum skill required to work with this system.

I'm not opposed to simplifying processes where it makes sense, but this doesn't make sense to me.

Instead of taking something targeted to a 12th grader and making it understandable by a 5th grader, you're taking something targeted to a 5th grader and making it for a 4th grader.
It's not that much of a simplification, and in fact can be detrimental because you're obfuscating such a core component that it makes troubleshooting more difficult, particularly if cwms-cli decided to stop working.

EDIT:
What might be useful, however, is some testing functions. Have some way of triggering various fetch/update tests with the current environment.

[msweier]

Not to be a downer, but what's the point of this?
Managing environment variables aren't difficult. It's kind of the bare minimum skill required to work with this system.

That's a fair point and something I was wondering myself. Here's a few reasons why I think it may be useful:

1. There are some admins who are part time admin and part time do regulation/modeling. They don't have a much of CS background. Or a backup sys admin. A lot of districts want multiple people in house that know how to do this so they can operate if the primary is on vacation or leaves. This could help prevent posting an .env to a repo or a LLM.
2. Once we can shorthand environments, that opens some utility in having arguments like --source-env and --target-env. Easy to move stuff to dev or local host.
3. The .env's add up. If you have few repos you can have redundant .envs depending on how you set it up.

[DanielTOsborne]

1. There are some admins who are part time admin and part time do regulation/modeling. They don't have a much of CS background. Or a backup sys admin. A lot of districts want multiple people in house that know how to do this so they can operate if the primary is on vacation or leaves.

What happens if there's an OS update and python breaks, and cwms-cli no longer works? It's happened enough on my system I've had to change how I operate. Someone who doesn't do this regularly will really be in a bad position. Though I guess it may not matter, since what would they be doing if cwms-cli broke anyway?

2. Once we can shorthand environments, that opens some utility in having arguments like --source-env and --target-env. Easy to move stuff to dev or local host.

I'll admit that's handy, though see below.

3. The .env's add up. If you have few repos you can have redundant .envs depending on how you set it up.

Honestly, I don't work with .env files at all. At least willingly. Too risky for accidental credential leakage. I'm not sure why they became so popular.

Actual environment variables can't be committed to github. So I'll set multiple, like CDA_ROOT_DEV, CDA_ROOT_TEST, etc., then just reference them in the command-line of the process. They have their own security issues of course, but different.
It's also more self-documenting. You know which variables you need, so even if your .env file disappears, you can recreate it.

[msweier]

Though I guess it may not matter, since what would they be doing if cwms-cli broke anyway?

Yeah the intent is to help with using python/cwms-cli.

It's opt in, and it just swaps out the CDA_API_ROOT and CDA_API_KEY variables and assign an ENVIRONMENT variable. That way it works with the defaults that cwms-cli is looking for if you are just working in one environment.

[msweier]

Refactored to ditch keyring. Now stores envs as json alongside token store location. Updated load function to take the --source-env and --target-env. Add a connectivity/authentication check in cwms-cli env show --check. Summary of commands:

1. activate (activate env in new shell)
2. delete (delete and env from the config)
3. export (export env to .env or to current shell)
4. setup (setup enviroments...ships with cwbi-prod preconfig)
5. show (show configured env and optionally connectivity)

[krowvin]

Looks good!

See if you can merge if you are ready to bring this in @msweier