Feature: Make Accelerator independent of ConfigurationManager

[TeresiaOlsson]

Description, motivation and use case

At the moment Accelerator.load uses the ConfigurationManager to load and build the configuration from a file. I think this makes it more complicated for the users to understand what happens during the loading process since they need to understand the ConfigurationManager despite that it in this case just loads a single file and calls Accelerator.from_dict.

It also makes the Accelerator dependent on the ConfigurationManager which might not be the most modular choice for the future.

There is also a use case for building single devices without building the whole accelerator and if the ConfigurationManager is made more modular it might be interesting in the future to extend it to not only aggregate an accelerator configuration from fragments but also extract fragments from an existing accelerator configuration to build single devices.

Proposed solution

Make the ConfigurationManager an external tool which can be used to build a single configuration from fragments. Instead of ConfigurationManager.build building an accelerator object it should build a configuration object/dict which can be feed into the factory to build any object.

The goal should be to eventually move it out from the pyaml repository into it's own repository.

Checklist

• I've assigned this issue to a project
• I've @-mentioned relevant people

[TeresiaOlsson]

@gupichon What do you think about this?

[gupichon]

I agree, but since @JeanLucPons asked me to use it for the integration tests, I am currently using it that way in pyaml. However, it's still a work in progress, so I can easily change my approach.
The PR: #306

[JeanLucPons]

I agree, to me this configuration manager is a bit heavy and difficult to understand.
However it allows to construct an accelerator by fragments. So you can use dynamic fragments and static fragments.
For the integration test this can be used to create dynamic fragments for the control systems but other solutions exist to handle this "prefix issue".
My wish is to be able to run CI test without changing any config files and that it is able to retrieve the TANGO_HOST of the VA.

[gupichon]

My wish is to be able to run CI test without changing any config files and that it is able to retrieve the TANGO_HOST of the VA.

How do you want to handle that if you have several VAs running on different servers? Currently, I'm using the TANGO_HOST environment variable.
You can set it before running the tests.

[TeresiaOlsson]

I think the configuration manager is a good idea but it should be used as an external tool to be able to manage the configuration separately. From it you can extract the full configuration in the format needed for loading into pyAML. Then only the users who are responsible for managing the configuration in their labs need to understand it while the rest more easily can understand the steps done by Accelerator.load.

For the prefix I plan to add that as the next PR after #304. I will just add that you can put an environment variable in the config and that is resolved when loading the config. Similar to how file paths are resolved at that stage. Then you can use TANGO_HOST or any other environment variable of your choice in the config instead of explicitly writing the string.

Only part left that I think @JeanLucPons is missing then is an easy way to know what the TANGO_HOST of the twin is but if I understood it correctly from @gupichon you can set it yourself when you start the twin? Only if you don't set anything you get by default the first free port. But that @Sulimankhail said maybe could be changed to instead give you a port based on your user id so it more easily can be predicted by the user.

[gupichon]

I think the configuration manager is a good idea but it should be used as an external tool to be able to manage the configuration separately.

If you ask me, PyAML should just provide an API and the whole configuration should be an external project. So, yes, I agree.

I will just add that you can put an environment variable in the config and that is resolved when loading the config.

Yes, being able to use environment variables in the configuration is a good idea, and not just for this specific use case.

Only part left that I think @JeanLucPons is missing then is an easy way to know what the TANGO_HOST of the twin is but if I understood it correctly from @gupichon you can set it yourself when you start the twin?

The thing is, you can run as many twins as you want on different ports or servers. Right now, I have found a way to run it through a simple script with the TANGO_HOST env variable, but it's not perfect. I'm waiting to have a fully merged version of the twin from @Sulimankhail with all our requirements to use it, but that is going to take some time. It is a complex task, but @Sulimankhail is working actively on it and I'm very happy with what he is delivering. There are a lot of ways to achieve this, we just have to agree on one.

[TeresiaOlsson]

I mostly made this issue now because I got worried that I needed in PR #279 to also add the new way of doing validation inside the ConfigurationManager. But I managed to avoid it for now by putting it in Accelerator.from_dict instead.

So moving the ConfigurationManager out of the Accelerator can wait until later.

[gupichon]

Another advantage of using the ConfigurationManager is that it allows us to see all the files used in a single, relatively short call. Otherwise, with all the nested file inclusions, it becomes very difficult to track which file is used for which test.