Have configurable links to other components

[bhermann]

As an authenticated user, I would like to be able to configure the application to add links to other components to enable the specific functionality for this component.
This includes:

• The docker instance (or swarm, or other means to control docker)
• the crawler component
• the web api component
• the web app component (still unsure what to control there)
• ElasticSearch

These links need to be persisted in some way. This not necessarily needs to be a relational database... Think out of the box here. It should be something easy.

[anitha181992]

Our Understanding:
1)When we try to start one component all the other components should be run in parallel
Approach 1:We can implement a multithreading concept here, if the webapp application as a main thread when started can spawn the threads corresponding to the other components. Hence all the threads are being run in parallel.
Approach 2: In ABM, where we try to add users and configure users, we can have a place where we add the components for Delphi.
Approach 3:When we select one component say the Crawler Component, the user gets an option to select which other components , the user wants to run with it, and configure the same in System configuration file which contains the configurations for all the components.
Questions:
1)”To enable the specific functionality for this component” in the problem statement, does it mean whether an instance or the whole functionality of the component should be enabled
2)When we do SBT run, whether all of these components should start? Should we configure the SBT run for the same?

[bhermann]

Towards Approach 1 and Question 2: There is no need to implement a custom multithreaded approach here. As the components may be deployed on different systems, they need to be running in different processes. There is already a sbt run configuration that starts all components in configured in https://github.com/delphi-hub/delphi .

Towards Approach 2: ABM and Delphi should stay independent with the exception of ABM calling Delphi to get data. The configuration requested here should happen in the delphi-management app and is local for this application only.

Towards Approach 3: Almost... Assume that all other components are already running. For the management application to reach those component, we need their base URLs. And these are the links to be configured.

Towards Question 1 : That's a bit tricky... For some components we just need some monitoring. For instance for the webapp or the webapi. For some components we need measures of control. The crawler, for example, will need some functionality for control.

In a nutshell, there should be some types of components (crawler, webapp, webapi, ...) and links to reach instances of those components. This configuration data needs to be stored somewhere.

I hope this clears it up a bit.

[anitha181992]

As per my understanding,
1)Create a Java interface which stores the Components and the links to them as “Key value ” pairs
-->key being the Component name and value being the url of the component
2)Using this Configuration interface , we can generate an .xml file and store it in properties folder of the project.

[bhermann]

That is not the idea. Let's keep this issue open for next week for you and talk about this in person tomorrow.

[24santoshr]

1)Since we need the data from the Crawler Service, if the Service is implemented is such a way that

Datamodel: A Pojo which has all the setData() and getData() for the URL, statistical data etc.

Datamodel interface: which consists of the queries that fetch data from database and writes to it.

Service: which actually has the core functionality to process the data from the elastic search.

2)We can have the datamodel and service dependency in the Application project of Delphi and calls those methods to display the data to the admin, and also the data changed by the admin can be sent to the service .

3)Similar logic can be applied to the other components as well.

[bhermann]

Sounds good. Please get started on that.
I added a service endpoint to the crawler yesterday, which allows you to query its version.

[bhermann]

Are you still working on this issue, @24santoshr @anitha181992 ?

[anitha181992]

Yes we are still working on it. Since the framework is new, we are taking some time , we were able to understand the AKKA framework, but before we start with the implementation, we wanted to get well versed with the current workflow.

[bhermann]

Just wanted to know. Thanks for clarifying.

[anitha181992]

@bhermann : I added a service endpoint to the crawler yesterday, which allows you to query its version.
Just for a part of clarification:
val resp = client.execute { search in "myindex" / "mytype" query "version"}.await println(resp.toString)

Is this what yields us the version ??