Merge pull request #1 from mbaudis/main

re-write in mkdocs

.github/workflows/mk-docs.yaml

@@ -0,0 +1,19 @@
+name: mk-docs 
+on:
+  push:
+    branches:
+      - main
+jobs:
+  deploy:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v2
+      - uses: actions/setup-python@v2
+        with:
+          python-version: 3.x
+      - run: pip install mkdocs-material 
+      - run: pip install mkdocs-macros-plugin
+      - run: pip install pymdown-extensions
+      - run: pip install mkdocs-mermaid2-plugin
+      - run: pip install mdx_gh_links
+      - run: mkdocs gh-deploy --force

.gitignore

@@ -0,0 +1,3 @@
+
+docs/.DS_Store
+.DS_Store

docs/adminGuide/add-beacon.md

@@ -0,0 +1,88 @@
+---
+date: 2024-10-22
+---
+
+
+# Add a Beacon in the ELIXIR Beacon Network
+
+In this section you will learn how to add a Beacon in the ELIXIR Beacon Network. There are two sections depending on your role:
+
+- Beacon Owner: You are an owner or mantainer of a Beacon and you want to add it to an ELIXIR Beacon Network.
+- ELIXIR Beacon Network (EBN) mantainer: You are an owner or mantainer of an ELIXIR Beacon Network and a Beacon has requested to add their API URL to the network.
+
+## Add a Beacon to a Network (Beacon owner view)
+
+### Add an ID
+
+To being able to classify your beacon in the EBN, you need to create an unique identifier to your Beacon. This ID should be located in the `info` endpoint, inside the `meta` object. Here it is an example:
+
+```
+"meta":{
+	"apiVersion":"vX.X.X",
+	"beaconId":"your.beacon.id",
+	"returnedSchemas": {...}
+}
+```
+
+The `beaconId` must be unique inside the network and it is mandatory to enable individual identification and prevention of duplicates.
+
+### Validate your Beacon
+
+Before asking to add your beacon to the network, you should pass your Beacon to the validator. The Validator can be accessed via Frontend in most of the EBN instances, but it is not mandatory. Here it will be described how to validate your Beacon via command-line:
+
+To validate a Beacon before adding it to the ELIXIR Beacon Network, follow these steps:
+
+1. Ensure you have [Apache Maven](https://maven.apache.org/) installed.
+
+2. Download the validator  repository and navigate to the `tool` folder:
+
+```
+git clone https://github.com/elixir-europe/neat-beacon-v2-validator
+cd neat-beacon-v2-validator
+```
+
+3. Build the code using Maven: 
+
+```
+mvn install
+```
+
+4. After building, you'll find a `target` directory within the `neat-beacon-v2-validator-tool` folder. This directory contains the validator Java application.
+
+5. Run the validator application with the Beacon's API endpoint to be validated. You can use the following command:
+
+```
+java -jar neat-beacon-v2-validator.jar -f https://beacons.bsc.es/beacon/v2.0.0/
+
+java -jar neat-beacon-v2-validator.jar -f https://beacons.bsc.es/beacon/v2.0.0/ -o report.json
+```
+
+This command will validate the beacon instance located at the specified API endpoint.
+Optionally, you can specify an output file for the validation report using the `-o` or `--output` flag. 
+
+### Send it to the Network
+
+After passing the validator, the Beacon owner might need to send an email to the ELIXIR Beacon Network mantainer asking them to add the Beacon to the network. It is recommended to send the validator output, if any, or specify that the Beacon has passed the validation.
+
+The Beacon owner is responsible for addressing any errors reported by the validator. While the Beacon may still function within the Beacon Network despite reported errors, critical errors may prevent it from returning data.
+
+{: .important }
+> ELXIIR Beacon Network maintainers may request the output of the validator before adding the instance to the network.
+
+## Add a Beacon to a Network (ELIXIR Beacon Network mantainer view)
+
+The ELIXIR Beacon Network Mantainer is responsible for requesting the validator output file to confirm that any Beacon meets validation criteria to add it to the network. Subsequently, the Beacon owner should forward the validated Beacon's URL to the EBN Maintainer.
+
+Then, the EBN Maintainer will incorporate the URL of the Beacon into the ELIXIR Beacon v2 Network. This involves adding the Beacon's URL to the `wildfly/BEACON-INF/beacon-network.json` file. 
+
+The structure of this JSON file should be an array containing the URL of the validated Beacon, like this:
+
+```
+[
+	"new.beacon.url/api"
+]
+```
+
+After adding the URL to the `beacon-network.json` file, the Beacon Network will automatically fetch the new Beacon, ensuring its integration into the network. This process facilitates the incorporation of Beacons into the ELIXIR Beacon Network.
+
+To remove the Beacon, you just need to remove the URL from the `beacon-network.json` file. 

docs/adminGuide/beacon-aggregator.md

@@ -0,0 +1,32 @@
+---
+date: 2024-10-22
+---
+
+
+# Beacon Aggregator
+
+The Beacon Aggregator (BA) is the agent in charge of querying the Beacons of the network and gather the results. It also aggregates and parses their metadata.
+
+## Meta-Beacon
+
+The Beacon Aggregator is itself a standard Beacon v2. As it is created to query other Beacons, it is also referred to as Meta-Beacon.
+
+In order to differentiate the Meta-Beacon to a simple Beacon, besides having aggregation functionalities, we have listed the following additional properties:
+
+- `isBeaconNetwork`: If `true`, it means that the beacon is a Meta-Beacon.
+- `beaconId`: Unique identifier for each beacon. It is needed to differentitate where the data comes in a network.
+
+All the beacons in a network also need to have the `beaconId` in their `info` page, as it is needed for the EBN to differentiate each instance.
+
+## Query Policies
+
+The Beacon aggregator sends the query to the multiple Beacons in the network. To reduce to the minimum the queries sent, thus reducing the payload of the server, it follows these policies:
+
+ - First, if a Beacon requires authentication, the BA must provide a header with the authentication of the user to the given Beacon. If the authentication is not needed, it goes to the next step.
+
+ - The BA parses the `/map` endpoint and looks if the Beacon has the model schema needed (individuals, biosamples, etc).
+
+ - Check if the filtering terms of the query exists.
+
+ - In the network, there can be Beacons with different schemas, such as Phenopackets or different versions of Beacon. If the network is not able to read these schemas, it must quit the query for the given Beacon.
+

docs/adminGuide/beacon-registry.md

@@ -0,0 +1,38 @@
+---
+date: 2024-10-22
+---
+
+
+# Beacon Registry
+
+The role of the Beacon registry is to check the Beacons available in the network and gather their basic information. 
+
+## Cache
+
+The cache of the registry store the following information about the Beacons:
+
+- Entry types available to search.
+
+- Type of schemas for representing data and metadata.
+
+- Filtering terms that the beacon support.
+
+- Authentication/Authorization mechanism needed to ensure that only authorized users access the data.
+
+- Basic metadata from the `info` endpoint.
+
+As it can be seen, the data stored in the cache is directly related with the queries policy. So, it needs to be updated every time there is a change in any Beacon.
+
+## Update Registry
+
+Beacon Aggregator will check if the `updateDateTime` property has changed since the last time a Beacon has been updated. If true, I will reload again all the cache of such Beacon.
+
+If a Beacon maintainer wants to force an update of the metadata, they should run the following in the terminal:
+
+```
+ curl -H "Accept: application/json" -H "Cache-Control: no-cache" {url_of_your_ebn}/info
+```
+
+## Status
+
+Heartbeat mechanism that regularly checks the status of the Beacons in the network. This ensures the availability of the different Beacons when querying. (Still in development)

docs/adminGuide/best-practices.md

@@ -0,0 +1,51 @@
+---
+date: 2024-10-22
+---
+
+# Best Practices
+
+## Adding your own filters
+
+As an ELIXIR Beacon Network (EBN) maintainer, along its steering commitee, you decide which filtering terms a Beacon Network (BN) can query. You can avoid all the filtering terms of the beacons in the network, and only use the ones from the static ontologies of the Aggregator.
+
+## User certificates
+
+To ensure the privacy and anonymity of the data, a BN should keep track of the metadata of the queries done by the users. As an example, a user can make multiple queries to retrieve all the genomic data of a patient. This can potentially compromise the privacy of the patient and confidentiality of the data.
+
+To address the issue, Beacon Networks can use User Certificates to know who makes queries to the Beacons. By monitoring their activity we can prevent users from accessing too much data at once and identify suspicious activities. A possible solution is to limit the number of queries per user in a day or to stop giving access to a user when the queries are mistrustful.
+
+This mechanism will build trust and confidence among users and data owners.
+
+## Known issues
+
+Here there are issues that we have found in different Beacons when importing them to the network. In order to solve them, the propietary of such beacon need to be contacted and fix the issue.
+
+### Query per second
+
+When the EBN fetches the Beacons, first it needs to query the metadata in order to parse and put it in the cache. These queries are done in consequential order. If one of your Beacons API has limited the queries per second, this would cause trouble to the Network as it won’t let it search for all the needed metadata.
+
+### Correct paths
+
+The most common mistake when inserting a beacon to the network is having wrong URL paths in the `/map` endpoint. If you are a Beacon maintainer, you must check that all the URLs work properly.
+
+The ELIXIR Beacon Network need to check all the paths in order to query them or retrieve the correct specification. If the path returns any value, the beacon will not be queried.
+
+### Beacon ID and general information
+
+The Beacon ID of the beacon must be unique. If not, the Beacon Network won't be able to process the responses.
+
+If you deploy your beacon based on another instance or [Reference Implementation](https://github.com/EGA-archive/beacon2-ri-api), you should change the general information of the given beacon with the name of your organisation, project, and other info.
+
+### Filtering terms
+
+The scope of the filtering terms endpoint must be per schema.
+
+## Types of Beacon Network
+
+### BN Opaque
+
+This type of Beacon Network allows the user to query the data without knowing its origin. The user knows the data is present in the network but is unaware of which specific Beacon it comes from. This type is particularly useful for rare diseases or situations where identifying the data source is not necessary.
+
+### BN Transparent
+
+Type of Beacon Network where the source is known. When querying data, the BN will provide information about the Beacon holding that information. It will allow users to track the source of data and be able to ask for access to the full dataset, thus allowing data sharing.

docs/adminGuide/light-beacon-network.md

@@ -0,0 +1,102 @@
+---
+date: 2024-10-22
+---
+
+# Light a Beacon v2 Network
+
+When multiple institutions or organizations deploy a Beacon and make it accessible to a network, they form a Beacon Network. This network enables federated querying across multiple beacons without the need of centralized data storage or direct access to the data. Thus, it allows researchers to discover data from multiple beacons in a single query.
+
+If you want to deploy the ELIXIR Beacon v2 Network instance made by the [BSC](https://www.bsc.es/discover-bsc/organisation/scientific-structure/national-institute-bioinformatics-elixir-node-0) (Backend) and [EGA-CRG](https://www.crg.eu/en/programmes-groups/ega-team) (Frontend), follow the next steps.
+
+## Full Docker install (Recommended)
+
+Download the code from this [repository](https://github.com/elixir-europe/beacon-network-docker.git):
+
+```
+git clone https://github.com/elixir-europe/beacon-network-docker.git
+cd beacon-network-docker
+```
+
+Edit the `config.json` file located in [frontend/](frontend/) to point to the URLs where you will be making the queries. Here's an example of how the `config.json` file should look:
+
+```
+{
+  "API_URL": "http://localhost:8080/beacon-network/v2.0.0",
+  "REDIRECT_URL": "https://yourUIdomain.com",
+  "KEYCLOAK_URL": "https://yourKEYCLOAKdomain.com"
+}
+```
+Then, replace [http://localhost:8080/beacon-network/v2.0.0](http://localhost:8080/beacon-network/v2.0.0) and the other URLs with the appropriate URL where your backend is hosted.
+
+Run [docker compose](https://docs.docker.com/compose/) to build and start the containers:
+
+``` 
+ docker compose up -d
+```
+
+Congrats! You have succesfully deployed ELIXIR Beacon Network FrontEnd in [http://localhost:8080/](http://localhost:8080/) and the backend in [http://localhost:8080/beacon-network/v2.0.0/](http://localhost:8080/beacon-network/v2.0.0/). Adjust the URLs based on your setup.
+
+
+## Backend Installation
+
+Download the code from this [repository](https://github.com/elixir-europe/beacon-network-backend.git), and run [docker compose](https://docs.docker.com/compose/):
+
+```
+git clone https://github.com/elixir-europe/beacon-network-backend.git
+cd beacon-network-backend/docker
+docker compose up -d
+```
+
+Congrats! You have deployed the ELIXIR Beacon Network backend in [http://localhost:8080/beacon-network/v2.0.0/](http://localhost:8080/beacon-network/v2.0.0/). You can now view and query the backend with API GET/POST queries. Adjust the URLs as necessary based on your setup.
+
+## FrontEnd Installation
+
+Download the code from this [repository](https://github.com/elixir-europe/beacon-network-ui.git):
+
+```
+git clone https://github.com/elixir-europe/beacon-network-ui.git
+cd beacon-network-ui
+```
+
+Create a `.env` file inside the folder and modify the following variables:
+
+```
+REACT_APP_CLIENT_ID="ID of your LS Login"
+REACT_APP_CLIENT_SECRET="password of your LS Login"
+```
+
+Run [docker compose](https://docs.docker.com/compose/):
+
+```
+docker-compose up -d –build
+```
+
+Edit `config.json` from [frontend/src](frontend/src) to point the URLs where you have the backend, main webpage and keycloak. Example:
+
+```
+{
+  "API_URL": "https://yourAPIdomain.com/beacon-network/v2.0.0",
+  "REDIRECT_URL": "https://yourUIdomain.com",
+  "KEYCLOAK_URL": "https://yourKEYCLOAKdomain.com"
+}
+```
+
+You have deployed the Beacon Network Frontend in [http://localhost:3000](http://localhost:3000). Adjust the URLs as needed based on your setup.
+
+# Update the EBN
+
+To update the ELIXIR Beacon Network (EBN) with a new image, follow these steps:
+
+- Pull the latest image:
+
+```
+docker-compose pull
+```
+
+- Recreate and rebuild the containers with the new image:
+
+```
+docker-compose up --force-recreate --build -d
+```
+
+These commands will download the latest image for the EBN and recreate the containers using the new image. The `--force-recreate` flag ensures that the containers are recreated even if they are already running, and the `--build` flag rebuilds the containers if necessary. The `-d flag runs the containers in detached mode.