Nesting vs. using references

[FlorianRhiem]

Various nodes have a relationship to other nodes, e.g. a comment will have an author, a dataset has parts, etc. These can either be represented in a nested way, e.g. like this:

{
  "@id": "#alice",
  "@type": "Person",
  "name": "Alice",
  "description": "One of hopefully many Contextual Entities"
}

or they can only contain the @id of another node in the graph and serve as references:

{"@id": "#alice"}

The examples in the RO Crate spec use the reference form (unless I missed one?) and that form has the advantage that there can be no conflicting information for a node. However examples for elabftw and kadi4mat use the nested form, in the case of elabftw even without an "@id" so that it is difficult to detect duplicates.

Is this intentional? If yes, what is the motivation behind it? If not, could we perhaps agree on using the reference form to avoid duplicates altogether and make parsing easier, or at least require the use of an @id to make sure duplicates can be detected reliably?

[nicobrandt]

I would agree with always using an @id at least. Actually, it might even be required as per RO-Crate specification.

About the Kadi4Mat example specifically, after discussing it again with @jmanideep, we also agree that we should simply use the @id instead of the whole nested author node, as typically there might e.g. be lots of files in a Kadi4Mat record that share the same author. This would reduce the overall size of the JSON-LD quite a bit, in our case at least. In general though, I think this aspect should be left to the respective ELN, though I personally wouldn't mind standardizing it as well.

[NicolasCARPi]

@FlorianRhiem could you point me to where the missing @id is please?

[FlorianRhiem]

I noticed it for Person nodes in the elabftw examples, e.g. here.

[NicolasCARPi]

on the doc they use the orcid as ID, but in my app the orcid is not necessarily here, so I use identifier attribute.

But anyway, I agree with you that we should use references everywhere and only define something once. I'll see how I can make sure that all nodes have an @id.

[nicobrandt]

If anyone wants to take a look, the latest Kadi4Mat example (https://github.com/TheELNConsortium/TheELNFileFormat/tree/master/examples/kadi4mat) now uses a flattened form for all entities, namely authors, licenses and even the organization used for the publisher.

[NicolasCARPi]

Hey @nicobrandt @FlorianRhiem ,

So I have done these changes:

an author is now a node with an @id

The URI is a shasum of firstname + lastname + orcid if it exists

an author property simply points to that node with that @id

comments have now an id too, which is an internal URI:

Here the URI is the date of the comment.

As you can see, I've made some custom URI for the @id. Comments are welcome.

@nicobrandt I see that Kadi uses https://localhost:5000/users/34, but IMHO this is information that should not be published in the ELN. The numerical ID of a given user is somewhat secret, or at least should not be disclosed purposefully. Think of an attack vector where an attacker needs to know:

• the instance URL
• victim userid
• victim email (easy to find)

Here you're giving the two missing information to the attacker ;)

Note that my URI scheme isn't resistant to collisions. For authors: two "Li Li" will have the same hash, unless one of them has an orcid! There are also not stable (if they change their name). For comments: two comments at the same second will clash.

But for the intended purpose, it seems good enough.

As I said, your comments are very welcome! I'm open to suggestions on how to improve this.

[nicobrandt]

I see that Kadi uses https://localhost:5000/users/34, but IMHO this is information that should not be published in the ELN.

I appreciate the concern, but I'm not too worried about the numerical IDs at the moment, since this would be more of a "security of obscurity" thing. We already use the instance URL all over the place anyways, and brute forcing the user ID in addition would not be hard if one knew (e.g. from reading the code) that the IDs are simply sequential, especially if there are not too many users in an instance. I also recently implemented the possibility for users to specify their ORCID iD, so these will be used instead if present similar as in eLab, though I realize this only helps if people actually use it ;)

About your IDs, I think it's an interesting idea, but ultimately not something other ELNs could really make use of (same as with our internal URLs of course). Therefore, it's mostly useful for referencing authors within the metadata, but here the potential collisions may be a problem. If you want to stick with this scheme, maybe some additional user-specific information could be included? Since it is hashed, this information should not be reconstructible.

[jmanideep]

Hey @NicolasCARPi ,

I think while flattening, we should use @id in the flattened structure. This @id will be the same in the referenced object.

In your 2nd screenshot, for eg: author should be an object with only @id. Then we use this @id in the author object.

A small snippet of the flattened structure of author object from Kadi4Mat:

"author": {
        "@id": "http://localhost:5000/users/1"
      },

[NicolasCARPi]

If you want to stick with this scheme, maybe some additional user-specific information could be included? Since it is hashed, this information should not be reconstructible.

Yes, I'll add the register_date to the mix! It's immutable and unique!

I think while flattening, we should use @id in the flattened structure.

Yes, my mistake, it should indeed be as you show, not a string directly but an array with @id. I'll fix that!

Thanks for your feedback both of you :)