Discuss via Slack #releases:dbpedia.slack.org

NOTE PLEASE READ THE MANUAL HERE This page below is not up to date yet and will mostly contain info about the development of the software and later the DataID Ontology.

Databus Maven Plugin

Aligning data and software lifecycle with Maven

The plugin was developed to use the features of the Maven software build automation tool for data releases and metadata generation. The tool has the following features:

• once configured properly (1-3 hours), data can be released and released systematically in minutes
• auto-detect RDF formats with syntax validation
• RDF is NOT a requirement, any data can be released (binary, csv, xml), however with RDF the tool has more features
• auto-detect compression variant
• private key signature, sha256sum and provenance (WebID) generation
• generation of metadata compatible to:
  • RSS feeds

Metadata Standards

DBpedia's DataID fulfills 31 of 35 Best Practices from the W3C Data on the Web Best Practices Working Group, cf. implementation report

Roadmap

We are planning the following features:

• DCAT and DCAT-AP interoperability
• FAIR Data principles
• automatic generation of H2020 EU Data Management Plan Deliverables
  • feature exists, but is not yet integrated:
  • https://wiki.dbpedia.org/use-cases/data-management-plan-extension-dataid
• automatic upload of metadata to other repositories:
  • http://lod-cloud.net
  • CKAN
  • RE3

Did we forget something? suggest more interoperability in the issue tracker: https://github.com/dbpedia/databus-maven-plugin/issues

Table of Contents

Development

License

License of the software is AGPL with intended copyleft. We expect that you spend your best effort to commit upstream to make this tool better or at least that your extensions are made available again. Any contribution will be merged under the copyright of the DBpedia Association.

Development rules

• configuration values taken from Maven are configured in Properties.scala, use its 'sub-trait' Locations.scala to derive filesystem locations from these and Parameters.scala to compute all other values derived from the original Maven properties (refactoring into this separation is not yet complete, but please heed this guidelines for additional configuration-derived fields nontheless)
• Datafile.scala is a quasi decorator for files, use getInputStream to open any file
• Use the issue tracker, do branches instead of forks (we can give access), we will merge with master
• Document options in the archetype pom and here

Troubleshooting

Download from http://databus.dbpedia.org:8081/repository/ fails, no dependency information available

Note: this section can be removed after completion of dbpedia#12 Possible reason: we have installed a dev archiva for now. Depending on your org's network configuration, code might only be accepted from Maven Central and local/allowed maven repos.

• [WARNING] The POM for org.dbpedia.databus:databus-maven-plugin:jar:1.0-SNAPSHOT is missing, no dependency information available
• Could not resolve dependencies for project org.dbpedia.databus:databus-maven-plugin:maven-plugin:1.0-SNAPSHOT: Failure to find org.dbpedia.databus:databus-shared-lib:jar:0.1.4

Can potentially fixed by locally installing the shared-lib:

• Download Jar: http://databus.dbpedia.org:8081/#artifact-details-download-content/org.dbpedia.databus/databus-shared-lib/0.1.4
• Install dependency: https://maven.apache.org/guides/mini/guide-3rd-party-jars-local.html

Then clone the repo and run mvn install which will install the databus-maven-plugin locally

databus plugin goals are not found after installing the plugin via sources (mvn install)

[ERROR] Could not find goal 'metadata' in plugin org.dbpedia.databus:databus-maven-plugin:1.1-SNAPSHOT among available goals -> [Help 1]
org.apache.maven.plugin.MojoNotFoundException: Could not find goal 'metadata' in plugin org.dbpedia.databus:databus-maven-plugin:1.1-SNAPSHOT among available goals 

Try to wipe (make a copy of it and then delete the original) your m2 (maven local repository) and then build it again.

BUILD FAILURE, no mojo-descriptors found (when using mvn install to install the databus-maven-plugin)

This is most likely caused by using an old maven version (observed in version 3.0.5) A workaround for this would be replacing:

<plugin>
	<groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-plugin-plugin</artifactId>
	<version>3.4</version>
</plugin>

with

<plugin>
	<groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-plugin-plugin</artifactId>
	<version>3.4</version>
        <configuration>
		<skipErrorNoDescriptorsFound>true</skipErrorNoDescriptorsFound>
	</configuration>
        <executions>
		<execution>
		        <id>mojo-descriptor</id>
		        <goals>
                            <goal>descriptor</goal>
                        </goals>
		</execution>
	</executions>
</plugin>

in databus-maven-plugin/pom.xml

UTF-8 - Encoding Errors in the produced data

On Unix: run: grep "LC_ALL" .* in your /root/ directory and make sure

.bash_profile:export LC_ALL=en_US.UTF-8
.bashrc:export LC_ALL=en_US.UTF-8

is set.