Provenance socioeconomics (#303)

* Starting provenance socioeconomic-data.md * Add prov socioec tools tool_and_resource_list.yml * Second update of socioeconomic-data.md * Update socioeconomic-data.md * New contributors to CONTRIBUTORS.yaml * Update tool_and_resource_list.yml * Update socioeconomic-data.md * Update socioeconomic-data.md * Update CONTRIBUTORS.yaml * Adding provenance socioeco to sidebar main.yml * Update tool_and_resource_list.yml * move tools * move tools for real * add news item * EOL * Apply suggestions from code review Co-authored-by: Bert Droesbeke <[email protected]> --------- Co-authored-by: Bert Droesbeke <[email protected]> Co-authored-by: bedroesb <[email protected]>
elixir-europe · Oct 4, 2024 · 24f7dca · 24f7dca
1 parent 666fa53
commit 24f7dca
Show file tree

Hide file tree

Showing 5 changed files with 79 additions and 20 deletions.
diff --git a/_data/CONTRIBUTORS.yaml b/_data/CONTRIBUTORS.yaml
@@ -284,6 +284,21 @@ Pierluca Piselli:
 Isabel Kemmer:
   orcid: https://orcid.org/0000-0002-8799-4671
   affiliation: EuroBioImaging
+Mari Kleemola:
+  git: mari-k
+  email: [email protected]
+  orcid: 0000-0001-8855-5075
+  affiliation: Finnish Social Science Data Archive, Tampere University and CESSDA
+Simone Sacchi:
+  git: simosacchi
+  email: [email protected]
+  orcid: 0000-0002-6635-7059
+  affiliation: European University Institute
+Irena Vipavc Brvar:
+  git:
+  email: [email protected]
+  orcid: 0000-0001-6068-0210
+  affiliation: University of Ljubljana, Faculty of Social Sciences, Slovenian Social Science Data Archives  
 Reagon Karki:
   git: jrkarki
   email: [email protected]
@@ -294,5 +309,3 @@ Francesco Messina:
   git: INMIbioinfo
   affiliation: IRCCS (INMI)
   Email: [email protected]
-
-
diff --git a/_data/news.yml b/_data/news.yml
@@ -146,4 +146,7 @@
   date: 2024-10-01
   linked_pr: 357
   description: Content was added to the Socioeconomic data page on Data Analysis. [Discover the page here](/data-analysis/socioeconomic-data)
-
+- name: "New page: Provenance of Socioeconomic data"
+  date: 2024-10-02
+  linked_pr: 303
+  description: Content was added to the Socioeconomic data page on Provenance. [Discover the page here](/provenance/socioeconomic-data)
diff --git a/_data/sidebars/main.yml b/_data/sidebars/main.yml
@@ -59,6 +59,8 @@ subitems:
     subitems:
       - title: Human clinical and health data
         url: /provenance/human-clinical-and-health-data
+      - title: Socioeconomic data
+        url: /provenance/socioeconomic-data
 
   - title: Quality control
     url: /quality-control/

diff --git a/_data/tool_and_resource_list.yml b/_data/tool_and_resource_list.yml
@@ -1226,3 +1226,23 @@
   id: trifacta
   name: Trifacta
   url: https://www.trifacta.com/
+- description: Searchable list of tools available to help you work with DDI, from authoring and editing metadata to data transformations. The tools have been developed independently by a variety of organizations from the global DDI community.
+  id: ddi
+  name: DDI Tools
+  url: https://ddialliance.org/resources/tools
+- description: The NUTS (Nomenclature of territorial units for statistics) classification, developed by eurostat, is a hierarchical system for dividing up the economic territory of the EU and the UK.
+  id: nuts
+  name: NUTS
+  url: https://ec.europa.eu/eurostat/web/nuts/overview
+- description: SDMX Fusion Metadata Registry
+  id: sdmx-registry
+  name: SDMX Registry
+  url: https://registry.sdmx.org/overview.html
+- description: The Data Structure Wizard (DSW) is a Java standalone desktop application that supports version 2.0 & 2.1 of the SDMX standard.
+  id: dsw
+  name: Data Structure Wizard (DSW)
+  url: https://sdmx.org/?page_id=4524
+- description: The SDMX-Reference Infrastructure (SDMX-RI) is a set of pick-and-choose building blocks and tools that allow data to be exposed to the external world through access rights by using web services.
+  id: sdmx-ri
+  name: SDMX-Reference Infrastructure (SDMX-RI)
+  url: https://sdmx.org/?page_id=4666
diff --git a/provenance/socioeconomic-data.md b/provenance/socioeconomic-data.md
@@ -1,27 +1,48 @@
 ---
 title: Socioeconomic data
-description: Tracking data and analysis steps.
-contributors: []
-no_robots: true
-search_exclude: true
-sitemap: false
+description: Tracking socioeconomic data and documenting the analysis steps applied to it.
+contributors: [Rudolf Wittner, Mari Kleemola, Simone Sacchi, Diana Pilvar, Irena Vipavc Brvar, Eva Garcia Alvarez, Robin Navest]
 page_id: sed_provenance
 redirect_from: /socioeconomic-data/provenance
-rdmkit:
-  - name:
-    url:
-training:
-  - name:
-    registry:
-    url:
-# More information on how to fill in this metadata section can be found here https://www.infectious-diseases-toolkit.org/contribute/page-metadata
+
 ---
 
-**We are still working on the content for this page.** If you are interested in adding to the page, then:
+## Introduction
+Provenance of socioeconomics data refers to the history of the data, starting from their origin and gathering all the processes applied to them. Socioeconomics data provenance is directly related to their utility, trustworthiness and to some quality dimensions such as reliability. It provides data users the information they need to analyse the data in a meaningful manner, enabling reproducibility.
+
+## Considerations
+When generating the provenance of socioeconomics data, there are some factors that must be taken into account:
+*	Identify and document all actors that took part in the processing of the data from their collection. 
+*	Identify and document the collection methods and the data sources.
+*	Include information about the socio-economic situation that was in place when the data were generated. 
+*	Document he processes the data went through including the quality measures applied.
+*	Document different version of the data.
+In addition, the provenance data should follow standards, enhancing interoperability.
+
+## Existing approaches
+### Data Documentation Initiative metadata standards
+Scenarios and use cases, as well as encoding provenance metadata within the widely used Data Documentation Initiative (DDI) metadata standard, are discussed by [Lagoze et al. (2013)](https://doi.org/10.1007/978-3-319-03437-9_13). In this publication they propose an encoding of the provenance metadata using the Data Documentation Initiative (DDI) metadata standard.
+
+[Data Documentation Initiative (DDI)](https://ddialliance.org/products/overview-of-current-products) is a suite of products to describe quantitative and qualitative research data in the social, behavioural, economic, and health sciences. Using DDI one can document and manage different stages of the research data lifecycle, such as conceptualization, collection, processing, distribution, discovery, and archiving. Documenting data with DDI facilitates understanding, interpretation, and use of data by people, software systems, and computer networks. [Controlled vocabularies](https://ddialliance.org/controlled-vocabularies) in several languages are in place, which makes the presentation of metadata in different languages easier.
+
+DDI standards cover the following areas:
+* Conceptual objects: concept, unit, unit type, universe, population, geographic structures, and representation.
+* Methodological objects: approaches to sample selection, data capture, weighting, quality control, and process management.
+* Processing: data capture, data processing, analysis, and data management.
+* Quantitative and qualitative data objects: concept, universe, representation, usage, data type, record, record relationships, storage, access, and descriptive statistics
+* Data management: ownership, access, rights management, restrictions, quality standards, organization, agent management, relationship between products, versioning, and provenance.
+
+As a practical example, the [CESSDA Metadata Model](https://zenodo.org/doi/10.5281/zenodo.4672413) is based on the DDI Lifecycle standard.
+
+### Structured Data Transformation Language
+[Structured Data Transformation Language (SDTL)](https://ddialliance.org/products/sdtl/1.0) is an independent intermediate language for representing data transformation commands. Statistical analysis packages (e.g., SPSS, Stata, SAS, and R) provide similar functionality, but each one has its own proprietary language. SDTL consists of JSON schemas for common operations, such as RECODE, MERGE FILES, and VARIABLE LABELS. SDTL provides machine-actionable descriptions of variable-level data transformation histories derived from any data transformation language. Provenance metadata represented in SDTL can be added to documentation in [DDI](https://ddialliance.org/products/overview-of-current-products) and other metadata standards.
 
-[Feel free to contribute](/contribute/){: class="btn btn-primary btn-lg rounded-pill"}
+### Social Sciences and Humanities Open Cloud Reference Ontology
+The [Social Sciences and Humanities Open Cloud Reference Ontology (SSHOCro)](https://www.sshopencloud.eu/sshocro) proposes an ontological model and RDF (Resource Description Framework) schema to be used as a top-level ontology for organizing knowledge and information found distributed across various primary sources of information in the Social Sciences and Humanities Open Cloud (SSHOC). It provides a semantic interoperability framework for the description of the data life cycle used by Social Science and Humanities researchers. SSHOCro is modeled as an extension of [CIDOC CRM](https://www.cidoc-crm.org/).
 
-This is a community-driven website, so contributions are welcome! You will, of course, be listed as a contributor on the page.
+### Statistical Data and Metadata eXchange
+As described in the [official site for the Statistical Data and Metadata eXchange (SDMX) community](https://sdmx.org/?page_id=2555/), SDMX is designed to describe statistical data and metadata, normalise their exchange, and improve their efficient sharing across statistical and similar organisations. It provides an integrated approach to facilitating statistical data and metadata exchange, enabling interoperable implementations within and between systems concerned with the exchange, reporting and dissemination of statistical data and their related meta-information.
+It consists of technical standards (including the Information Model), statistical guidelines and an IT architecture and tools and is an [ISO standard](http://www.iso.org/iso/catalogue_detail.htm?csnumber=52500).
 
-New content is announced on the [home page](/) and [news page](/about/news), so please check for updates there. You can also watch for changes on this page by using a free service like [Visual Ping](https://visualping.io/) or [Distill Web Monitor](https://distill.io/), or by using a [browser add-on](https://chrome.google.com/webstore/detail/distill-web-monitor/inlikjemeeknofckkjolnjbpehgadgge?hl=en).
+Many IT tools have been developed to support the use and implementation of SDMX, examples of such tools are the {% tool "sdmx-registry" %}, the {% tool "dsw" %} and the {% tool "sdmx-ri" %}.