Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provenance socioeconomics #303

Merged
Show file tree
Hide file tree
Changes from 17 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 15 additions & 2 deletions _data/CONTRIBUTORS.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -284,6 +284,21 @@ Pierluca Piselli:
Isabel Kemmer:
orcid: https://orcid.org/0000-0002-8799-4671
affiliation: EuroBioImaging
Mari Kleemola:
git: mari-k
email: [email protected]
orcid: 0000-0001-8855-5075
affiliation: Finnish Social Science Data Archive, Tampere University and CESSDA
Simone Sacchi:
git: simosacchi
email: [email protected]
orcid: 0000-0002-6635-7059
affiliation: European University Institute
Irena Vipavc Brvar:
git:
email: [email protected]
orcid: 0000-0001-6068-0210
affiliation: University of Ljubljana, Faculty of Social Sciences, Slovenian Social Science Data Archives
Reagon Karki:
git: jrkarki
email: [email protected]
Expand All @@ -294,5 +309,3 @@ Francesco Messina:
git: INMIbioinfo
affiliation: IRCCS (INMI)
Email: [email protected]


4 changes: 4 additions & 0 deletions _data/news.yml
Original file line number Diff line number Diff line change
Expand Up @@ -146,4 +146,8 @@
date: 2024-10-01
linked_pr: 357
description: Content was added to the Socioeconomic data page on Data Analysis. [Discover the page here](/data-analysis/socioeconomic-data)
- name: "New page: Provenance of Socioeconomic data"
date: 2024-10-02
linked_pr: 303
description: Content was added to the Socioeconomic data page on Provenance. [Discover the page here](/provenance/socioeconomic-data)

2 changes: 2 additions & 0 deletions _data/sidebars/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,8 @@ subitems:
subitems:
- title: Human clinical and health data
url: /provenance/human-clinical-and-health-data
- title: Socioeconomic data
url: /provenance/socioeconomic-data

- title: Quality control
url: /quality-control/
Expand Down
20 changes: 20 additions & 0 deletions _data/tool_and_resource_list.yml
Original file line number Diff line number Diff line change
Expand Up @@ -1226,3 +1226,23 @@
id: trifacta
name: Trifacta
url: https://www.trifacta.com/
- description: Searchable list of tools available to help you work with DDI, from authoring and editing metadata to data transformations. The tools have been developed independently by a variety of organizations from the global DDI community.
id: ddi
name: DDI Tools
url: https://ddialliance.org/resources/tools
- description: The NUTS (Nomenclature of territorial units for statistics) classification, developed by eurostat, is a hierarchical system for dividing up the economic territory of the EU and the UK.
id: nuts
name: NUTS
url: https://ec.europa.eu/eurostat/web/nuts/overview
- description: SDMX Fusion Metadata Registry
id: sdmx-registry
name: SDMX Registry
url: https://registry.sdmx.org/overview.html
- description: The Data Structure Wizard (DSW) is a Java standalone desktop application that supports version 2.0 & 2.1 of the SDMX standard.
id: dsw
name: Data Structure Wizard (DSW)
url: https://sdmx.org/?page_id=4524
- description: The SDMX-Reference Infrastructure (SDMX-RI) is a set of pick-and-choose building blocks and tools that allow data to be exposed to the external world through access rights by using web services.
id: sdmx-ri
name: SDMX-Reference Infrastructure (SDMX-RI)
url: https://sdmx.org/?page_id=4666
55 changes: 38 additions & 17 deletions provenance/socioeconomic-data.md
Original file line number Diff line number Diff line change
@@ -1,27 +1,48 @@
---
title: Socioeconomic data
description: Tracking data and analysis steps.
contributors: []
no_robots: true
search_exclude: true
sitemap: false
description: Tracking socioeconomics data and analysis steps done to them.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This reads weird to me, but it might be me!

EvaGarciaAlvarez marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
description: Tracking socioeconomics data and analysis steps done to them.
description: Tracking socioeconomic data and documenting the analysis steps applied to it.

contributors: [Rudolf Wittner, Mari Kleemola, Simone Sacchi, Diana Pilvar, Irena Vipavc Brvar, Eva Garcia Alvarez, Robin Navest]
page_id: sed_provenance
redirect_from: /socioeconomic-data/provenance
rdmkit:
- name:
url:
training:
- name:
registry:
url:
# More information on how to fill in this metadata section can be found here https://www.infectious-diseases-toolkit.org/contribute/page-metadata

---

**We are still working on the content for this page.** If you are interested in adding to the page, then:
## Introduction
Provenance of socioeconomics data refers to the history of the data, starting from their origin and gathering all the processes applied to them. Socioeconomics data provenance is directly related to their utility, trustworthiness and to some quality dimensions such as reliability. It provides data users the information they need to analyse the data in a meaningful manner, enabling reproducibility.

## Considerations
When generating the provenance of socioeconomics data, there are some factors that must be taken into account:
* Identify and document all actors that took part in the processing of the data from their collection.
* Identify and document the collection methods and the data sources.
* Include information about the socio-economic situation that was in place when the data were generated.
* Document he processes the data went through including the quality measures applied.
* Document different version of the data.
In addition, the provenance data should follow standards, enhancing interoperability.

## Existing approaches
### Data Documentation Initiative metadata standards
Scenarios and use cases, as well as encoding provenance metadata within the widely used Data Documentation Initiative (DDI) metadata standard, are discussed by [Lagoze et al. (2013)](https://doi.org/10.1007/978-3-319-03437-9_13). In this publication they propose an encoding of the provenance metadata using the Data Documentation Initiative (DDI) metadata standard.

[Data Documentation Initiative (DDI)](https://ddialliance.org/products/overview-of-current-products) is a suite of products to describe quantitative and qualitative research data in the social, behavioural, economic, and health sciences. Using DDI one can document and manage different stages of the research data lifecycle, such as conceptualization, collection, processing, distribution, discovery, and archiving. Documenting data with DDI facilitates understanding, interpretation, and use of data by people, software systems, and computer networks. [Controlled vocabularies](https://ddialliance.org/controlled-vocabularies) in several languages are in place, which makes the presentation of metadata in different languages easier.

DDI standards cover the following areas:
* Conceptual objects: concept, unit, unit type, universe, population, geographic structures, and representation.
* Methodological objects: approaches to sample selection, data capture, weighting, quality control, and process management.
* Processing: data capture, data processing, analysis, and data management.
* Quantitative and qualitative data objects: concept, universe, representation, usage, data type, record, record relationships, storage, access, and descriptive statistics
* Data management: ownership, access, rights management, restrictions, quality standards, organization, agent management, relationship between products, versioning, and provenance.

As a practical example, the [CESSDA Metadata Model](https://zenodo.org/doi/10.5281/zenodo.4672413) is based on the DDI Lifecycle standard.

### Structured Data Transformation Language
[Structured Data Transformation Language (SDTL)](https://ddialliance.org/products/sdtl/1.0) is an independent intermediate language for representing data transformation commands. Statistical analysis packages (e.g., SPSS, Stata, SAS, and R) provide similar functionality, but each one has its own proprietary language. SDTL consists of JSON schemas for common operations, such as RECODE, MERGE FILES, and VARIABLE LABELS. SDTL provides machine-actionable descriptions of variable-level data transformation histories derived from any data transformation language. Provenance metadata represented in SDTL can be added to documentation in [DDI](https://ddialliance.org/products/overview-of-current-products) and other metadata standards.

[Feel free to contribute](/contribute/){: class="btn btn-primary btn-lg rounded-pill"}
### Social Sciences and Humanities Open Cloud Reference Ontology
The [Social Sciences and Humanities Open Cloud Reference Ontology (SSHOCro)](https://www.sshopencloud.eu/sshocro) proposes an ontological model and RDF (Resource Description Framework) schema to be used as a top-level ontology for organizing knowledge and information found distributed across various primary sources of information in the Social Sciences and Humanities Open Cloud (SSHOC). It provides a semantic interoperability framework for the description of the data life cycle used by Social Science and Humanities researchers. SSHOCro is modeled as an extension of [CIDOC CRM](https://www.cidoc-crm.org/).

This is a community-driven website, so contributions are welcome! You will, of course, be listed as a contributor on the page.
### Statistical Data and Metadata eXchange
As described in the [official site for the Statistical Data and Metadata eXchange (SDMX) community](https://sdmx.org/?page_id=2555/), SDMX is designed to describe statistical data and metadata, normalise their exchange, and improve their efficient sharing across statistical and similar organisations. It provides an integrated approach to facilitating statistical data and metadata exchange, enabling interoperable implementations within and between systems concerned with the exchange, reporting and dissemination of statistical data and their related meta-information.
It consists of technical standards (including the Information Model), statistical guidelines and an IT architecture and tools and is an [ISO standard](http://www.iso.org/iso/catalogue_detail.htm?csnumber=52500).

New content is announced on the [home page](/) and [news page](/about/news), so please check for updates there. You can also watch for changes on this page by using a free service like [Visual Ping](https://visualping.io/) or [Distill Web Monitor](https://distill.io/), or by using a [browser add-on](https://chrome.google.com/webstore/detail/distill-web-monitor/inlikjemeeknofckkjolnjbpehgadgge?hl=en).
Many IT tools have been developed to support the use and implementation of SDMX, examples of such tools are the {% tool "sdmx-registry" %}, the {% tool "dsw" %} and the {% tool "sdmx-ri" %}.

Loading