Please keep this file up to date and document the new importers or update current ones if necessary.
- Linked Events importers and commands
- Removed importers
First, some terminology is in order. Starting with ontology:
In computer science and information science, an ontology encompasses a representation, formal naming and definition of the categories, properties and relations between the concepts, data and entities that substantiate one, many or all domains of discourse.
YSO, short for Yleinen suomalainen ontologia, is basically a +30,000 collection of * concepts* in Finnish culture, translated in 3 languages.
General Finnish Ontology YSO is a trilingual ontology consisting mainly of general concepts. YSO has been founded on the basis of concepts in Finnish cultural sphere.
From https://finto.fi/yso/en/. You can browse the concepts there if you wish
Basic example would be, let's say a Portuguese language teacher coming to Helsinki to teach Portuguese to children and also visit a sauna.
We might categorize it with "chldren", "education", "Portuguese", and "sauna" concepts.
These concepts come from YSO.
Linked Events application has events. Events must be described by concepts. Therefore, all events depend on YSO.
add_helsinki_audience also depends on this importer.
Import all keywords in YSO into the database, without maintaining the existing hierarchy between the concepts.
python manage.py event_import yso --keywords --allThis is scheduled to run daily on the instance as data doesn't change that often.
tprek, short for Toimipisterekisteri, is a location registry service giving access to locations in Helsinki through a REST API.
Basic example would be the location of a concert or a Kung Fu course in a certain sport facility.
Linked Events application has events. Events must have locations.
By default, all event locations are from tprek location registry.
Therefore, all events depend on tprek locations to be present.
Import all locations, which usually have addresses, coordinates and other metadata, from tprek location registry into the database.
python manage.py event_import tprek --placesThis is scheduled to run hourly on the instance as data changes often.
Let's get terminology in order.
address is a field in a location object. location can have other metadata.
Pääkaupunkiseudun osoiteluettelo is an address registry which only contains addresses. osoite is a Metropolitan location list importer, which creates a location object with only one field of address from the addresses imported from Pääkaupunkiseudun osoiteluettelo.
Basic example would be the location of a one-off random event in a private location. Locations not meant to be used regularly are not usually added to tprek.
If you want to give the users of your application the ability to add events with locations which are not present in tprek, you need osoite.
In other words, osoite is fall-back for tprek.
Create locations from osoite address registry into the database.
python manage.py event_import osoite --placesThis is scheduled to run daily on the instance as origin database updates daily too.
espoo is an importer for Espoo city Events from https://espoo.fi.
Basic example would be an event organized by Espoo city for its residents, which you can import with this importer.
If you want to populate the database with events organized by Espoo city, you need espoo importer.
Import all events organized by Espoo city into the database.
python manage.py event_import espoo --eventsThis is scheduled to run hourly on the instance as data changes often.
add_helsinki_audience is a django management command for creating a set of audience
keywords in the database which is then exposed through /keyword_set endpoint.
Basic example would be an event whose target audience are elderly immigrants.
"Elderly" and "Immigrant" are keywords that the event creator would pick from the choices in the audience keywords set created by this command.
Any YSO keyword can be used as audience in the backend. However, if you're running the Linked Events UI, you need this command as the UI only allows the keywords set with this command to be used.
You are free to make your own UI that uses any YSO concept for the target audience and you can skip this command.
Keywords in this set are all from YSO except the ones defined
in NEW_SOTE_KEYWORDS_DATA
in add_helsinki_audience.py.
Concepts defined in NEW_SOTE_KEYWORDS_DATA were not found in YSO, and therefore were
hard-coded manually.
Create a Helsinki audience keywords set into the database.
python manage.py add_helsinki_audienceadd_helsinki_topics is a django management command for creating a set of topics
keywords in the database which is then exposed through /keyword_set endpoint.
Basic example would be an event in which they would talk about welfare.
"Discussion" and "Welfare" are keywords that the event creator would pick from the choices in the topics keywords set created by this command.
All the topics used by this command are existing YSO concepts, hand-picked in add_helsinki_topics.py.
If you're running the Linked Events UI, you need this command as the UI needs to show a list of event topics keywords for the event creator to choose for the event they would like to create.
Create a Helsinki topic keywords set into the database.
python manage.py add_helsinki_topicsimport_organizations is an importer for importing all City of Helsinki organization divisions while maintaining their hierarchy into the database.
Basic example would be, say City of Helsinki has A, B, and C divisions, and A and B have two sub-divisions themselves.
This importer would import all 5 divisions while maintaining the hierarchy within the divisions.
If you're having an application whose target users are City of Helsinki organization employees, you need this importer.
Then, you can assign required permissions to users in django-admin based on which division they belong to.
Import all City of Helsinki organization divisions into the database while maintaining their hierarchy.
python manage.py import_organizations https://dev.hel.fi/paatokset/v1/organization/?limit=1000 -c openahjo -s OpenAhjoAPI:ahjoinstall_templates is a django management command for giving the browsable API which comes from django-rest-framework a Helsinki brand look and feel and also instructions for the API user.
Basic example would be the live version
of Linked Events API for Helsinki city
which is blue and also includes instructions on how to query the API when you'd like to
apply certain filters to the /event endpoint.
If you're having an application for Helsinki city and you'd like the browsable API to have Helsinki brand look and feel and also instructions for API user, you need this django management command.
Install Helsinki city templates for the browsable API.
python manage.py install_templates heleventsgeo_import is an importer which creates divisions data structure for each place object in database.
For example, a simplified place object from Sello library can be like so:
{
...
"id": "tprek:15417",
"position": {
"type": "Point",
"coordinates": [
24.80992,
60.21748
]
},
"name": {
"fi": "Sellon kirjasto",
"sv": "Sellobiblioteket",
"en": "Sellon kirjasto"
},
...
},And by running this importer, you'd add divisions to the object above:
{
...
"id": "tprek:15417",
"divisions": [
{
"type": "muni",
"ocd_id": "ocd-division/country:fi/kunta:espoo",
"municipality": null,
"name": {
"fi": "Espoo",
"sv": "Esbo"
}
}
],
"position": {
"type": "Point",
"coordinates": [
24.80992,
60.21748
]
},
"name": {
"fi": "Sellon kirjasto",
"sv": "Sellobiblioteket",
"en": "Sellon kirjasto"
},
...
},This allows the API user to query the /place endpoint by adding division of Espoo as
an example:
/place/?division=Espoo
Or directly querying the /event endpoint by filtering division of Espoo as an example:
/event/?division=Espoo
If you'd like to add support for division based filtering of events or places in your application, then you need this importer.
Import all municipalities in Finland from https://kartat.kapsi.fi/.
python manage.py geo_import finland --municipalitiesImport all divisions in Helsinki from
the Helsinki district division datasets.
This allows more specific filtering for Helsinki city based on district names,
sub_district names, and neighborhood names.
python manage.py geo_import helsinki --divisionskulke, short for Kulttuurikeskukset, is an importer which imports events and courses from City of Helsinki cultural centers (http://annantalo.fi, http://vuotalo.fi, http://malmitalo.fi etc.).
NOTE: To get access to data from City of Helsinki cultural centers, you need to have access to City of Helsinki internal network. Specifics of how to get this access for City of Helsinki developers will be documented here later.
If you'd like to have events and courses organized by City of Helsinki cultural centers in your Linked Events application instance, you need this importer.
Import all events organized by City of Helsinki cultural centers.
python manage.py event_import kulke --eventsImport all courses organized by City of Helsinki cultural centers.
python manage.py event_import kulke --coursesThese are scheduled to run hourly on the instance as data changes often.
lippupiste is an importer which imports events from Lippupiste Oy (https://lippu.fi); currently used only for importing Helsinki City Theater events.
NOTE: To get access to data from Lippupiste, you need to have an agreement with Lippupiste Oy. The data is not publicly available at the moment.
If you'd like to have events marketed by Lippupiste anywhere in Finland in your Linked Events application instance, you need this importer.
Import events marketed by Lippupiste Oy into the database. Currently, the importer only imports plays by the Helsinki City Theater; the imported locations and events can be filtered in the importer code.
python manage.py event_import lippupiste --eventsThis is scheduled to run hourly on the instance as data changes often.
Enkora is an importer which imports courses from Enkora ERP used by Sports and Leisure.
Please note, Enkora is not a public system and any access to it requires credentials. Enkora acts as a data provider for other systems, like LinkedEvents with information on activities and courses.
Activities from Enkora are published on number of websites and applications by City of Helsinki.
This importer is merely a bridge to transmit and transform activity data to LinkedEvents.
Dependencies:
- Credentials into Enkora API
- YSO keywords imported to SQL
- TPrek places imported to SQL
Import Enkora courses with a single command:
python manage.py event_import enkora --coursesRunning this imports all active courses from ERP into the database.
- Enkora location is mapped to TPrek place
- Enkora keyword is mapped to audience keywords
- Enkora description is mapped to keywords and audience keywords
A MarkDown-document how mapping works can be auto-generated.
Importer code is available in commit: https://github.com/City-of-Helsinki/linkedevents/blob/a1857fea87eb754f8eaf92b8d3a47634a6b69f4d/events/importer/helmet.py
helmet is an importer for Helsinki Metropolitan Area Libraries Events from https://helmet.fi.
Basic example would be an event organized by Helsinki city center library for children, which you can import with this importer.
If you want to populate the database with events organized by Helmet, you need helmet importer.
Import all events organized by Helmet into the database.
python manage.py event_import helmet --eventsThis is scheduled to run hourly on the instance as data changes often.
Importer code is available in commit: https://github.com/City-of-Helsinki/linkedevents/blob/c1d5fef030a2a1bfbc570c99cf516d878ee46fe7/events/importer/harrastushaku.py
Harrastushaku is an importer which imports courses from Harrastushaku website.
If you'd like to have courses from Harrastushaku website, you need this importer.
First, you need to import Harrastushaku locations:
python manage.py event_import harrastushaku --placesThen, you can import Harrastushaku courses:
python manage.py event_import harrastushaku --coursesImports all courses from Harrastushaku website into the database.