levinsohn_discourse_features.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "toc_visible": true,
      "authorship_tag": "ABX9TyOyUXelpcGhDp2IDuNlrMZL",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/ryderwishart/biblical-machine-learning/blob/main/levinsohn_discourse_features.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "This notebook queries a `macula-greek` API endpoint to gather all of the features available from Levinsohn's discourse features, and then it queries a specific label in a specified passage in order to return all of the instances of that feature from the passage."
      ],
      "metadata": {
        "id": "fmps5H8zIwiM"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import requests\n",
        "import json"
      ],
      "metadata": {
        "id": "k-GcNpRVJXx3"
      },
      "execution_count": 1,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "id": "yIqHDPiOAUKE"
      },
      "outputs": [],
      "source": [
        "ENDPOINT = 'https://macula-atlas-api-qa-25c5xl4maa-uk.a.run.app/graphql/'\n",
        "headers = {\"Content-Type\": \"application/json\"}"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Query API for all of the discourse-feature types"
      ],
      "metadata": {
        "id": "kuHtcX9QUAQ-"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "annotation_types_response = requests.post(ENDPOINT, data=json.dumps(\n",
        "    {\n",
        "    \"query\": \"\"\"\n",
        "        query {\n",
        "            annotationFeatures {\n",
        "                data\n",
        "            }\n",
        "        }\n",
        "    \"\"\",\n",
        "}\n",
        "), headers=headers)\n",
        "\n",
        "if annotation_types_response.status_code == 200:\n",
        "    types_raw = annotation_types_response.json()\n",
        "    annotation_types_response_size_kb = len(annotation_types_response.content) / 1024\n",
        "    print(f\"annotation_types_response size: {annotation_types_response_size_kb:.2f} KB\")\n",
        "else:\n",
        "    print(f\"Error: {annotation_types_response.status_code} - {annotation_types_response.text}\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8WKCnf-QQDk5",
        "outputId": "f7b77a49-323d-42b8-ff76-076b2b538e18"
      },
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "annotation_types_response size: 21.15 KB\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Inspect one of the discourse feature types data\n",
        "types_raw['data']['annotationFeatures'][1]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ReDACbSWSVIv",
        "outputId": "60f78f47-f224-4563-f56b-c1876e848552"
      },
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'data': {'name': 'Historical Perfect',\n",
              "  'type': {'name': 'markup'},\n",
              "  'license': {'url': 'https://github.com/biblicalhumanities/levinsohn/blob/master/LICENSE.md'},\n",
              "  'product': {'text': 'Levinsohn Greek New Testament Discourse Features',\n",
              "   'version': '1.0'},\n",
              "  'copyright': '©2016 SIL International',\n",
              "  'description': 'Highlights not the speech or act to which it refers but the event(s) that follow (DFNTG §12.2).',\n",
              "  'defaultstyle': {'label': '',\n",
              "   'bkcolor': '',\n",
              "   'brdrpos': 'A,U,B,E',\n",
              "   'newline': '',\n",
              "   'brdrline': 'Solid',\n",
              "   'brdrcolor': '#FF00FF00',\n",
              "   'fontcolor': '',\n",
              "   'fontstyle': 'None',\n",
              "   'indentlevel': '',\n",
              "   'newlineafter': '',\n",
              "   'indentlevelafter': ''}}}"
            ]
          },
          "metadata": {},
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Extract just the most useful information into a `types` dictionary \n",
        "types = {}\n",
        "for t in types_raw['data']['annotationFeatures']:\n",
        "    if t['data']:\n",
        "        types[t['data']['name']] = {\n",
        "            'description': t['data']['description'],\n",
        "            }\n",
        "\n",
        "# Print out all the discourse features types with their descriptions\n",
        "for t in types:\n",
        "    print(t + ':\\t', types[t])"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "L1GGMERsRBtW",
        "outputId": "3fce8839-d3b6-4c3c-9c1f-569d14c62152"
      },
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Historical Perfect:\t {'description': 'Highlights not the speech or act to which it refers but the event(s) that follow (DFNTG §12.2).'}\n",
            "Specific Circumstance:\t {'description': 'The function of ἐγενετο ‘it came about’ and an immediately following temporal expression varies with the author (see DFNTG §10.3). In Matthew’s Gospel, it usually marks major divisions in the book (e.g. Mt 7:28). In Luke-Acts, in contrast, ‘it picks out from the general background the specific circumstance for the foreground events that are to follow’ (ibid.), as in Acts 9:37 (see also Mt 9:10).'}\n",
            "Verb Focus+:\t {'description': 'Verb in final position in clause demonstrates verb focus.'}\n",
            "Articular Pronoun:\t {'description': 'Articular pronoun, which often introduces an ‘intermediate step’ in a reported conversation.'}\n",
            "Topical Genitive:\t {'description': 'A genitival constituent that is nominal is preposed within the noun phrase for two purposes: 1) to bring it into focus; 2) within a point of departure, to indicate that it is the genitive in particular which relates to a corresponding constituent of the context.(DFNTG §4.5)'}\n",
            "Embedded DFE:\t {'description': \"'Dominant focal elements' embedded within a constituent in P1.\"}\n",
            "Reported Speech:\t {'description': 'Reported speech.'}\n",
            "Ambiguous:\t {'description': 'Marked but ambiguous constituent order.'}\n",
            "Over-encoding:\t {'description': 'Any instance in which more encoding than the default is employed to refer to an active participant or prop. Over-encoding is used in Greek, as in other languages: to mark the beginning of a narrative unit (e.g. Mt 4:5); and to highlight the action or speech concerned (e.g. Mt 4:7).'}\n",
            "Highlighter:\t {'description': 'Presentatives - Interjections such as ἰδού and ἴδε ‘look!, see!’ typically highlight what immediately follows (Narr §5.4.2, NonNarr §7.7.3).'}\n",
            "Referential PoD:\t {'description': 'Pre-verbal topical subject other referential point of departure (NARR §3.1, NonNarr §4.3, DFNTG §§2.2, 2.8; as in 1 Th 1:6).'}\n",
            "annotations:\t {'description': 'Inline annotations.'}\n",
            "Left-Dislocation:\t {'description': 'Point of departure - A type of SENTENCE in which one of the CONSTITUENTS appears in INITIAL position and its CANONICAL position is filled by a PRONOUN or a full LEXICAL NOUN PHRASE with the same REFERENCE, e.g. John, I like him/the old chap.”'}\n",
            "Focus+:\t {'description': 'Constituents placed in P2 to give them focal prominence.'}\n",
            "Tail-Head linkage:\t {'description': 'Point of departure involving renewal - Tail-head linkage involves “the repetition in a subordinate clause, at the beginning (the ‘head’) of a new sentence, of at least the main verb of the previous sentence (the tail)” (Dooley & Levinsohn 2001:16).'}\n",
            "Postposed them subject:\t {'description': 'When a subject is postposed to the end of its clause (following nominals or adjuncts), it is marked ThS+ (e.g. Lk 1:41 [twice]). Such postposing typically marks as salient the participant who performs the next event in chronological sequence in the story (see Levinsohn 2014).'}\n",
            "EmbeddedRepSpeech:\t {'description': 'Embedded reported speech - speech that is reported within a reported speech.'}\n",
            "Futuristic Present:\t {'description': 'Highlights not the speech or act to which it refers but the event(s) that follow (DFNTG §12.2).'}\n",
            "OT quotes:\t {'description': 'Old Testament quotations.'}\n",
            "Constituent Negation:\t {'description': 'Negative pro-forms when they are in P2 indicate that the constituent has been negated rather than the clause as a whole.'}\n",
            "Split Focal:\t {'description': 'The second part of a focal constituent with only the first part in P2 (NonNarr §5.5, DFNTG §4.4).'}\n",
            "Right-Dislocated:\t {'description': 'Point of departure - A type of SENTENCE in which one of the CONSTITUENTS appears in FINAL position and its CANONICAL position is filled by a PRONOUN with the same REFERENCE, e.g. ... He’s always late, that chap.'}\n",
            "Appositive:\t {'description': 'Appositive'}\n",
            "Situational PoD:\t {'description': 'Situational point of departure (e.g. temporal, spatial, conditional―(NARR §3.1, NonNarr §4.3, DFNTG §§2.2, 2.8; as in 1 Th 3:4).'}\n",
            "Historical Present:\t {'description': 'Highlights not the speech or act to which it refers but the event(s) that follow (DFNTG §12.2).'}\n",
            "Noun Incorporation:\t {'description': 'Some nominal objects that appear to be in P2 may precede their verb because they have been “incorporated” (Rosen 1989) in the verb phrase. Typically, the phrase consists of an indefinite noun and a “light verb” such as “do, give, have, make, take” (Wikipedia entry on Light Verbs).'}\n",
            "Thematic Prominence:\t {'description': 'Thematic prominence - In Greek, prominence is given to active participants and props who are the current centre of attention (NARR §4.6) by omitting the article (DFNTG §§9.2.3-9.4), by adding αυτος ‘-self’ (e.g. in 1 Th 3:11), by using the proximal demonstrative οὗτος (NARR chap. 9, Appendix 1; e.g. in 3:3), and by postposing the constituent concerned (e.g. Mt 14:29). If such constituents are NOT in postion P1, they are demonstrating topical prominence.'}\n",
            "Cataphoric Focus:\t {'description': 'An expression that points forward to and highlights something which ‘is about to be expressed.’'}\n",
            "Cataphoric referent:\t {'description': 'The clause or sentence to which a cataphoric reference refers when NOT introduced with ὅτι or ἵνα.'}\n",
            "DFE:\t {'description': 'Constituents that may be moved from their default position to the end of a proposition to give them focal prominence include verbs, pronominals and objects that follow adjuncts (NonNarr §5.3, DFNTG §3.5). Such constituents, also called ‘dominant focal elements’or DFEs (Heimedinger 1999:167).'}\n",
            "Embedded Focus+:\t {'description': 'A constituent of a phrase or embedded clause preposed for focal prominence.'}\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Query the API for a specific discourse feature by label in a given book or passage"
      ],
      "metadata": {
        "id": "py20xnw-UElr"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "FEATURE = 'OT quotes'"
      ],
      "metadata": {
        "id": "Q-bleRe3UZII"
      },
      "execution_count": 13,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "PASSAGE = '2CO'"
      ],
      "metadata": {
        "id": "GxbKKtxxMkv7"
      },
      "execution_count": 15,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "query = \"\"\"\n",
        "query AnnotationFeatures($filters1: AnnotationFeatureFilter, $filters2: AnnotationFilter) {\n",
        "  annotationFeatures(filters: $filters1) {\n",
        "    label\n",
        "    uri\n",
        "    instances(filters: $filters2) {\n",
        "      uri\n",
        "      tokens {\n",
        "        ref\n",
        "        wordValue\n",
        "        xmlId\n",
        "        lemma\n",
        "        data\n",
        "      }\n",
        "    }\n",
        "  }\n",
        "}\n",
        "\"\"\"\n",
        "\n",
        "variables = {\n",
        "  \"filters1\": {\n",
        "    \"reference\": PASSAGE\n",
        "  },\n",
        "  \"filters2\": {\n",
        "    \"reference\": PASSAGE,\n",
        "    \"featureLabel\": FEATURE\n",
        "  }\n",
        "}\n",
        "\n",
        "payload = {\n",
        "    \"query\": query,\n",
        "    \"variables\": variables,\n",
        "}"
      ],
      "metadata": {
        "id": "OAuJ4qLnJU-J"
      },
      "execution_count": 25,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "response = requests.post(ENDPOINT, data=json.dumps(payload), headers=headers)\n",
        "\n",
        "if response.status_code == 200:\n",
        "    data = response.json()\n",
        "    # print(json.dumps(data, indent=2))\n",
        "    response_size_kb = len(response.content) / 1024\n",
        "    print(f\"Response size: {response_size_kb:.2f} KB\")\n",
        "else:\n",
        "    print(f\"Error: {response.status_code} - {response.text}\")\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "oGpeZkumJaVb",
        "outputId": "bb696948-3478-48ef-ec8d-c129f642e50d"
      },
      "execution_count": 26,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Response size: 56.95 KB\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Filter out empty feature sets from results\n",
        "results = [feature for feature in data['data']['annotationFeatures'] if len(feature['instances']) > 0]"
      ],
      "metadata": {
        "id": "WrBatddtV9Rp"
      },
      "execution_count": 28,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "for feature in results:\n",
        "    # print(feature['uri'])\n",
        "    \n",
        "    # Print any non-zero results\n",
        "    if len(feature['instances']) > 0:\n",
        "        print(feature['label'], '- Number of instances: ', len(feature['instances']))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "U0Azn89MJc1f",
        "outputId": "7b0c5709-67a4-499c-fd69-ddd5b19a43cb"
      },
      "execution_count": 29,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "OT quotes - Number of instances:  21\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Print out the first 5 instances of text\n",
        "for instance in results[0]['instances']:\n",
        "    tokens = instance['tokens']\n",
        "    first_ref = tokens[0]['ref']\n",
        "    last_ref = tokens[-1]['ref']\n",
        "    text = ' '.join([token['wordValue'] for token in tokens])\n",
        "    glosses = ' '.join([token['data']['gloss'] for token in tokens])\n",
        "    print(first_ref, '-', last_ref)\n",
        "    print(text)\n",
        "    print(glosses + '\\n')"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Z2TObwcJT5i5",
        "outputId": "b24b1ed7-ed67-438c-f21d-bde1702c7063"
      },
      "execution_count": 31,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "2CO 4:13!11 - 2CO 4:13!13\n",
            "Ἐπίστευσα διὸ ἐλάλησα\n",
            "I believed therefore I have spoken\n",
            "\n",
            "2CO 6:2!3 - 2CO 6:2!6\n",
            "Καιρῷ δεκτῷ ἐπήκουσά σου\n",
            "In a time acceptable I listened to you\n",
            "\n",
            "2CO 6:2!7 - 2CO 6:2!12\n",
            "καὶ ἐν ἡμέρᾳ σωτηρίας ἐβοήθησά σοι\n",
            "and in a day of salvation I helped you\n",
            "\n",
            "2CO 6:16!19 - 2CO 6:16!23\n",
            "Ἐνοικήσω ἐν αὐτοῖς καὶ ἐνπεριπατήσω\n",
            "I will dwell in them and will walk among [them]\n",
            "\n",
            "2CO 6:16!24 - 2CO 6:16!27\n",
            "καὶ ἔσομαι αὐτῶν Θεός\n",
            "and I will be their God\n",
            "\n",
            "2CO 6:16!28 - 2CO 6:16!32\n",
            "καὶ αὐτοὶ ἔσονταί μου λαός\n",
            "and they will be to Me a people\n",
            "\n",
            "2CO 6:17!2 - 2CO 6:17!5\n",
            "ἐξέλθατε ἐκ μέσου αὐτῶν\n",
            "come out from the midst of them\n",
            "\n",
            "2CO 6:17!6 - 2CO 6:17!7\n",
            "καὶ ἀφορίσθητε\n",
            "and be separate\n",
            "\n",
            "2CO 6:17!10 - 2CO 6:17!13\n",
            "καὶ ἀκαθάρτου μὴ ἅπτεσθε\n",
            "and unclean nothing touch\n",
            "\n",
            "2CO 6:17!14 - 2CO 6:17!16\n",
            "κἀγὼ εἰσδέξομαι ὑμᾶς\n",
            "and I will receive you\n",
            "\n",
            "2CO 6:18!2 - 2CO 6:18!2\n",
            "ἔσομαι\n",
            "I will be\n",
            "\n",
            "2CO 6:18!4 - 2CO 6:18!5\n",
            "εἰς Πατέρα\n",
            "for a father\n",
            "\n",
            "2CO 6:18!6 - 2CO 6:18!6\n",
            "καὶ\n",
            "and\n",
            "\n",
            "2CO 6:18!9 - 2CO 6:18!11\n",
            "μοι εἰς υἱοὺς\n",
            "to Me for sons\n",
            "\n",
            "2CO 8:15!3 - 2CO 8:15!7\n",
            "Ὁ τὸ πολὺ οὐκ ἐπλεόνασεν\n",
            "He the much not did abound\n",
            "\n",
            "2CO 8:15!8 - 2CO 8:15!13\n",
            "καὶ ὁ τὸ ὀλίγον οὐκ ἠλαττόνησεν\n",
            "and he the little not had lack\n",
            "\n",
            "2CO 9:9!3 - 2CO 9:9!6\n",
            "Ἐσκόρπισεν ἔδωκεν τοῖς πένησιν\n",
            "He has scattered abroad He has given to the poor\n",
            "\n",
            "2CO 9:9!7 - 2CO 9:9!13\n",
            "ἡ δικαιοσύνη αὐτοῦ μένει εἰς τὸν αἰῶνα\n",
            "the righteousness of Him abides to the age\n",
            "\n",
            "2CO 10:17!1 - 2CO 10:17!1\n",
            "Ὁ\n",
            "The [one]\n",
            "\n",
            "2CO 10:17!3 - 2CO 10:17!6\n",
            "καυχώμενος ἐν Κυρίῳ καυχάσθω\n",
            "boasting in [the] Lord let him boast\n",
            "\n",
            "2CO 13:1!6 - 2CO 13:1!14\n",
            "ἐπὶ στόματος δύο μαρτύρων καὶ τριῶν σταθήσεται πᾶν ῥῆμα\n",
            "In [the] mouth of two witnesses or three will be established every matter\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "FyZDL-aEF5vX"
      },
      "execution_count": null,
      "outputs": []
    }
  ]
}