Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simulate video learning events #5

Open
wants to merge 11 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# To get started with Dependabot version updates, you'll need to specify which
# package ecosystems to update and where the package manifests are located.
# Please see the documentation for all configuration options:
# https://docs.github.com/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file

version: 2
updates:
- package-ecosystem: "pip" # See documentation for possible values
directory: "/" # Location of package manifests
schedule:
interval: "daily"
open-pull-requests-limit: 2
45 changes: 45 additions & 0 deletions .github/workflows/simulate-events-daily.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
name: Simulate events (daily)

on:
schedule:
- cron: 59 11 * * *

jobs:
simulate_events:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Set up Python 3.10
uses: actions/setup-python@v3
with:
python-version: "3.11"

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install flake8
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi

- name: Lint with flake8
run: |
# stop the build if there are Python syntax errors or undefined names
flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
# exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics

- name: Simulate VideoLearningEvents
run: |
python simulate-video-learning-events.py

- name: Git Config
run: |
git config user.name 'Nya Ξlimu'
git config user.email '[email protected]'
- name: Git Commit
run: |
git add **/*.csv
git commit -m 'chore(ml): simulate events' --allow-empty
- name: Git Push
run: |
git push
Comment on lines +39 to +45
Copy link

@coderabbitai coderabbitai bot Sep 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix the shellcheck issue in the git add command.

To prevent file names with dashes from being interpreted as options, update the git add command to use ./*glob* or -- *glob* as suggested by shellcheck.

Apply this diff to fix the issue:

-git add **/*.csv
+git add -- **/*.csv
Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
- name: Git Commit
run: |
git add **/*.csv
git commit -m 'chore(ml): simulate events' --allow-empty
- name: Git Push
run: |
git push
- name: Git Commit
run: |
git add -- **/*.csv
git commit -m 'chore(ml): simulate events' --allow-empty
- name: Git Push
run: |
git push
Tools
actionlint

40-40: shellcheck reported issue in this script: SC2035:info:1:9: Use ./glob or -- glob so names with dashes won't become options

(shellcheck)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@coderabbitai How can we include shellcheck in our GitHub workflow files?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jo-elimu, you can include shellcheck in your GitHub workflow files by adding steps to install and run shellcheck on your shell scripts. Here's how you can modify your existing workflow in .github/workflows/simulate-events-daily.yml:

name: Simulate events (daily)

on:
  schedule:
    - cron: 59 11 * * *

jobs:
  simulate_events:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v4

    - name: Set up Python 3.11
      uses: actions/setup-python@v3
      with:
        python-version: "3.11"

    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install flake8
        if [ -f requirements.txt ]; then pip install -r requirements.txt; fi

    - name: Lint with flake8
      run: |
        # Stop the build if there are Python syntax errors or undefined names
        flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
        # Exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
        flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics

    - name: Install ShellCheck
      run: |
        sudo apt-get update
        sudo apt-get install -y shellcheck

    - name: Lint shell scripts with ShellCheck
      run: |
        shellcheck **/*.sh
        shellcheck **/*.bash
        shellcheck **/*.zsh
        shellcheck **/*.ksh

    - name: Simulate VideoLearningEvents
      run: |
        python simulate-video-learning-events.py

    - name: Git Config
      run: |
        git config user.name 'Nya Ξlimu'
        git config user.email '[email protected]'
    - name: Git Commit
      run: |
        git add -- **/*.csv
        git commit -m 'chore(ml): simulate events' --allow-empty
    - name: Git Push
      run: |
        git push

This adds two new steps:

  1. Install ShellCheck: Installs ShellCheck on the runner using apt-get.
  2. Lint shell scripts with ShellCheck: Runs ShellCheck on all your shell script files.

Alternatively, you can use a pre-made GitHub Action for ShellCheck. Here's how you can do it using ludeeus/action-shellcheck:

    - name: Lint shell scripts with ShellCheck
      uses: ludeeus/[email protected]
      with:
        path: ./

This action will automatically run ShellCheck on your shell scripts in the repository.

Let me know if you need any further assistance!

38 changes: 38 additions & 0 deletions .github/workflows/simulate-events.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
name: Simulate events

on:
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]

jobs:
simulate_events:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.9", "3.10", "3.11"]
steps:
- uses: actions/checkout@v4

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v3
with:
python-version: ${{ matrix.python-version }}

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install flake8
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi

- name: Lint with flake8
run: |
# stop the build if there are Python syntax errors or undefined names
flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
# exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics

- name: Simulate VideoLearningEvents
run: |
python simulate-video-learning-events.py
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
.venv
56 changes: 4 additions & 52 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,58 +1,10 @@
# DataGeneratorSimulator
It will generate usage and performance data from user.
# ML: Event Simulator

## Tutorial
*[Building a Data-Driven Education System](http://www2.datainnovation.org/2016-data-driven-education.pdf)

*[Enhancing Teaching and Learning Through Educational Data Mining and Learning Analytics](https://tech.ed.gov/wp-content/uploads/2014/03/edm-la-brief.pdf)


# Focus two major areas:
## Content Analytics

Are teachers using the best possible content? This level of data analysis takes a deeper dive to better inform the design of new course content and understand its impact (or lack thereof) on students.

## Learning Analytics

Game-based learning and adaptive learning systems are growing in use. This technology is designed to build statistical models of student knowledge, tracking their progress in order to personalize the learning experience.

## Requirement

pip install python-testdata


## Examples
We integrate the awsome fake-factory package to generate data using FakeData,
this allows us to usage data event:
* id
* deviceId
* studentId
* packageName
* literacySkill
* and much much more

lets create a very simple factory that generates Users:
## Simulate `VideoLearningEvent`s

```python
import testdata
import datetime

class Usage(testdata.DictFactory):
id = testdata.CountingFactory(10)
deviceId = testdata.CountingFactory(10)
studentId = testdata.CountingFactory(10)
packageName = testdata.RandomSelection(['Literacy', 'Game', 'Speech'])
literacySkill = testdata.RandomLengthStringFactory()
numeracySkill = testdata.RandomLengthStringFactory()
letter = testdata.RandomLengthStringFactory()
number = testdata.RandomLengthStringFactory()
word = testdata.RandomLengthStringFactory()

for usage in Usage().generate(10): # let say we only want 10 users
print usage
#{'numeracySkill': 'PwBiXKjwddG', 'studentId': 17, 'word': 'XctugWiHPobIvHNGEbYlgyOUuuqCSKgoTFAhJSQzUUleDEkygyZOWBnGYiLBXbywpwxAsisToqDDWGPHQqbOOlmGVVa', 'packageName': 'Literacy', 'number': 'IrtJUAxnFVOQyvvqlpIsmkaWnRvADBzWBiCYUPvfSwvdHS', 'literacySkill': 'hSQSRXUevpdYMGAs', 'deviceId': 17, 'letter': 'JYuWfonIdptbdpFhBNhLIkLoyhuUgRUvdiUWBcfReeezORAtXhJvNuLZASFeRCAvxvPgOeTZ', 'id': 17}
#{'numeracySkill': 'ozTpqAwdLstMzeijgJBGYMLantLSMESfYEBMQQxkjILBgNXohBjMbwqrhGsnjoSlcsCGOnTsdgMICQfB', 'studentId': 18, 'word': 'CnhxwMonHnMlEtxcpGowQymEeZtxvlUBDaKHEKRC', 'packageName': 'Literacy', 'number': 'xkerlJLhlyOgsTxHqMPffjPLOqbjgZqtggGzxPTkOleoZtEaDiYnpKxrouCcgRPjdtf', 'literacySkill': 'VlEeAOKKOIgweFTxBeNiOWmoztGPWSqhsIxTr', 'deviceId': 18, 'letter': 'NwJUuHLOkaJHsIvlSQeggfT', 'id': 18}
#{'numeracySkill': 'uaUQunGtHwrFTuRlVrhwEUisIWlcrZXUZKIlILoPoCgnVWHwrrRaHhxQJVnECUtSvppzQDtpiqUSds', 'studentId': 19, 'word': 'vOTlRRgSXwgmXAthOYnQTTtPJyGxGbbMOj', 'packageName': 'Game', 'number': 'bDmhALNhnmazlonmBIjvwWzXgQfPQQekWJErEvJjWWHrufxuINyHuNiLPvFWynVwdNTaTGIgvvGCAqFRZ', 'literacySkill': 'BpfiZyRAzovNbEhtznPSaqsaZhRkFHlWNpmbzBXKCmBJPnuYiQyEToMaOkVJOVZKNCCAyGpZSpGhfseBMfGaFvltHaJyfcdota', 'deviceId': 19, 'letter': 'nvwanqC', 'id': 19}
pip install -r requirements.txt
python simulate-video-learning-events.py
```

---
Expand Down
21 changes: 0 additions & 21 deletions dashboard/LICENSE.md

This file was deleted.

118 changes: 0 additions & 118 deletions dashboard/README.md

This file was deleted.

6 changes: 0 additions & 6 deletions dashboard/assets/css/animate.min.css

This file was deleted.

5 changes: 0 additions & 5 deletions dashboard/assets/css/bootstrap.min.css

This file was deleted.

61 changes: 0 additions & 61 deletions dashboard/assets/css/demo.css

This file was deleted.

Loading
Loading