How can I get feedback on my quiz answers?

[angelikatyborska]

How can I verify that my answers to the quiz questions are correct?

The README says this will happen automatically on GitLab CI:

elixir-secure-coding/README.md

Lines 87 to 89 in 733c41c

	- You will receive feedback as to whether you completed it or not in the CI stage of your version control system<sup>*</sup>
	*<sup>\*</sup>Feedback functionality will initially work in GitLab, GitHub support is [in the works](https://github.com/podium/elixir-secure-coding/issues/17) but lower priority*

But:

• I am not using GitLab
• I don't see any GitLab CI configuration file in this repository that would allow me to at least look up what kind of commands are running there

There's a grader.exs script that says to run mix run grader.exs:

elixir-secure-coding/grader.exs

Line 1 in 733c41c

# Usage: mix run grader.exs

But there's no mix.exs file in this repository, so I get this error:

$ mix run grader.exs 
** (Mix) Cannot execute "mix run" without a Mix.Project, please ensure you are running Mix in a directory with a mix.exs file or pass the --no-mix-exs option

When I run mix run grader.exs --no-mix-exs or elixir grader.exs, I get this error:

$ mix run grader.exs --no-mix-exs
Evaluating: modules/1-introduction.livemd
** (MatchError) no match of right hand side value: {:error, {:undef, [{Livebook.LiveMarkdown, :notebook_from_livemd, ["# ESCT: Part 1 - Introduction\n\n## Overview\n\nFound in this series of modules is a curriculum for teaching Secure Coding concepts and ideas centered around the Elixir ecosystem. Core principles of Application Security have been sourced from other available resources within the community and pieced together into this Elixir Livebook format to allow for an interactive spin.\n\nIt is worth stating that this material is a work in progress and is open to contributions in order to make this the one-stop shop for Developer Secure Coding Training (for Elixir). The initial training material was originally crafted by the Product Security team at [Podium](https://www.podium.com/) and as such, contains very opinionated lessons to help contribute to the Secure SDLC of Podium's engineers. The more general this material can be made through outside contributors, the more secure we can make the Elixir community.\n\n## Who This Is For\n\nThis curriculum is for any Software Developer / Engineer / Maker / Hacker looking to better their own knowledge of the Web Application Security space, especially as it pertains to Elixir Phoenix applications.\n\nThis Training material is also ideally used in an educational environment for organizations to level up their Engineering teams Security knowledge. Quiz questions have been crafted within and an auto-grader that can be deployed in the CI/CD pipeline for local forks of this repo will be made available soon.\n\n## How To Use This Livebook\n\n### Livebooks\n\nIf you've never used an Elixir [Livebook](https://livebook.dev/) before, you're in for a treat! They are a very exciting new tool that is actively under development - very similar in application to [Jupyter Notebooks](https://jupyter.org/), but for the Elixir ecosystem!\n\nIt would not do the Livebook any justice to try and summize here how to fully take advantage of all its capabilities, for a better introduction there is a great tutorial offered in local installations of Livebook.\n\n**For the purposes of this Training material, just know that you need to run the \"Setup\" step for the \"Notebook dependencies and setup\" section at the very top of EVERY module before running any code samples found within the module you're working on.**\n\n### Examples & Quiz Questions\n\nSpread throughout the Training material, you will find sections labeled <span style=\"color:blue;\">**Example**</span> and <span style=\"color:red;\">**Quiz**</span>. The idea here is those are relevant (and runnable) code examples in Elixir for the section you're learning about. <span style=\"color:blue;\">**Examples**</span> are just for your education, whereas there will be graded component to <span style=\"color:red;\">**Quiz**</span> questions.\n\n**Don't worry!** If you've done the reading for the associated section, you should breeze through it and each question will outline what specifically needs to be done to successfully complete it! Here's an example <span style=\"color:red;\">**Quiz**</span> question layout:\n\n### <span style=\"color:red;\">**Quiz**</span>\n\n**This is what the question prompt would look like!**\n\n*This is the description on what the auto-grader is looking for in order to pass the question successfully*\n\n### Auto-grader\n\nMeticulous care has been put into the <span style=\"color:red;\">**Quiz**</span> questions thus far in order to allow for programmatic grading of answers. This has been done to accommodate the usage of these Training materials en masse for organizations to level up the entirety of their Engineering teams.\n\nAs such, each <span style=\"color:red;\">**Quiz**</span> question is very specific about what to change and what not to change in the code sample - this is to maintain the integrity of the grader and provide immediate feedback to the taker if they succeeded or not. **Please do not unnecessarily change the code examples more than what is asked of you in the question!**\n\n## Training Modules\n\n1. Introduction (You Are Here)\n2. [OWASP](./2-owasp.livemd) - ~40 minutes\n3. [Secure SDLC Concepts](./3-ssdlc.livemd) - ~15 minutes\n4. [GraphQL Security](./4-graphql.livemd) - ~15 minutes\n5. " <> ...], []}, {Grader.Client, :init, 1, [file: 'grader.exs', line: 24]}, {:gen_server, :init_it, 2, [file: 'gen_server.erl', line: 851]}, {:gen_server, :init_it, 6, [file: 'gen_server.erl', line: 814]}, {:proc_lib, :init_p_do_apply, 3, [file: 'proc_lib.erl', line: 240]}]}}
    grader.exs:9: Grader.Client.run_and_save/1
    (elixir 1.14.1) lib/enum.ex:975: Enum."-each/2-lists^foreach/1-0-"/2
    grader.exs:76: Grader.App.main/0

What am I doing wrong?

[houllette]

Hey @angelikatyborska! Sorry for the confusion, we are still working on implementing / optimizing the grader mechanism - getting that finalized and pushed out publicly is our goal for the 1.0 release.

Under the hood the grader is anticipated to work by executing the code blocks within the livemd files, saving the outputs and comparing them against an answer key. We're looking into making this a more fluid and editable process, which is part of what is causing the delay in releasing the functionality.

As an aside, we will keep the following related GitHub issues updated with progress:

• Autograder as GitHub Action #17
• Autograder as GitLab CI job #18