Skip to content

Latest commit

 

History

History
91 lines (50 loc) · 5.79 KB

review-rubrics.md

File metadata and controls

91 lines (50 loc) · 5.79 KB
Raw

OLS Application review rubric for selection of applications

This document describes the rubric our mentors use to review all the applications. Our applicants may have a look at this to ensure that they are aware of the criteria their applications will be assessed against.

Rubric for Reviewers

Review of all applications will be scored between 1 and 3 for 'Readiness' for the program, 'Goals' of the projects, 'Openness' in the context of Open Science and 'Willingness' to learn. Applicants who mostly score 1s are not ready for the program and may not understand the work, the overall goals of Open Science and openness in general, how openness promotes community engagement, or how to be while learning in the community or through mentorship. Applicants who score 2s are enthusiastic if not wholly ready for the work, have a personal or professional goal to be more open (in general) and are ready to learn about Open Science from experts, mentors, and peers alike. Applicants who score 3s are ready, goal-oriented, interested in applying Open Science to their work, and excited to learn from experts, mentors, and peers alike. Please check 'I understand' to proceed.

  • I understand

Project title as provided by the applicants.

[Open text]

Scores between 1 and 3:

  1. Applicants who score 1s are not ready for the program and may not understand the work, the overall goals of Open Science and openness in general, how openness promotes community engagement, or how to be while learning in the community or through mentorship.

  2. Applicants who score 2s are enthusiastic if not wholly ready for the work, have a personal or professional goal to be more open (in general) and are ready to learn about Open Science from experts, mentors, and peers alike.

  3. Applicants who score 3s are clearly ready, goal-oriented, interested in applying Open Science to their work, and excited to learn from experts, mentors, and peers alike.

Criteria

Readiness for the program

Are the candidate and their project ready for the OLS training and mentoring?

  1. (not ready) Does not provide enough information or seems to misunderstand the nature of the program or of Open Science, in general

  2. (enthusiastic) Seems to have a clear understanding of the program and brings along a specific project or idea for one

  3. (clear) Seems to have a clear understanding of the program and a clear understanding of how it might help open their work in academia

Goals

Has the candidate clearly defined some goals in their project and from their participation in OLS?

  1. (not ready) Shares vague or general goals, goals that are unrelated to Open Science, or no goals at all

  2. (enthusiastic) Shares clear, overly ambitious goals related to Open Science that can likely be managed and revised during the program

  3. (clear) Shares clear, achievable goals related to Open Science that fit the program and are likely to be met through the applicant’s participation.

Openness and Open Science

Is the candidate keen to learn and apply open science practices, both technically and through the involvement from an open community?

  1. (not ready) Goals and purposes for the program seem almost entirely self-centered and about the applicant’s status, rather than about opening their community or project.

  2. (enthusiastic) Goals and purposes for the program address openness, but in a general or limited way, like creating an openly licensed product instead of building an open community of contribution

  3. (clear) Goals and purposes for the program address openness in many ways and are likely to help the applicant empower others to work open and become open leaders in their own rights.

Willingness to learn

Is the candidate willing to practice open science in and beyond their work in OLS?

  1. (not ready) Seems closed to new ways of working or more interested in open-washing a publication or project than in learning about working Open Science

  2. (enthusiastic) Seems excited to learn about working open in science, but in a general way without much understanding of what those things mean yet

  3. (clear) Seems excited to learn about working open and joining the Open Science movement to join and advocate the Open Science movement

Community

Does the candidate have an understanding of working with an open science community? In their response have they shown their commitment to or understanding of Equity, Diversity, Inclusion and Accessibility (EDIA) aspects that they could draw from when working with diverse communities as OLS?

  1. (not ready) Seem not to have an understanding of what working with a diverse community in open science may entail
  2. (enthusiastic) Seems excited to learn about working with the community, but in a general way without much understanding of what considerations for EDI may involve
  3. (clear) Seems excited to learn about working with the community and bring a commitment to and/or understanding of EDIA concerns

Recommendation

Would you recommend this applicant and their project for their participation in the upcoming Open Seed OLS cohort?

  • Yes: I would like to recommend this application
  • No: I would not like to recommend this application
  • Unsure: I can not make an informed recommendation for this application

Mentoring preference

Would you like to mentor this project if they are the right match for you and if the candidate is accepted to participate in OLS?

  1. Top preference
  2. Second preference
  3. Third preference

Summary

Please provide a short sentence capturing the main theme of this project and insights from your review of this application. This will particularly help the OLS team and potential mentors choosing this application if you would recommend this application, but are not willing to mentor this applicant.

[Open Text]