Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
meezlung authored Dec 18, 2024
1 parent fe64403 commit 9d8efd1
Showing 1 changed file with 33 additions and 8 deletions.
41 changes: 33 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,32 @@ courseURLS sample input: ```https://crs.upd.edu.ph/preenlistment/class_search/56

<br />

## Simple Testing
1. Clone the repository
```
git clone https://github.com/meezlung/CRScraper.git
cd CRScraper/
```
2. Search for your preferred courses in CRS and copy paste the input as follows:
```
all_course_table_schedule_url_cs_test = ["https://crs.upd.edu.ph/preenlistment/class_search/19405",
"https://crs.upd.edu.ph/preenlistment/class_search/19398",
"https://crs.upd.edu.ph/preenlistment/class_search/19403",
"https://crs.upd.edu.ph/preenlistment/class_search/19404",
"https://crs.upd.edu.ph/preenlistment/class_search/19480",
] # Sample format for CS 136, CS 21, CS 33, CS 132, and Eng 30
# Note: Each URL corresponds to a search result table of a DESIRED COURSE,
# obtained after searching for a class in the CRS search bar. You may edit this list as you please.
```
4. Open ```test.py``` in a text editor, and modify the ```all_course_table_schedule_url_cs_test``` variable.
5. Feel free to edit the ```filename``` variable as well.
6. Run ```test.py``` in the terminal.
```
python test.py
```
7. The CSV output will be in the same directory as ```test.py```.
## Use the App through Docker
1. Download [Docker](https://docs.docker.com/desktop/) (if you don't have one yet).
2. Clone the repository
Expand Down Expand Up @@ -55,8 +81,8 @@ https://github.com/user-attachments/assets/8c4a7ae4-fd49-46dd-9710-ea5d16769eba
### crs_scraper.py
- This just scrapes everything from the CRS website then outputs a data in the form of **list[dict[str, str | list[str]]]**.
- I used this last Midyear CRS enlistment (2023) and outputted the data as ```data.json``` and ```data.txt```.
- They are raw data, so I really had to use ```data_sorter.py``` to sort important property data. What I'm thinking now is that I should really optimize this by sorting and organizing properties while scraping the website (see ```optimized_crscraper.py```).
- I used this last Midyear CRS enlistment (2023) and outputted the data as ```raw_data_CS_2ndYear_1stSem_AY_2024-2025.json``` and ```raw_data_CS_2ndYear_1stSem_AY_2024-2025.txt```.
- They are raw data, so I had to use ```data_sorter.py``` to sort important property data. What I'm thinking now is that I should really optimize this by sorting and organizing properties while scraping the website (see ```optimized_crscraper.py```).
### crs_data.py (for debugging purposes only)
Expand All @@ -79,19 +105,18 @@ https://github.com/user-attachments/assets/8c4a7ae4-fd49-46dd-9710-ea5d16769eba
### crs_main.py
- This controls everything, including crs_data.py, data_sorter.py, crs_scraper.py
- This will also serve as the main backend file for the Svelte frontend.
- This will also serve as the main backend file for the Svelte frontend via Flask.
<br />
# Mga Kulang Pa (Pero ewan ko kung gagawin ko pa to):
- Ranking system based on [Rate UP Profs (RUPP)](https://rupp.onrender.com/) or Restrictions/Remarks (Still don't know how to implement)
- Ranking system based on [Rate UP Profs (RUPP)](https://rupp.onrender.com/) or Restrictions/Remarks. **(Still don't know how to implement)**
- There should be a ```crscraper.py``` authenticator for each input of username, password, and courseURL, depending how the CRS website responded to those input.
- If kaya, host sana 'yung app sa internet.
- Mas epic din kung iupload sa Google Extension so it's accessible by many.
- I still need a way to host my backend, frontend online.
- JavaScript scraper in the future so we can upload to Google Extensions?!
- If kaya, host sana 'yung app sa internet. I still need a way to host my backend and frontend online.
- I think I need to install Linux server on a laptop then try to host from there.
- I think we need to scrape Preenlistment Priority as well to feed it into the Course Probability Calculator
- Also add criterias for organizing schedules (e.g. no weekend classes, no classes after 4 pm, no class start before 9 per se).
- Also add criterias for organizing schedules (e.g. no weekend classes, no classes after 4 pm, no class start before 9 per se). **(Still don't know how to implement)**

0 comments on commit 9d8efd1

Please sign in to comment.