This repository aims to get all of available matakuliah and dosen information from all of prodi and facutly in ITB for Dosen Rank Project. Built using python

I do not own the data. All of the data here are owned by ITB

1. argparse
2. bs4
3. requests

1. Clone this repository
2. create virtual environment from python virtualenv venv
3. install all requirements pip install -r requirements.txt
4. provide your nim and cookie from ITB sso service
   1. To get your cookie, you need to logged in in SIX
   2. Right click > inspect > application
   3. Grab the khongguan cookie from there

1. To get all cleaned data, first is run the scrape.py to get all of available data in SIX ITB
2. The data then get cleaned by using clean.py
3. Cleaned data then get converted to json or sql using output.py

.
├── README.md
├── clean.py 
├── data
│   ├── cleaned.json
│   ├── dosen_id_name.json   
│   ├── dosen_matkul_map.json
│   ├── fakultas.json        
│   ├── fakultas_shorthand.json
│   ├── id_prodi_map.json
│   ├── matkul_dosen_map.json
│   ├── matkul_id_name.json
│   ├── prodi.json
│   └── saved.json
├── output.py
├── requirements.txt
├── scrape.py
└── sql
    ├── dosen.sql
    ├── fakultas.sql
    ├── matkul.sql
    ├── matkul_dosen.sql
    └── prodi.sql