Table of Contents
The historical concept of democracy, liberry and freedom and its development through time is important to our understanding of the past. The goal of the HGEAR project is to chart shifts in the character and structure of political discourse during the era of the American Revolution. By comparing shifts in language usage (via ShiCo (Shifting Concepts Through Time) with changing formations in correspondence networks, we are able to measure how networks of communication within the Founders Online corpus and other resources such as the Letters of Delegates to Congress shaped the character, distribution, and spread of political ideas during the Revolutionary era. This project and associated analyses offer powerful new methods for addressing longstanding debates in the field of early American history. Also, by means of additional collected data, we show how the here included scripts and analyses can be easily accomodated to include other letter correspondence sources as well. Ultimately, this allows researchers to get a more complete picture of the fluxes in the political discourse during the American Revolution.
The project is organized as follows:
Here's the link to learn more about the project's background.
├── LICENSE
├── README.md <- The top-level README for developers and users using this project.
├── data
│ ├── external <- Data from third party sources.
│ ├── interim <- Intermediate data that has been transformed.
│ ├── processed <- The final, canonical data sets for modeling.
│ └── raw <- The original, immutable data dump.
│
├── w2v_models <- word2vec models to be used with shico software.
│ ├── delegates <- Delegates of Congress.
│ ├── ecco <- Eighteenth Century Collections Online.
│ ├── evans <- Evans Early American Imprints.
│ └── founders <- Founders Online.
│
├── documentation <- Data dictionaries, manuals, and all other explanatory materials.
│
├── output <- Generated analysis as HTML, PDF, LaTeX, etc.
│ └── figures <- Generated graphics and figures to be used in reporting.
│ ├── logs <- Place for logs.
│ ├── tables <- Location for documents describing results of data exploration, including tables.
│
├── requirements.txt <- The requirements file for reproducing the analysis environment
│
├── scripts <- Source code for use in this project.
│ ├── 1. data_collection <- Including scraping data.
│ ├── 2. data_preparation <- Data cleaning/preprocessing to enhance the data quality by organizing raw data in a suitable format.
│ ├── 3. exploratory_data_analysis <- Getting a first and deeper understanding of the data.
│ └── 4. analysis_and_modeling <- Data analysis and modeling.
│ └── 5. deployment <- Geographical analysis.
│ └── functions <- Applied accross the project and sourced in the code.
│
├── notebooks <- Example of analysis)
These instructions will get you a copy of the project up and running on your local machine for development and testing purposes.
To install hgear from this GitHub repository, do:
git clone [email protected]:https://github.com/h-gear/revolution.gitHere, we give a step by step instructions that tell you how to get you up and running. The code is written mostly in R and parts in Python. For R , RStudio is recommended, while for Python, we recommend anaconda.
To ensure a consistent and reproducible environment, this project uses the venv package in R. Follow these steps to set up the virtual environment and install the necessary packages:
Navigate to the Project Directory:
cd revolution
Run the Setup Script:
Execute the provided setup script (setup.bat) to automatically create and activate the virtual environment, and install the necessary packages:
setup.bat
This script will install the renv package, create and activate the virtual environment, and install the required R packages specified in renv.lock.
By running the provided setup script, Windows users can easily set up the required environment without worrying about manually installing packages. This approach streamlines the process and ensures that users have the correct dependencies in place.
Note: Ensure that the script is executed with administrative privileges. If prompted, right-click on setup.bat and choose "Run as administrator."
Run Your Code:
After the setup script completes, you are ready to run the code within the virtual environment. Execute your R scripts or run your Rmarkdown documents as usual.
Deactivate the Virtual Environment:
When you are done, deactivate the virtual environment:
venv::deactivate()
We recommend installing hgear in a new virtual environment to avoid dependency conflicts.
Run the following commands to create a virtual environment: For Windows:
python -m venv ./.venv
For Unix or MacOS:
python3 -m venv ./.venv
Activate the Virtual Environment: For Windows:
.venv\Scripts\activate.bat
For Unix or MacOS:
source .venv/bin/activate
Install Dependencies: Once the virtual environment is created and activated you can install the dependencies by running:
pip install -r requirements.txt
At this stage, you should be able to run the scripts.
Deactivate the Virtual Environment: When you are done, deactivate the virtual environment:
deactivate
If you are interested in learning more about the data used in the project, then go here.
The code for the analyses can be found in the subfolder scripts. It is structured in parallel to the structure explained above:
- 1_data_collection
- 2_data_preparation
- 3_exploratory_data_analysis
- 4_analysis_and_modeling
- 5_deployment
- functions (applied accross the project and sourced in the code)
In adition, the code and procedures to create the word2vec models needed as input for running shico can be found in the subfolder w2v_models/delegates. The code is applied to the data from the delegates of congress, but can be easily adapted and eqaully applied to create word2vec models for all other datasets in this project as well.
If you want to contribute to the development of hgear, have a look at the contribution guidelines.Please submit pull requests to us in case you want to participate. Suggestions, improvements, and edits are most welcome!
This project is licensed under the Apache License 2.0 - see the licence file for details. This means that this project can be used, modified and redistributed for free, even for commercial purposes.
- Jay Timm: https://github.com/jaytimm/founders-online-corpus
- People of the founding era: https://pfe.upress.virginia.edu/
- Thomas Hoekstra: https://github.com/Rohrym/letters_of_the_revolution_project
- Shico: https://github.com/NLeSC/ShiCo
- Shico deployment: https://github.com/c-martinez/ShiCo-deploy
- Shico updated for Python 3: https://github.com/h-gear/ShiCo/
If you find any bugs or experience difficulties when using the code in this project, please create a issue on this Github page. If you have any specific questions with respect to our research, or methodology used, please email Thijs Vroegh [email protected]
This code has been used in the upcoming research article:
- Vroegh, T., Tjong Kim Sang, E.,Olthof, J., & Thompson, M.L. (in progress). Drivers and dynamics of liberal and republican thought: Social Networks in the American Revolution.
Please cite if you use it. You can do this easily with one of these options:
- The
cite this repositorymenu in the right tab. - The citation file.
- If you prefer to copypaste, here you have the APA and BibTex strings:
Vroegh, T. h-gear [Computer software]. https://github.com/h-gear/revolution
@software{Vroegh_hgear,author = {Vroegh, Thijs},license = {Apache 2.0},title = {{h-gear}},url = {https://github.com/h-gear/revolution}}
This code is being developed by the Netherlands eScience Center in collaboration with the faculty of American Studies at the University of Groningen. Our thanks go out to the authors that shared or open-sourced their code, enabling us to create this project that can hopefully be of service to many. Particularly, we like to thank the following people:
- Cristian Marocico
- Thomas Hoekstra
- Ramona Roller
🔼 Back to top