Skip to content

v1.0.0
Compare
Choose a tag to compare
@gordonje gordonje released this 27 May 18:35
· 609 commits to main since this release
v1.0.0
e09a5fe
  • Enhanced resume behavior
    • Allow previously interrupted updates to resume at any stage of the process: downloading, cleaning or loading.
    • Users will be prompted to resume (if possible). User may decline and re-start the entire update.
    • Removed --resume-download option from updatecalaccessrawdata and downloadcalaccessrawdata in favor of prompting the user to resume.
    • Removed --database option from all commands. Multi-database users are encouraged to use Django's database routers.
  • Raw data file archiving
    • Added CALACCESS_STORE_ARCHIVE setting. When enabled, management commands will save each version of the downloaded .zip file, the extracted .tsv files and cleaned .csv files to the Django project's MEDIA_ROOT.
    • Added FileFields to RawDataVersion and RawDataFile in order to link the database records with the archived files they reference.
  • Completed documentation of all 80 raw data models and 1,467 fields
    • Defined hundreds of choices for 182 look-up fields.
    • Published expanded Django project documentation. Added re-directs from old app-specific documentation.
    • Integrated references to official documents and filing forms into data models. PDFs on DocumentCloud.
  • Expanded unit testing of data model documentation
    • Wider scope of choice field testing.
    • Verify that each model has a UNIQUE_KEY attribute set.
    • Verify that each model has a document reference.
    • Verify that each choice field has a document reference.
    • Verify that each model with a form_type or form_id field (with a few exceptions) is linked to filing forms.
    • Introduced reportcalaccessrawdata command, which generates a report outlining the number / proportion of files / records cleaned and loaded.
  • Model Re-modeling:
    • Moved BallotMeasuresCd from other.py to campaign.py. Same with admin.
    • Moved remaining models in other.py to common.py. Removed other.py. Same with admins.
    • Re-ordered models into related groups.
  • Bug fixes
    • Truncate time portions of raw datetime values #1457.
    • Strip newlines when loading into MySQL.
Assets 2
Loading