-
Notifications
You must be signed in to change notification settings - Fork 10
Decision: Display of rules and NPRMs in sidebar
Thing | Info |
---|---|
Relevant features | Supplemental content sidebar |
Date started | 2021-10-18 |
Date finished | 2022-01 |
Decision status | Done |
Summary of outcome | We decided to implement grouped FR docs in sidebars, and we did! |
When you need to interpret or update a piece of regulation, you often need to look up the final rules and proposed rules (NPRMs) that contributed to that piece, because they include essential background information about CMCS regulatory decisions, particularly in the rule preambles. It can be very tedious to track down whole sequences of NPRMs and final rules.
We want to display relevant final rules in the sidebar subpart-by-subpart (and ideally also section-by-section), and we want those final rules to be connected to their relevant NPRMs. See our supplemental content sidebar prototype, under "Rules with Preambles". Our users have been very excited about this prototype.
Our users asked us to include RFIs in eRegs too, since they can also be helpful for understanding CMCS decisions.
How can we do this?
How far can we get to that goal in an automated fashion? What elements might need to be done by hand?
What should we do as helpful intermediate steps in iterative development toward this goal?
Each rule and NPRM includes unstructured information about which sections it amends. For example, for this proposed rule, it says:
- In the Preamble: "we are proposing to amend § 447.502"
- In the end part where it describes the specific proposed change: "Amend § 447.502 by revising the definitions of “States” and “United States” to read as follows"
Each rule and NPRM also includes a structured list of the parts it is related to. For example, for that same rule, the right sidebar includes: 42 CFR 433
42 CFR 438
42 CFR 447
42 CFR 456
. The Federal Register API includes this structured data for documents (cfr_references
).
RFIs often mention some sections or parts of the regulations but may not suggest any specific amendments to a section or part. The FR structured data for an RFI may say "42 CFR chapter undef" - for example, see this January 2020 RFI and this May 2020 RFI. However, the second RFI mentions 431.52 in the unstructured (handwritten) text, and it could be helpful to readers researching 431.52.
The FR API only has data going back to 1994. Documents from the 1990s may have incomplete information in the Federal Register API, either plain text or scans without OCR - for example, see this 1994 rule available in plain text.
There are many older FR documents that are still important to our readers. For example, 42 CFR § 456.2 has the following footnote indicating that its sources are all rules from the 1980s: [46 FR 48566, Oct. 1, 1981, as amended at 50 FR 15327, Apr. 17, 1985; 51 FR 43198, Dec. 1, 1986]. The best that the Federal Register website can do in an automated way is provide a link to a not-very-readable yellowish scan. This system has some errors - it doesn't always fetch a relevant PDF.
The Library of Congress also has PDFs of old rules available, typically in better scans like this one. When we tested this aspect of cross-reference linking, our readers preferred the LoC PDFs that Rebecca had selected by hand for the older rules. From our study summary:
Both options for older rules had issues: page load timeouts with the Library of Congress (LOC) scans vs Federal Register’s less readable scans. Participants vastly preferred being linked to the Library of Congress PDFs for older rules instead of the Federal Register citation url, even though they ran the risk of page load timeouts.
CFR text includes semi-structured data about its relationship to FR documents:
-
Each part has a "Source" (FR document) statement at the beginning.
- Example: Part 438 says "Source: 67 FR 41095, June 14, 2002, unless otherwise noted."
-
Some subparts: have a "Source" statement at the beginning.
- Example: Part 438 Subpart A says "Source: 81 FR 27853, May 6, 2016, unless otherwise noted."
-
Most sections have footnotes that tell which FR documents established that section and updated it.
- Example: Part 438, Subpart A, Section 438.4 says: "[81 FR 27853, May 6, 2016, as amended at 85 FR 72837, Nov. 13, 2020; 85 FR 72837, Nov. 13, 2020]"
Each FR document has a structured "Agency/Docket Number". We may be able to use this docket number to automatically group Final Rules and NPRMs.
- For example, for that same May 2021 proposed rule, the number is
CMS-2482-P2
. This number is available via the FR API (docket_ids
). - That proposed rule says that it is proposing to delay the implementation of a December 2020 final rule (85 FR 87000) that has the docket ID of
CMS-2482-F
. It has the same list of related parts:42 CFR 433
42 CFR 438
42 CFR 447
42 CFR 456
. - That final rule references a June 2020 proposed rule with the number of 85 FR 37256, but the FR site links it to an unrelated rule because there appears to be a typo (?!?). The actual proposed rule it should reference is 85 FR 37286, which has a docket ID of
CMS-2482-P
. Also same list of related parts:42 CFR 433
42 CFR 438
42 CFR 447
42 CFR 456
.
We're not sure how consistent CMS has been over the last few decades about using these docket numbers in a structured way consistently.
Also, as that example shows, there is not a simple 1:1 relationship between a Final Rule and an NPRM. It's messy:
- A Final Rule can be related to a later followup NPRM
- A Final Rule can be related to multiple earlier NPRMs
- A NPRM can be related to multiple earlier Final Rules
- There can be a NPRM that never has a Final Rule
- There can be an Interim Final Rule that didn't have an NPRM (although rare)
So far, we've used the FR API to create lists of documents relevant to specific parts (rules, NPRMs, and RFIs). We display them in our subpart sidebars and on our homepage.
- Can we collect up the semi-structured FR citations in the CFR text and use them to create section-by-section and subpart-by-subpart lists of related final rules? (Can we automate that reliably?)
- Can we use the docket numbers to group together related NPRMs and Final Rules? (Can we automate that reliably? If we can't automate that reliably, can we build a way for a program to do a rough pass that a person can then correct in our admin UI?)
This is a question both from a UX perspective (how should it behave?) and a data perspective: should there be some way for a content author to input additional FR documents by hand that can be aggregated with the automated data?
This is also a question about adding manual data to enrich the automated data.
People have asked for a multi-faceted search that would cover reg text and the titles of supplemental content - should the FR documents be in there too?
The Federal Register API includes a way to search the FR. We would need to review the fields available for filtering and determine what fields we want to filter on.
Stories that exist:
Please note that all pages on this GitHub wiki are draft working documents, not complete or polished.
Our software team puts non-sensitive technical documentation on this wiki to help us maintain a shared understanding of our work, including what we've done and why. As an open source project, this documentation is public in case anything in here is helpful to other teams, including anyone who may be interested in reusing our code for other projects.
For context, see the HHS Open Source Software plan (2016) and CMS Technical Reference Architecture section about Open Source Software, including Business Rule BR-OSS-13: "CMS-Released OSS Code Must Include Documentation Accessible to the Open Source Community".
For CMS staff and contractors: internal documentation on Enterprise Confluence (requires login).
- Federal policy structured data options
- Regulations
- Resources
- Statute
- Citation formats
- Export data
- Site homepage
- Content authoring
- Search
- Timeline
- Not built
- 2021
- Reg content sources
- Default content view
- System last updated behavior
- Paragraph indenting
- Content authoring workflow
- Browser support
- Focus in left nav submenu
- Multiple content views
- Content review workflow
- Wayfinding while reading content
- Display of rules and NPRMs in sidebar
- Empty states for supplemental content
- 2022
- 2023
- 2024
- Medicaid and CHIP regulations user experience
- Initial pilot research outline
- Comparative analysis
- Statute research
- Usability study SOP
- 2021
- 2022
- 2023-2024: 🔒 Dovetail (requires login)
- 🔒 Overview (requires login)
- Authentication and authorization
- Frontend caching
- Validation checklist
- Search
- Security tools
- Tests and linting
- Archive