Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rich Indexing #3113

Open
wants to merge 80 commits into
base: master
Choose a base branch
from
Open

Rich Indexing #3113

wants to merge 80 commits into from

Conversation

dbolack-ab
Copy link
Collaborator

@dbolack-ab dbolack-ab commented Oct 27, 2023

Semi-Dynamic Index building for Markdown+

Build simple, rich, or hybrid indexes with internal links.

Usage

Both usage cases center around the use of the index marked extension.

#IndexName:Topic/Subtopic

Multiple Indices

This system is capable of tracking multiple indices. Entries without a declared index in their formatting will be placed in the default index Index. An index marker is a block-level marker.

Simple Index

Anywhere in the document add an index marker with no label and a list of terms to be indexed. Terms will be searched for in a case and whitespace insensitive manner.

#Topic

Complex Index Entry

There are a number of variations that can be employed here. The only requirement is the line starts with a #, has no space, and contains a Topic - and Index and a Subtopic are optional.

This generates an entry for Fireball under the topic of Elemental Spells in the Appendix: Magic index.

#Appendix\: Magic:Elemental Spells|Fireball

This generates the topic of Elemental Spells in the Appendix: Magic index.

#Appendix\: Magic:Elemental Spells

This generates the topic of Elemental Spells in the default index.

#Elemental Spells

This generates an entry for Fireball under the topic of Elemental Spells in the default index.

#Elemental Spells|Fireball

This generates an entry for Fireball under the topic of Elemental Spells in the default index.

#Elemental Spells|Fireball

Crossreferences

This system supports crossreferencing index entries. Crossreference hyperlinks link to the crossreference target rather than a page number. Crossreference targets do not need to exist in the brew prior to being referenced; missing targets will result in the references being silently dropped on index generation.

Crossreferences are implemented by appending | followed by an optional key and then the index entry to reference.

Examples:

#Cross Index Name:CrossReference Topic/CrossReference Subtopic 1|Index Name:Topic 

#Cross Index Name:CrossReference Topic/CrossReference Subtopic 2|+Index Name:Topic 1/SubTopic 1

#Cross Index Name:CrossReference Topic 2|Index Name:Topic 2

#Cross Index Name:CrossReference Topic 2/CrossReference SubTopic 3||Index Name:Topic 2

#Cross Index Name:CrossReference Topic 2/CrossReference SubTopic 4||+Index Name:Topic 2/SubTopic 2

The optional keys are:

Key Reference Type
None See
+ See Under
| See Also
|+ See Also Under

The reference type is prepended to the target. The code does not enforce any particular style guide.

The above example would generate:

{{index,wide
##### Cross Index Name

- CrossReference Topic
  - CrossReference Subtopic 2
    see under [Topic 1: SubTopic 1](#idx_indexname_topic1_subtopic1)
- CrossReference Topic 2
    see [Topic 2](#idx_indexname_topic2)
  - CrossReference SubTopic 3
    see also [Topic 2](#idx_indexname_topic2)
  - CrossReference SubTopic 4
    see also under [Topic 2: SubTopic 2](#idx_indexname_topic2_subtopic2)

}}

Generating the Index

The rich index snippet, when executed, combines the simple and rich index entries into a single, alphabetized index.

If you have simple and rich entries, rich entries will be folded under the simple entries as follows:

  1. Simple entries are created as top-level, un-indented entries.
  2. Child entries are indented under their top-level parent.
  3. If a Rich entry has no parent, it will be inserted as a top-level entry.
  4. If a Rich entry has a parent and that parent does not exist in the top-level entries, the parent will be added as a new top-level entry and the Rich entry will be added as a child of that top-level entry.
  5. If a Rich entry has a parent and the parent exists in the top-level entries, and the Rich entry is not listed as a child of the parent, it will be added as a new child.
  6. If a Rich entry has a parent and the parent exists in the top-level entries, and the Rich entry exists, the child will be updated with an additional page reference to the new Rich entry.
  7. Results are poured into the brew inside a {{index,wide }} container. If muliple indices exist, they will appear in alphabetical order separated by a \page.

Known issues.

  • The snippet is not smart enough to know that it is scanning markup when looking for terms. This could create bad entries.
  • The Styling is meh. I mostly emulated the existing Index snippet. We could do some dot padding, etc.

Other notes

I'm not the most efficient JS coder. I'd be very unsurprised if some of this could be condensed.

Test Brew ShareID: https://homebrewery-pr-3113.herokuapp.com/share/vtI5G0Zl_fan

This was modeled off the pattern for links. This generates an HTML Anchor in the active paragraph.
The intent is to use a snippet to generate the final index by scanning all pages and ordering the index accordingly.

Format:

@Index Entry Name@(Index Parent Entry 1| Index Parent Enry 2|..)

Key:

Index Entry Name: This is the label in the index the reader will look up.
Index Parent Entry: This is an optional list of parent index entries that the anchor could be a child of.

Example:

@fireball@(Fire Spells|Ranged Spells)

This could/should output something like:

...
Fire Spells
   Fireball    p 16
...

...
Ranged Spells
   Fireball    p 16

Entries without parents shhould be treated as top level.

Notes and other lunacy:
   Any characters valid for links except @ are permitted.
   This may not be the best set of delimiters for this purpose. Open to suggestions.
   Does not currently prevent duplicate anchors.
   Anchor target values are created by removing whitespace and lowercasing the Index Entry Name.

To Do:
  Add a Page level anchor to the \page code
  Create a Snippet to generate the actual index
  Determine a sane way to do a light index ( list of words )
  Determine a sane way to merge both Index styles
Needs hyperlinks.
@ericscheid
Copy link
Collaborator

The simple index lookup assumes page-level anchor links in the format #page## ( e.g. #page1 ) that do not currently exist

Homebrewery does have page-level anchor's on the <div class="page"> elements, in the form of id="p1". These get used by the TOC snippet.

image

@dbolack-ab
Copy link
Collaborator Author

The simple index lookup assumes page-level anchor links in the format #page## ( e.g. #page1 ) that do not currently exist

Homebrewery does have page-level anchor's on the <div class="page"> elements, in the form of id="p1". These get used by the TOC snippet.

Oh. Wow. I just learned a new thing. I had no idea that the anchor behavior also applied to any ID'd tag. That's got to have been there for years?!

🤯

This changes the index page level links to use #p325 isntead of #page325
where the reference page is 325.

Thanks Eric!
@5e-Cleric
Copy link
Member

Yes, but due to how rendering works this is funky, Id for headers are not unique between pages, and Partial Page Rendering(PPR, which applies to 50+ pages brews) blocks them altogether. We are working on some fixes for that.

@dbolack-ab
Copy link
Collaborator Author

Yes, but due to how rendering works this is funky, Id for headers are not unique between pages, and Partial Page Rendering(PPR, which applies to 50+ pages brews) blocks them altogether. We are working on some fixes for that.

Fair, but as long as the ones for the pages are, it works for this purpose so long as someone doesn't manage to construct a collision between tags.

I considered concatenating the page number with the munged label ( i.e. p1-thisismylabel ) but I didn't see an elegant way to grab that...

@G-Ambatte
Copy link
Collaborator

If/when #3089 goes live, each Markdown token will be contained inside pageBlock tokens, which will have a pageNumber property, so it will all be accessible from the Markdown Lexer output.

@dbolack-ab
Copy link
Collaborator Author

I am unsure what is going wrong with this conflict

@calculuschild calculuschild temporarily deployed to homebrewery-pr-3113 January 5, 2024 19:33 Inactive
Remove unneeded console.log in tests
- Changes @*.@(*.) pattern to @[*.]()
- Updates editing color coding for leading @ to colormatch ! on images.

Sample Document.

```
@[](This|That|The other)
Page 1
\page
Page 2
@[here](there2)
@[onehere](there1)
\page
Page 3
@[ore](there3)
@[here](there1)
this
that
\page
Page 4
this
the other
\page
```

When the Rich Index is run, generates:

```
{{index,wide

- That ... pg. [1](#p0), [3](#p2)
- there1
  - here ... pg. [3](#here)
  - onehere ... pg. [2](#onehere)
- there2
  - here ... pg. [2](#here)
- there3
  - ore ... pg. [3](#ore)
- This ... pg. [1](#p0), [3](#p2), [4](#p3)

}}
```
@calculuschild calculuschild temporarily deployed to homebrewery-pr-3113 October 7, 2024 19:36 Inactive
@Gazook89
Copy link
Collaborator

Gazook89 commented Oct 8, 2024

Just doing a quick look at this now, and I likely will come back to this comment to add more bits later as I find them. This is a checkbox list not because everything has to be done, but just in case you want to check them as "looked at" as you read it.

  • a. At first I was confused on why it was generating multiple pages in your example brew but now I understand that it's because there are multiple appendices, so it makes more sense. I don't have an answer for this; but what if this snippet only generated the {{index ...}} portions, with no page breaks between them, and users could sort them out themselves? Here is what that would look like: (sidenote: you have a stray ` on page 1 inside your code fence which is causing bad formatting in the generated index entry.):
image
  • b. Is it worth considering generating an id for each index, since we are generating stuff anyway? So you'd get `{{#spell-index,index,wide ...}}? It would make it easier for users to target it with CSS later. But maybe not worth the extra clutter.

  • c. I think your index styling needs a little more. It should be a five-column layout at minimum. And for practicality, I suggest two properties: column-span: all for headers, and column-fill: balance for the index. The first one isn't true to the PHB, but with short indices with few entries they will look much better. And the column-fill works better with balance because the default auto requires either a fixed height or manual column breaks. Here are my recommendations:

	.index {
		column-count: 5;
		font-size : 0.218cm;

		h1, h2, h3, h4, h5 {
			column-span: all;
		}
   ...
  • d. An alternative to (c) is to change the .page/.columnWrapper itself to have 5 columns rather than 2. Basically drop the {{index }} block wrapper, switch to {{index}} span wrapper and use it like the cover pages where you use :has(.index). That way you'd have a fixed-height columnWrapper that is set to 5 columns and can use column-fill: auto. I still recommend column-spanning headers though. The generator wouldn't wrap each index anymore, but just stamp {{index}} at the top of the page, followed by index, and the CSS something like this:
.page {
	&:has(.index) .columnWrapper {
		column-count: 5;
	    column-fill: auto;
		font-size : 0.218cm;
		...
  • e. Rather than ellipsis, should it just be a comma between the entry and the page number (and leaving off pg. entirely)? That is how the PHB is.
  • f. Expanding on (e), the generator currently adds a trailing comma on every entry. It should likely be doing some sort of join(',') on an array of results rather than just adding a comma to each result? I haven't looked deep at the actual js yet.

That's enough to chew on for now I think. I haven't looked at crossreferences yet (the example doc doesn't seem to have them, which is fine for now, but should likely get them before we merge).

@dbolack-ab
Copy link
Collaborator Author

dbolack-ab commented Oct 12, 2024

I'm going to respond to these individually where I do.

  • a. At first I was confused on why it was generating multiple pages in your example brew but now I understand that it's because there are multiple appendices, so it makes more sense.

I probably could explain that more clearly. The section on using optional additional Indices doesn't do a great job of spelling out that they will be rendered distinctly.

I don't have an answer for this; but what if this snippet only generated the {{index ...}} portions, with no page breaks between them, and users could sort them out themselves?

I think that's kind of an "a or b" decision where one value isn't any better or worse than the other as a default. I think we can push this decision off to be part of deciding the final default styling.

@dbolack-ab
Copy link
Collaborator Author

dbolack-ab commented Oct 12, 2024

  • b. Is it worth considering generating an id for each index, since we are generating stuff anyway? So you'd get `{{#spell-index,index,wide ...}}? It would make it easier for users to target it with CSS later. But maybe not worth the extra clutter.

There should be an implicit id generated for the H#. If not, we should add one.

@dbolack-ab
Copy link
Collaborator Author

dbolack-ab commented Oct 12, 2024

C through E are really just styling thoughts. The styling is incomplete. It is the existing index styling with no tweaks. This tool should be style agnostic beyond the very minimal. Style should go in the theme.

  • f. Expanding on (e), the generator currently adds a trailing comma on every entry. It should likely be doing some sort of join(',') on an array of results rather than just adding a comma to each result? I haven't looked deep at the actual js yet.

That isn't doing a join, but there isn't a solid reason for not doing one.

That's enough to chew on for now I think. I haven't looked at crossreferences yet (the example doc doesn't seem to have them, which is fine for now, but should likely get them before we merge).

Are you using https://homebrewery-pr-3113.herokuapp.com/edit/vtI5G0Zl_fan or https://homebrewery-pr-3113.herokuapp.com/edit/PXQAJqD9pVbK Both should have them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature 🔍 R0 - Needs first review 👀 PR ready but has not been reviewed
Projects
Status: Waiting for Calc's First Review
Development

Successfully merging this pull request may close these issues.

7 participants