Single pass TOC help

[Cicorione]

Hi @pgundlach,

I am trying to integrate your Single Pass TOC example on my code but I lack the necessary skills to understand your abstraction... 🤦‍♂️

1. master pages

I have three master page so far: one that is used for opening each section which is always false and left & right that are always true based on test: odd or even. Your single master doesn't fit here but I feel that it is not here the problem.

2. First <Record> section.

Here I am difficult to understand if you are just creating dummy data or if this is actually important for the TOC, although I see an <action> tag:

<Action>
    <Mark select="concat('chapter',position())" pdftarget="yes" />
</Action>

3. Second <Record> section.

Here you state:

Here follows the core and entry point for data processing.

Copy and paste this won't work (of course); and I am very confused about these points:

1. that must be at the end of all other recording data.
2. You are not using any <Mark> tag to get the information about the section but you are loop across the "chapter".
3. Not sure how to call the TOC MASTER

Well a lot of confusion... Indeed... 😭

[pgundlach]

2. The data has three chapter elements:

<data>
    <chapter title="The first chapter" paragraphs="13"/>
    <chapter title="The second chapter" paragraphs="5"/>
    <chapter title="The last chapter" paragraphs="3"/>
</data>

So this is dummy data in the sense that it not meaningful, but actual data. It creates three marks (chapter1, chapter2, and chapter3).

These are used in step 3 (<A link="chapter{$n}">)

Third question:

1. When you have all information about the table of contents, you are ready to process it. So you must process all the data since you need to find out which page has the beginning of the chapter (or whatever you want to have in the TOC).
2. I am using Mark in the “collecting information” phase (see line 45 of the layout)
3. Not sure what the TOC MASTER is. The magic is in InsertPages and SavePages. You prepare the contents with SavePages and insert the contents with InsertPages. The trick is that in PDF you can re-organize pages. This is used internally to make the page with SavePages appear at the position set with InsertPages.

Please keep on asking questions.

[Cicorione]

Thank you Patrick, I am going to read what you wrote several times.

Just a quick comment: the TOC-MASTER is the master I would like to use for the Table of Content.

Thanks again! 🙏

[Cicorione]

Hi @pgundlach

I am still stuck here, I wonder if I can record at <Pagetype> level because this is the only place when I recall each section; is that feasible?

Thanks... 🙏

[pgundlach]

In the example you mention in the first post, I add a page type:

    <Pagetype name="mypage" test="false()">
        <PositioningArea name="text">
            <PositioningFrame height="{sd:number-of-rows() - 2}" width="{sd:number-of-columns() }" column="1" row="1" />
        </PositioningArea>

        <AtPageCreation>
            <PlaceObject column="2mm" row="2mm">
                <Textblock>
                    <Paragraph>
                        <Value>Page type mypage</Value>
                    </Paragraph>
                </Textblock>
            </PlaceObject>
        </AtPageCreation>
    </Pagetype>

and right before the InsertPages, I switch to that page type:

        <ClearPage pagetype="mypage" />
        <InsertPages name="table of contents" pages="1" />

Perhaps this works in your case as well.

[Cicorione]

Hi Patrick I am still struggling with this, and lack of time is making everything more complicated.

So far it looks like what I am doing is correct, I am not sure if I am actually recording because what I need is on the <Pagetype> level.

<!-- SECTION OPENING -->

	<Pagetype name="sectionstart" test="false()">
		<Margin left="20mm" right="20mm" top="20mm" bottom="20mm" />

		<AtPageCreation>

			<PlaceObject allocate="yes" column="0mm" row="0mm">
				<Image file="{$sectiontitleimage}" width="210mm" height="282mm" bleed="auto" />
			</PlaceObject>

			<PlaceObject row="12mm" column="77mm">
				<Box height="42mm" width="56mm" graphic="agl-logo-bkg" />
			</PlaceObject>

			<PlaceObject row="12mm" column="77mm">
				<Image width="46mm" file="assets/rdp.pdf" margin-top="5mm" margin-right="5mm" margin-bottom="5mm" margin-left="5mm" />
			</PlaceObject>

			<PlaceObject column="10mm" row="66mm">
				<Textblock width="190mm" minheight="4mm">

					<!-- MARK "SECTIONS"-->

					<Action>
						<Mark select="$secname"/>
					</Action>

					<Message select="concat('TOC: ', $secname)" />

					<Paragraph color="darkaglgray" fontfamily="opensecnamebig" textformat="centered">
						<Value select="$secname" />
					</Paragraph>
					<Paragraph color="white" fontfamily="opensecname" textformat="centered">
						<Value select="$yr" />
						<Value> Catalog</Value>
					</Paragraph>
				</Textblock>
			</PlaceObject>

		</AtPageCreation>

	</Pagetype>

I can see the messages I put on <Pagetypes> however I am not sure if the <marker> can store them at that level. However I guess the TOC page should be created nevertheless even though empty, and this is not happening.

Now I am going to create an box outside the page to see if I can recording the markers in another way... 🤔

I'll keep you posted! 👍

[Cicorione]

Hi Patrick,

I decided to go one step at the time, and I am testing if I understood and capable to save a page and move this page at the very beginning.

This is my current code:

<Record element="catalog">
    <ProcessNode select="section" />
    <!-- SAVE PAGE TEST -->
    <ClearPage pagetype="toc-page" />
    <InsertPages name="toc" pages="1" />
    <SavePages name="toc">
        <PlaceObject column="1" row="10">
            <Table padding="4pt">
                <Columns>
                    <Column width="7cm" />
                </Columns>
                <Tr>
                    <Td>
                        <Paragraph fontfamily="subtitle">
                            <Value>FIRST PAGE</Value>
                        </Paragraph>
                    </Td>
                </Tr>
            </Table>
        </PlaceObject>
    </SavePages>
    <ClearPage />
</Record>

In my code this <record> serves to process the section nodes but I used it also to save that page.
The page is not created, nor it is at beginning, I have only an additional blank page with the last layout used.

Possible reasons:

1. The code is wrong and it is a huge pebcak (90% truth)
2. I hit a bug because my code is convoluted (10% truth)

Also I can't see in the protocol.xml file anything about saving the page, therefore I can actually saying yes I made it wrong, perhaps the output should report about saving and moving pages. 🤷‍♂️

Please let me know if you want inspect the code, thanks! 🙏

[pgundlach]

I am unable to reproduce the problem since I don't have the data file, but your code looks okay to me. There is currently no logging information about SavePages/InsertPages.

The following layout creates three pages

1: Some text
2: hello, world
3: more stuff

In this special case it does not make sense to have InsertPages and SavePages next to each other, you could just add the contents here. See the example below where you create information (here an assignment to a variable, but usually to build a table of contents) after inserting the first page.

<Layout xmlns="urn:speedata.de:2009/publisher/en"
	xmlns:sd="urn:speedata:2009/publisher/functions/en">
	<Pagetype name="toc-page" test="false()">
		<AtPageCreation>
			<PlaceObject column="0mm" row="0mm">
				<Textblock>
					<Paragraph>
						<Value>toc-page</Value>
					</Paragraph>
				</Textblock>
			</PlaceObject>
		</AtPageCreation>
	</Pagetype>
	<Record element="data">
		<!-- SAVE PAGE TEST -->
		<PlaceObject>
			<Textblock>
				<Paragraph>
					<Value>Some text</Value>
				</Paragraph>
			</Textblock>
		</PlaceObject>
		<SetVariable variable="greeting" select="'world'"></SetVariable>
		<ClearPage pagetype="toc-page" />
		<InsertPages name="toc" pages="1" />
		<SavePages name="toc">
			<PlaceObject column="1" row="10">
				<Table padding="4pt">
					<Columns>
						<Column width="7cm" />
					</Columns>
					<Tr>
						<Td>
							<Paragraph fontfamily="text">
								<Value select="concat('hello, ', $greeting)"></Value>
							</Paragraph>
						</Td>
					</Tr>
				</Table>
			</PlaceObject>
		</SavePages>
		<ClearPage />
		<PlaceObject>
			<Textblock>
				<Paragraph>
					<Value>more stuff</Value>
				</Paragraph>
			</Textblock>
		</PlaceObject>
	</Record>
</Layout>

<Layout xmlns="urn:speedata.de:2009/publisher/en"
	xmlns:sd="urn:speedata:2009/publisher/functions/en">
	<Pagetype name="toc-page" test="false()">
		<AtPageCreation>
			<PlaceObject column="0mm" row="0mm">
				<Textblock>
					<Paragraph>
						<Value>toc-page</Value>
					</Paragraph>
				</Textblock>
			</PlaceObject>
		</AtPageCreation>
	</Pagetype>
	<Record element="data">
		<!-- SAVE PAGE TEST -->
		<ClearPage pagetype="toc-page" />
		<InsertPages name="toc" pages="1" />
		<PlaceObject>
			<Textblock>
				<Paragraph>
					<Value>Some text</Value>
				</Paragraph>
			</Textblock>
		</PlaceObject>
		<SetVariable variable="greeting" select="'world'"></SetVariable>
		<SavePages name="toc">
			<PlaceObject column="1" row="10">
				<Table padding="4pt">
					<Columns>
						<Column width="7cm" />
					</Columns>
					<Tr>
						<Td>
							<Paragraph fontfamily="text">
								<Value select="concat('hello, ', $greeting)"></Value>
							</Paragraph>
						</Td>
					</Tr>
				</Table>
			</PlaceObject>
		</SavePages>
		<ClearPage />
		<PlaceObject>
			<Textblock>
				<Paragraph>
					<Value>more stuff</Value>
				</Paragraph>
			</Textblock>
		</PlaceObject>
	</Record>
</Layout>

Not sure if it becomes clearer. Can you reproduce the examples? I am still uncertain what you want to achieve. Do you want to create a table of contents or something similar?

[Cicorione]

Thank you Patrick for your patience!

I am going to test your code and I'll let you know soon!

[Cicorione]

Oh my gosh... 🤦‍♂️

It took me three weeks to understand that <SavePages> & <InsertPages> have a total different behavior... 🤦‍♂️

I was convinced that <SavePages> save the pages generated in the memory and that <InsertPages> meant the position where you want to place it/them. 🤦‍♂️

For three weeks you have been thinking that I was just a poor fool man... I am very, very sorry... 🤦‍♂️

I realized it when I was attempting to move the page on another position so I made: <InsertPages name="toc" pages="10" /> to move the TOC at page 10 and I got:

E:  [page 2 line 66] SavePages: incorrect number of pages. Expected 10, got 1

On my little brain I was thinking: I generate the TOC at the very end (and that's why it was always at end, of course) than Publisher, after generated the PDF, would had made a further operation placing the last page to the beginning.

Please don't ban me from this place! 🙏

[Cicorione]

This means I have to rethink this from the beginning, I am deeply sorry... 😩

[pgundlach]

Then there must be something wrong with the manual / the description. I will take a look.

I generate the TOC at the very end (and that's why it was always at end, of course) than Publisher, after generated the PDF, would had made a further operation placing the last page to the beginning.

I think this is correct, except for "beginning" should be "the place where <InsertPages> is"

[Cicorione]

Hi Patrick, I am still need to recover from the embarrassment... 😟

I don't believe your wrote anything wrong or incorrect, I believe my knowledge about PDF tools that allow to move pages around induct to think it was the same thing.

[Cicorione]

Good news I am fully recovered from the trauma... 😅

And after thinking, working and testing, now I am able to put the TOC as first page (now I found the trick) and I was able to apply its own master.

Next I am going to populate the TOC using the entries from the section master rather than processing the entries into a node, I haven't started it yet, but I believe I will find how to do it!

Thank you very much Patrick for this amazing software and for your continuous help... 🙏

[Cicorione]

Hi @pgundlach, I made the homework but my ability to abstract reached its limit, so now I have a code that does almost what I want the way I want, although still ugly, but I can't loop across the section name therefore it repeats always the last record:

Here is the block:

<!-- TOC -->
<Record element="catalog">
    <ClearPage pagetype="toc-page" />
    <InsertPages name="toc" pages="1" />
    <SetVariable variable="greeting" select="'TOC'" />
    <ProcessNode select="section" />
    <SavePages name="toc">
        <PlaceObject column="6cm" row="6cm">
            <Table padding="4pt">
                <Columns>
                    <Column width="7cm" />
                </Columns>
                <Tr>
                    <Td>
                        <Paragraph fontfamily="subtitle">
                            <Value select="concat('hello: ', $greeting)" />
                        </Paragraph>
                    </Td>
                </Tr>
                <ForAll select="section">
                    <Tr>
                        <Td>
                            <Paragraph fontfamily="subtitle">
                                <Value select="concat('Name: ', $secname), ' — ', {sd:pagenumber($secname)}" />
                            </Paragraph>
                        </Td>
                    </Tr>
                </ForAll>
            </Table>
        </PlaceObject>
    </SavePages>
    <Message select="concat('Testing1:, ', $greeting)" />
    <Message select="concat('Testing2:, ', $secname)" />
</record>

What is the correct way to loop across the section name?

Thanks... 🙏

[Cicorione]

This is where I have the marker, it is a master page:

<!-- SECTION OPENING -->
<Pagetype name="sectionstart" test="false()">
    <Margin left="20mm" right="20mm" top="20mm" bottom="20mm" />
    <AtPageCreation>
        <PlaceObject allocate="yes" column="0mm" row="0mm">
            <Image file="{$sectiontitleimage}" width="210mm" height="282mm" bleed="auto" />
        </PlaceObject>
        <PlaceObject row="12mm" column="77mm">
            <Box height="42mm" width="56mm" graphic="agl-logo-bkg" />
        </PlaceObject>
        <PlaceObject row="12mm" column="77mm">
            <Image width="46mm" file="assets/rdp.pdf" margin-top="5mm" margin-right="5mm" margin-bottom="5mm" margin-left="5mm" />
        </PlaceObject>
        <PlaceObject column="10mm" row="66mm">
            <Textblock width="190mm" minheight="4mm">
                <Paragraph color="darkaglgray" fontfamily="opensecnamebig" textformat="centered">
                    <Value select="$secname" />
                </Paragraph>
                <Paragraph color="white" fontfamily="opensecname" textformat="centered">
                    <Value select="$yr" />
                    <Value> Catalog</Value>
                </Paragraph>
                <!-- MARK "SECTIONS"-->
                <Action>
                    <Mark select="$secname"/>
                </Action>
                <Message select="concat('TOC1: ', $secname)" />
            </Textblock>
        </PlaceObject>
    </AtPageCreation>
    <AtPageShipout></AtPageShipout>
</Pagetype>

[pgundlach]

               <ForAll select="section">
                    <Tr>
                        <Td>
                            <Paragraph fontfamily="subtitle">
                                <Value select="concat('Name: ', $secname), ' — ', {sd:pagenumber($secname)}" />
                            </Paragraph>
                        </Td>
                    </Tr>
                </ForAll>

This doesn't work because $secname does not change. How do you set $secname? What does the XML structure of section looks like?

[Cicorione]

Hi Patrick this is the structure

<catalog>
    <section secname="Section 1" accent="aglyellow"
        xmlns:xi="http://www.w3.org/2001/XInclude">

        <img file="images/sec-yellow.png"/>
        
        <subsection subsecname="01 SubSection 1" gridx="3" gridy="2" vertical="true">

            <specbox label="sp01">
                <imgbox file="sampleS.png" textbox="Some Text, Some Text, Some Text, Some Text, Some Text!" />
            </specbox>

            <product title="Smartphone X" subtitle="Flagship model" year="2024"
                xmlns:xi="http://www.w3.org/2001/XInclude">
                <item>5.8-inch OLED display</item>
                <item>Triple camera setup</item>
                <item>256GB storage</item>
                <item>8GB RAM</item>
                <item>Face ID recognition</item>
                <item>5G connectivity</item>
                <item>4K video recording</item>
                <extra>Wireless charging</extra>
                <extra>Water resistance</extra>
                <img file="sampleD.jpg"/>
            </product>

            <product title="Smartphone X" subtitle="Flagship model" year="2024"
                xmlns:xi="http://www.w3.org/2001/XInclude">
                <item>6.1-inch AMOLED display</item>
                <item>Quad camera setup</item>
                <item>512GB storage</item>
                <item>12GB RAM</item>
                <item>Facial recognition</item>
                <item>5G connectivity</item>
                <item>Ultra-wide angle lens</item>
                <item>Fast charging</item>
                <extra>Bluetooth 5.1</extra>
                <extra>Wireless charging</extra>
                <img file="sampleG.jpg"/>
            </product>

        </subsection>
    </section>
</catalog>

And specifically this is the node <section>

<section secname="Section 1" accent="aglyellow"
 xmlns:xi="http://www.w3.org/2001/XInclude">
[...]
</section>

Thanks... 🙏

[pgundlach]

<ForAll select="section">
                    <Tr>
                        <Td>
                            <Paragraph fontfamily="subtitle">
                                <Value select="concat('Name: ', @secname), ' — ', {sd:pagenumber(@secname)}" />
                            </Paragraph>
                        </Td>
                    </Tr>
                </ForAll>

should work in the page type. (The focus is in the element <section> because of the ForAll, therefore you can access the attributes with @.)

[Cicorione]

Thank you @pgundlach, it was an easy fix although I am still affected by these mistakes, it didn't touch me the idea to try with the attributes instead of the variables at all... 😩

Set aside my pebcakness, your example on the documentation is totally on another level, extremely elegant and articulated! I wonder if what I "achieved" can be still considered valid to generate a decent TOC, excluding the layout because my focus was on saving and attaching the page later.

I look forward to read your comment!

Thanks again! 🙏

[pgundlach]

Actually I never had the idea put the Mark in the Pagetype/AtPageCreation, but this seems to work well.

The problem is that you “never know” (strictly not true) where the focus in the data-XML is when the AtPageCreation is called, therefore you cannot access the data-XML during AtPageCreation.

You work around this issue by using a variable ($secname), which works, but can be a limitation if you need more complex data.

That said: if the output is what you want, it is okay. I often do dirty tricks with tables layout for fine-tuning spacing by using white table rules with a certain thickness which are invisible on white background/white paper (sounds a bit like pre-CSS era when doing layouts in HTML),

[Cicorione]

This is a great comment, thanks. 🙏

Actually I have tried, with all my limitations, to do something simple and easy to maintain.