Feature Request: Query categories on demand

[FibreTTP]

According to your comments on the GetCategories() function, the top 100 categories on Twitch are grabbed, seemingly repeating until there are "no more":

ganymede/internal/twitch/category.go

Lines 47 to 49 in 47727d4

	// GetCategories gets the top 100 twitch categories
	// It then gets the next 100 categories until there are no more using the cursor
	// Returns a different number of categories each time it is called for some reason

This does not get every category on Twitch (which is probably unreasonable anyway), as I see when trying to add categories such as Minami Lane or Town of Salem 2 to a watched channel, they are not returned in the search box at all:

There is an API endpoint for searching for categories, so it'd be great if there was also an option to search Twitch for categories in the dropdown box, something like this:

which when clicked, would query Twitch with the string currently in the text box (with a first value of like, five), then commit new categories to the database and return them to the user.

[Zibbp]

It looks like the function to fetch categories is silently failing after some time, or somehow getting stuck. I deleted the twitch_categories table on my dev instance and restarted the API container to run the schedule again.

docker exec ganymede-db psql -U ganymede ganymede-prd -c "select count(id) from twitch_categories"
 count
-------
  3401
(1 row)

That number seems low. I restarted the API container again and it added some more.

docker exec ganymede-db psql -U ganymede ganymede-prd -c "select count(id) from twitch_categories"
 count
-------
  3917
(1 row)

Comparing this to my production instance, a lot are missing.

docker exec ganymede-db psql -U ganymede ganymede-prd -c "select count(id) from twitch_categories"
 count
-------
 35275
(1 row)

I'll investigate this some more. I prefer to fetch all the categories at the same the container is started versus querying the Twitch API. I'll get this converted to the new Workflow system and have it run once a day or something.

[Zibbp]

Re-familiarizing my self with this API endpoint. The /games/top endpoint returns categories of active streams. So it's likely that there are just no streams for the categories you're looking for. I'm not seeing an API endpoint that returns all categories/games. Over time you'll accumulate all the categories but that is unacceptable for a fresh instance.

Instead of implementing the search API you linked, I might edit the drop down to allow imputing your own category if it doesn't exist. It's simple and if you're adding a watched channel, chances are you already have the category in mind.

[FibreTTP]

inputting your own category if it doesn't exist

Would this be a query to /games instead, where the user would enter the exact name or ID of the category? (if so, you'd need some logic to decide if the user entered an ID or a name, or just restrict it to one of the two)

[Zibbp]

I was thinking of making the 'enter your own category' be an 'advanced' feature that has no validation (free-form text) to the /games Twitch API endpoint. Chances are the category you're looking for already exists if it's currently being streamed, so manually entering a category likely wouldn't be done often.
I'm trying to make everything as local-first as possible, though with the nature of archiving videos from a remote source it's not 100% possible. Pre-fetching and storing a list of categories in the DB is a great way to implement local-first instead of relying on an API to be working up when editing a channel.

I'm open to suggestions, not sure how to best proceed.

[FibreTTP]

I suggested a query because Ganymede currently saves all information about the category into the database; Name, ID, Cover art, IGDB ID (or so it seems):

ganymede/internal/twitch/category.go

Lines 29 to 45 in 4825322

	func SetTwitchCategories() error {
	categories, err := GetCategories()
	if err != nil {
	return fmt.Errorf("failed to get twitch categories: %v", err)
	}
	for _, category := range categories {
	err = database.DB().Client.TwitchCategory.Create().SetID(category.ID).SetName(category.Name).SetBoxArtURL(category.BoxArtURL).SetIgdbID(category.IgdbID).OnConflictColumns(entTwitchCategory.FieldID).UpdateNewValues().Exec(context.Background())
	if err != nil {
	return fmt.Errorf("failed to upsert twitch category: %v", err)
	}
	}
	log.Debug().Msgf("successfully set twitch categories")
	return nil
	}

Given that information is currently saved, I thought it would be used in the future (maybe for Ganymede's own game discovery system, or showing the game art in the chapter selection in the player). I also thought that categories were matched to the vods using IDs and not strings (which isn't the case), and would thus need a query of the game title to know the ID. Since that's not true, you can be local-first by simply string matching the user's inputted custom category to the ones found in the video, which is probably what you were thinking of doing.