[Feedback Requested] Swagger Model for Next Version of the Catalog Items APIs

[chapmanjw]

My team is working on the next iteration of the Catalog Items API. Attached to this discussion is a preview of the updated Swagger model and a preview response for the getCatalogItem operation with the model changes.

The notable differences include the following:

• The getCatalogItem and searchCatalogItems provide access to the same sets of data (i.e., attributes available on the searchCatalogItems operation).
• Identifier-based searches added to searchCatalogItems (allowing up-to 20 identifiers to be queried per request, providing a "batch"-like operation).
• Both operations include a new dimensions data set.
• The variations section has been expanded to include the variation theme details (such as the theme name and the attributes related to the theme).
• The summaries section has been expanded to include more identifying attributes, such as package quantity and part number.

These tools are useful in viewing and comparing this updated Swagger model with the previous model:

• https://editor.swagger.io/
• http://jsondiff.com/

As our team works on implementing these changes, we would love your feedback on any further data model adjustments that we should consider before launching this next version.

Thanks!

catalogItems_2022-04-01.json.txt
catalogItems_2022-04-01.getCatalogItem.example.json.txt

[teddy-codes]

I think it's worth it to add the identifier batch size to the identifiers query param under searchCatalogItems. Also, is the throttle rate now going to be dependent on the number of identifiers provided to searchCatalogItems? Didn't see anything talking about it, but just wanted to clarify.

IIRC after these changes are made, our team can start to work on migrating over (there is just one more open issue). Thanks a ton for the awesome work that you and your team have done!

Side note: Unsure who controls this, but answers aren't typically relevant for the Ideas category. It can be disabled in favor of comments.

[chapmanjw]

For the identifier lookups and page size parameters in searchCatalogItems, are you thinking that we would take the whole list of ASINs found (i.e. if 5 IDs turns into 200 ASINs) and return the data in chunks by page size? (i.e., not page size per input param, as we are returning one list, not one list per param).

Yea, agreed the Ideas category with answers isn't the best mechanism here, lol. Working with what we have setup :-).

[chapmanjw]

As far as throttling rate goes, TBD.

Rationalizing two different throttling rates (i.e., how many per request + how many total across requests) is awful IMO. We would ideally have a rate for the searchCatalogItems operation at a request level only, scaled to handle that traffic for anything within the scope of what one operation can request (i.e. 20). Unless we identify some compelling reason that we have to throttle per item instead of per request, I don't see us spending the resources to re-implement that per-item throttling. Since it doesn't exist in these new APIs today, would take more development resources to implement than to not implement.

[teddy-codes]

I would like to see that if 5 IDs turn into 200 ASINs, all 200 ASINs should be available to be paginated through. The Amazon catalog is a messy place... Lots of UPCs having 2-10+ listings (some which aren't even relevant).

In regards to throttling, I would hate to see that implemented. Having something anywhere remotely close to that would make the dev experience a ton worse and of course... as a consumer, much less than ideal. Glad to hear that the requests are going to continue getting throttled at the request level.

[callumb123]

Will there be any improvements included for #829 ?

There are still a large number of cases where searchCatalogItems returns zero results when listCatalogItems returns 10 for the exact same query string. I was hoping this would be included in the next release so we do not lose the functionality of listCatalogItems when it is deprecated.

[chapmanjw]

@callumb123 We are working to address the keyword search relevance issues, etc., independently. (i.e., once we change something there, it will apply to both the existing v2020-12-01 version and this upcoming version).

[callumb123]

Hi @chapmanjw would you be able to comment on the rough ETA for this given the recent release? The new catalog Items API is great but the above issue is the only reason I still need to use listCatalogItems over searchCatalogItems for any query using only keywords. listCatalogItems has a significantly slower restore rate (despite what the docs and response header say) so it really feels like a disadvantage using it.

[JeromeXapix]

@chapmanjw Would it be possible to get all ASINs from a selling partner as suggested in your comment here?
If not could we extend the limit of ASINs that could be queried?

Thanks!

[chapmanjw]

@JeromeXapix With this particular API, you would need to know the ASIN or SKU you are querying for and do so in batches of up-to 20.

Separately from this, we are working on better addressing the truly bulk use-cases here for ASINs related to a selling partners listings. Stay tuned for updates later in the year on that.

[JeromeXapix]

Sounds good! Thanks for the info @chapmanjw

[therubioftw]

@chapmanjw - thanks for all the hard work!
Before writing I've spent the better part of my morning reading through all the different discussions, and it really warms my heart seeing you so involved, I really appreciate it.

As for this thread - I wanted to recap and make sure I +1 the few things I've seen discussed so far:
a. Sorting for searchCatalogItems - for our use-case, we would love to have a "by rank" sorting option, to list the top X by ranking per category, for example.
b. Filtering for searchCatalogItems- ASINs for a specific partner would also be great. We have hundreds of clients, and today if we want each client's products, we can get a list easily for FBA listings, but if a selling partner has FBM products as well, it's up to a report to pick up the slack and that's not a good solution.
c. Querying searchCatalogItems - being able to use a more complex boolean query would also be lovely, and help in reducing the amount of data pulled.

[chapmanjw]

@therubioftw Some clarifying questions for each:

a -> We are evaluating options for our keyword-based search results to behave more like listCatalogItems (v0) and ListMatchingProducts (MWS), where they use similar relevance ranking to the retail websites. We've heard the feedback on that loud and clear ;-). Currently, the search index we are using does not take into account any buyer activities in its ranking, it's purely based on the full text index of the ASIN data. (Querying "M" brings back the items with the most "M"s in it, not the 1930s movie "M", which is not great).

If we get that addressed, what other sort options would you be looking for? With the classification refinements on browse nodes, does that work for the "ranking within a category" idea?

b -> With the "SKU" identifier type option, you could retrieve them if you already know the SKU. However, if you are looking to discover what listings a selling partner has, this is not the right API. We are working on a search API for the Listings Items API, which would help with that use-case (along with bulk mechanisms around that). Stay tuned for more updates on those later in the year.

c -> Do you have some examples of what you are looking for here? One idea we've been looking into is the ability to search by attribute values (like "part_number = X"). Is that what you are thinking there?

[therubioftw]

@chapmanjw
a. I think the idea for refining to a specific category would help a lot, but I would still like to be able to sort by rank. One of our main use-cases is assessing competition/threat for our clients, so pulling the more "important" products first, perhaps skipping the trailing duds altogether, would allow us to request less data, more frequently.
b. Indeed, when our clients first log in and connect us to their stores, we don't have a list of SKUs, we need to generate one via SP-API. Right now, since a lot of them use FBA and FBM, we have to go through the "all listings" report. I would love not to have to do that eventually... If you think of a faster way to do it in the meantime, I'd love to hear :)
c. Searching for a very specific boolean query would be cool ("shirt AND (boys or girls) AND Red NOT costume") but I agree isn't a top priority. It's something that not even the store UI really supports, but maybe is possible through the API?
Your example, of filtering using attributes would be very valuable, kind of goes back to the idea of filtering to a specific category but is a better implementation. Super useful.

[drewxa]

@chapmanjw
hi!

We are evaluating options for our keyword-based search results to behave more like listCatalogItems (v0) and ListMatchingProducts (MWS), where they use similar relevance ranking to the retail websites.

I was wondering if there has been any progress in sorting the search results by relevance? It has been some time since your last update and I was hoping to know if there are any new developments. Additionally, I have not been able to find any practical applications for the current API. I appreciate any information you can provide on this matter.

[Mike-the-one]

@chapmanjw I looked at current API, looks like current version only allows me to search by one SKU per call, this is very inefficient, just wondering when do you plan to release the change mentioned here?

[jesusvalle]

It would be very interesting to also have access to the "package_level" property, since it is a mandatory field when updating a product from a vendor account, and through the API (as far as I know), it is not available.

Thank you.

[fegbers]

Hi,
Any news when the new version will be released?
Thank you

[littleylv]

As the file name is catalogItems_2022-04-01.json.txt I thought it would be release at 2022-04-01 as version 2022-04-01. But seems nothing released today.

[teddy-codes]

The longer that this deployment is pushed off, the worse off us devs are because there still is the hard requirement to be migrated by end of September.

[txphi592]

Same boat, just waiting. You would think Amazon would want to tell us when to expect release so we can plan around it 🤷. Hope they give us enough time from update release date to September deadline to implement everything.

[Mike-the-one]

Same boat. The whole migration journey is very rough. We started last year before Amazon decided to postpone the deadline. It's like we move 1 step forward, then encounter something unexpected, wait for Amazon to fix or respond, then move another step, and repeat the same process again. We have wasted many many hours with this process. Also, seems they don't have a policy to monitor the issues on github. They should realize if they make the API better or answer the questions here as much as possible which will reduce the support ticket volume. Well, they don't care as they are big enough to ignore everyone.

[jesusvalle]

The product description is also not returned by the catalog call. Titles and bulletpoints are, so it would be great to get all 3!

Thank you!

[toddsay]

Our team would like to know whether there are any plans to include Category Tree details in this API. The V0 version included listCatalogCategories, which seemed analogous to the MWS' GetProductCategoriesForASIN. I see that the upcoming spec does include a category and subcategory, but as far as I see we cannot obtain the entire chain of parent categories up to the root.

[therubioftw]

Hi @chapmanjw - I see the April release notes are out, and this has still hasn't been updated.
We've been kind of waiting on this to continue work on our end, and it would be great to get some sort of alignment on development...

Thanks!

[littleylv]

I am waiting for the new Catalog Items APIs for my work too, but this April 2022 SP-API Release Announcement has nothing related to Catalog Items API 😭

[Mike-the-one]

Not sure if they are aware this is not an opensource project. Amazon sellers pay money to Amazon, and the sellers hire us to work for them. So it is a buyer (us) and seller (Amazon) relationship.

[toddsay]

The current getCatalogItem implementation requires special permissions to view the vendorDetails. Will this continue to be the case? I see that the new spec includes category/subcategory information there, but now I'm doubting we will be able to retrieve those.

It currently does not appear that we will have access to the category name for a product. Is there a way to obtain this?

[littleylv]

Hi, all

Exciting news! At last, the new version is released! May 2022 SP-API Release Announcement

[chapmanjw]

Thank you everyone for your amazing feedback. You have helped us to shape this updated version of the Catalog Items API and we are extremely grateful for your help.

https://developer-docs.amazon.com/sp-api/docs/catalog-items-api-v2022-04-01-reference

Marking this as the answer to close out this discussion. For future releases, please continue to provide your feedback as issues here on Github or via Developer Support.

Thanks!

[ericblade]

@chapmanjw It absolutely violates every rule in existence for dealing with currency to use floating point numbers to describe currency. Why do we continue to do this, so many years into this new API?

I sure do love getting floating point errors in currency math. Especially when Amazon sends me things like 109.999999999999999999999991 from an API to get the price of something.

What am I supposed to do with that?

[ericblade]

BTW, the documentation describes "includedData" as "attributes, dimensions, identifiers, images, productTypes, relationships, salesRanks, summaries, vendorDetails".

Asking for "dimensions", "relationships", or "vendorDetails" in a getCatalogItem request results in an error response. Including any of them in a searchCatalogItems request seems to result in an error response as well. The getCatalogItem does include dimensions regardless of whether you request them or not. searchCatalogItems seems to be absolutely busted.

There are separate sections for the includedData for each one, but they are identical sections. That doesn't seem to be the intention -- should they differ?

[ericblade]

Third issue -- API has described throttle limit of 2/2. Yet I can make one request for 20 items at a time. The way I understand the normal API documentation, is that a batched request for 20 items counts as the same as doing 20 individual requests -- so clearly, that's not possible. But, also, we are being penalized for NOT knowing if we have multiple requests to perform up front. But how can we know that, when we haven't got the data yet?

Per the way this seems to work, if I have 40 requests I can do them in one second, if I know they are coming up (ie, if i'm gathering data for an automated system), but if it's in response to a user request (ie, find me the data for this specific product i'm looking at), i can only do 2 of those a second. Even though they should resolve to the same thing.