Site caching and reducing traffic to AWS for hosted documentation

[daveverwer]

We’re now moved over to Cloudflare Cache Rules rather than the old Page Rules. Cache Rules are much more flexible and in learning about them today I think there’s a way we could consider finally allowing HTML responses from the main site to be cached. For example, package pages. We can also significantly reduce the data transfer requirements from AWS for documentation hosting.

We currently cache the following:

• Static resources (CSS styles, JavaScript, images, etc…)
• The home page (as it gets so much traffic)
• Package collection JSON files (as they are slow to generate)
• Hosted documentation (to reduce the data transfer from AWS)

Additionally, as of today we now cache depending on HTTP status code for 4XX/5XXs. If the site serves any page as a 4XX/5XX we now cache that response for 10 minutes. The requirement to implement this came from the recent (brief and package specific) documentation outage. This means any future issues like this will result in a 404 page for a maximum of 10 minutes.

All of our current caching uses a simple, static caching policy of 30 mins for the homepage and 2 hours for collections/documentation/others.

That’s where we are today.

Caching HTML responses

After learning much more about Cloudflare caching today, I believe we can now consider caching HTML pages without too much additional work.

We initially rejected caching HTML due to package data changing quite quickly in the first few minutes of a package’s life. When a package gets added it also gets added to the site homepage and people start visiting it, locking that version of the page into the cache for 2 hours. That version of the page has an empty compatibility matrix, which anyone visiting would see for far longer than necessary.

We could make two changes to package pages to pass along a sensible cache-control header that we can use to control the Cloudflare cache.

1. When we don’t have full compatibility data we could explicitly set the cache-control header to force Cloudflare to always serve the dynamic page with no caching (which is how we currently serve every package page). This means as the builds complete the grid gets filled in as it currently does.
2. Where we do have full compatibility data we can serve a cache-control that caches package pages for 2 hours as we do with other cached content.

We could alternatively set the Last-Modified and cache-control headers based on the most recent updated_at from Package/Repository/Version/Build data. This would remove the need for the special case based on having full compatibility data above as Build records would trigger a fresh version of the page, but I think the plan above is potentially simpler as we are already coalescing build data in our view model.

In the situations where we’re doing a full re-build of a Swift version and are missing compatibility data for multiple days we would always serve a dynamic version, but even then we’d only be falling back to exactly where we are now.

Reducing AWS documentation transfer

Then comes the really impactful change we could make. Smarter caching for documentation pages. I checked the AWS cost centre and (predictably) it’s not the requests themselves that are eating our AWS usage but the data transfer. We have actually spent $0 on requests and $LOTS on data transfer. We can do much better.

Once we generate and upload documentation it never changes (except when we do a re-build, but more on that later). Amazon generates and maintains an ETag for every file in a bucket and those tags never change once a set of documentation is uploaded.

First, once we have uploaded a documentation set we should use the S3 list-objects-v2 API to get a list of ETag values and store those in our database. We can do this either immediately after the upload either in the Lambda function or in the server app after we receive the build report.

Then, for every request for a documentation page that has previously been cached, Cloudflare will send us the last Etag it saw for that page. If we get an ETag, we then look up if we have it in our database and serve a 304 Not Modified response saying that the page is unchanged. We will not need to request the file from AWS unless Cloudflare expires it from its cache, and the Cloudflare cache can be configured to last one year.

For extremely long cache times like this, Cloudflare will occasionally check our server to revalidate its cache, but we just do exactly the same and return a 304.

Finally, we can use the Cloudflare API to purge package specific parts of the cache based on documentation uploads or re-builds. Again, we could either do this in the Lambda function or after receiving a build report.

At the point of a build or re-build we can expire the package documentation URLs per version to ensure we never serve stale documentation. Even on initial build and upload, we should just purge the cache for that version (e.g. /owner/repo/tag). It will have no effect at that point, but in the case of a rebuild that will be significant.

Conclusion

I may have missed something here, but I believe we could really improve how we are using the Cloudflare cache with some or all of these changes.

I’m opening this as a discussion as I’m sure there will be some views on it. If we go ahead with this, I think it should end up as two issues. One for caching package pages and adding cache-control headers across the site and another for implementing the data transfer reduction measures reduce access to AWS S3 files.

[daveverwer]

I had another thought overnight.

The biggest complexity in the AWS caching is going to be storing ETag data in the database. There is a way to avoid that, either as a stepping stone to the final solution or as a solution in itself.

Instead of doing a database lookup of the ETag, we could do a HEAD request to get the ETag when a request for a documentation file is made. This would be slightly slower than a database lookup, but would still avoid the data transfer fees.

It would avoid the complexity of storing all the ETag data and also the call to the AWS API to get all tags after uploading.

[finestructure]

Instead of doing a database lookup of the ETag, we could do a HEAD request to get the ETag when a request for a documentation file is made. This would be slightly slower than a database lookup, but would still avoid the data transfer fees.

It would avoid the complexity of storing all the ETag data and also the call to the AWS API to get all tags after uploading.

Yes, exactly. The complexity and storage overhead of that is prohibitive and really we'd only be trading HEAD request latency for db lookup latency. So it's unlikely to even be slower in any meaningful way, certainly not slower than our current proxy, on the contrary.