We could point out the following problem with the Akamai Cache (or CDN Cache) https://madsack.arcpublishing.com/alc/arc-products/arcwide/caching-within-arc/ :
The Akamai-Cache returns a cached html which contains all content-sources.
The content-sources are valid for the moment when this html is created. I think the Akamai-cached-HTML is cached 2 minutes.
When the Akamai-Cached-HTML is delivered the first time to the client all content-sources are valid
Now it is possible that a Content-Source expires whithin the 2 minutes while the Akamai-Cached-HTML is valid
So if a content source expires after e.g. 10 seconds, it is fetched every time in the last 1:50 minutes when the page gets reloaded by the client.
My idea to solve the problem:
When the Akamai-Cache is building the html and is fetching the content sources it should check if the content-sources are expiring within the 2 minutes.
If a content-source expires within the 2 minutes the content-source has to be updatet, so that the content source is valid as long as possible. That means use api/v3/content/update/ instead of api/v3/content/fetch/
(If a content-Source has a time to live which is lower than the 2 minutes it gets at least fetched as seldom as possible)
Is it possible to change the Akamai Cache in this way?
Marc Hekeler
Hi Fatih,
we finally got around to actually trying staticMode for this usecase and it doesn't work unless we make the component itself static or use a <Static> Block to make it partially static, as staticMode makes the content disappear from the contentCache entirely.
So for non-static components, of which we have quite a few, this approach of preventing the client-side refetch does not work.
Maybe you meant a different property and not staticMode?
Hi Fatih,
thanks for the info, that fills in the blank spots I was missing.
Hi Stefano,
Yes, the staticMode will defer the content to be cached at the CDN cache since it will not be refreshed at the client-side render. You can try to balance out the feature instances that can be set as staticMode so they are served with SSR and CDN cache and don't occur client-side calls, and leave the blocks that may potentially get updated/fresh content more frequently.
About the dynamic calculation; if your page has many feature instances that have different expiration dates, the calculation itself will practically result very soon expiration date - unless you are not setting very long custom TTLs from your content-sources (I think that's not the case for what you are describing).
That means the dynamic calculation will always result to something less than next 2 minutes timeframe. Even if Engine returns an expiration date less than that, CDN will be falling back to minimum 2min TTL and override that expiration date. So practically you are setting your CDN TTL to 2 minutes if your page has many feature instances, because there will always be something expiring within in the 2 minute window.
Currently, we don't allow customization on CDN TTLs which is set to be 5 minute default TTL but with manual/custom requests, it can be updated to 2 minutes by the delivery team. We are currently planning a capability to provide a way to let you customize the CDN TTL from Editor UI. Even though it's not exactly what you described, practically it will be the method you can use for this. Set a smaller TTL for a page or resolver if you have content that gets frequently updated or if you have a page that has many blocks using content sources (like homepage or section front page). We don't have a timeline or target dates set for this feature, but it's in the works and we'll launch this capability in the coming Engine and Editor releases.
Hi Fatih,
sorry for the delay, I was out sick. Thanks for the feedback.
I'm not sure I communicated my intent below correctly. Just to be sure:
I didn't want to reduce the TTL in the content-source, but rather set a dynamically calculated HTTP expires header when shipping the HTML to the CDN.
In other words: Engine builds the HTML, including fetching the content-sources. It then looks at the expiration dates of all used content-sources, takes the earliest expiration date and sets it as expires header on the HTML.
Would that be possible?
We'll look into the staticMode idea as well (we've actually used it before, but always in combination with the Static component afaik). The bad side of this approach is, that we'll still have content that might be out of date for longer than intended.
Hi Fatih, thank you. We will discuss that in our team in the next days and let you know what would be the best solution for us.
Hi Fatih,
Is it possible that there could be two TTLs? one for server-side rendering and one for client-side rendering. And by default the client side ttl could be two minutes bigger than the server side
Hi Marc,
Thanks for submitting this idea. I've reviewed the way you described what you are trying to do with the team. There is a baseline Arc caching policy that it can't be lower than 2 minutes at the CDN level. We're currently exploring ways to allow you to customize this at the resolver configuration but it will have the 2 minute minimum TTL enforced.
Team also reviewed the dynamic calculation of the earliest expiring time which can't be done because that calculation can't be executed on the CDN (edge nodes) and it has to be engine doing the calculation which defeats the purpose of CDN cache for handling larger traffic smoothly. In short, Engine can't do the dynamic calculation of the nearest expiring content source and decide to call or not call content source from client side.
From your description, it seems you are trying to prevent unnecessary client-side content source calls, currently these unnecessary calls can create strain on engine back-end, creating risk to hit rate limits to the apis used in content sources. There is a property called "staticMode" in getContent and fetchContent methods arguments from the fusion:consumer. staticMode configuration property tells engine to execute or not to execute client-side content source calls from feature code. The optimization you are trying to do may be done with this method. Can you check out the consumer api documentation where we talked about staticMode: https://redirector.arcpublishing.com/alc/arc-products/pagebuilder/fusion/developer-documentation/consumer-api/ (in getContent method properties list).
Let us know if this is what you are looking for. Otherwise, the way this idea in the description is not possible. For now, we're marking this idea will not implement.
I have a feeling, this is not possible. Alternative Idea: Could it be possible to set http headers according to the earliest content-source expiration, so that it expires earlier?