Asynchronous HST Components and Containers

Overview

HST components and containers support asynchronous rendering by default. Typical scenarios include:

• Displaying personalized content, such as a "stores near you" component.
• Integrating with slow external services, like a Twitter feed.
• Serving content that should not be cached or indexed by search engines.

Asynchronous rendering is only active for live requests. For preview requests or when viewing the application in the Experience manager, components marked as asynchronous are rendered synchronously.

Configure an HST Component for Asynchronous Rendering

To enable asynchronous rendering for an HST component or container, set the following property on the relevant hst:component or hst:container node in the JCR:

hst:async: true

When a page includes asynchronous components or containers, a JavaScript file is automatically added to the response. This script issues separate Ajax requests to load the output of each asynchronous component or container.

To ensure correct script execution, update your root renderer (JSP or Freemarker template) as follows:

1. If your renderer already includes hst:headContributions, add "scripts" to the categoryExcludes attribute. If you already use categoryExcludes, separate multiple categories with commas.

<hst:headContributions categoryExcludes="scripts" xhtml="true"/>
copy

2. At the end of the renderer, typically just before the closing </body> tag, include:

<hst:headContributions categoryIncludes="scripts" xhtml="true"/>
copy

This ensures that the asynchronous loading scripts are included at the correct point in the HTML, triggering Ajax calls to load asynchronous components.

Supported Asynchronous Modes

By default, setting hst:async = true uses client-side Ajax to fetch and aggregate asynchronous component content. If you prefer server-side aggregation, you can use ESI (Edge Side Includes) or SSI (Server Side Includes) modes. These modes allow aggregation by a server or CDN (such as Squid, Varnish, or Apache HTTPD).

Supported modes:

1. ajax (default)
2. ssi (Server Side Includes)
3. esi (Edge Side Includes)

To specify the asynchronous mode, set the hst:asyncmode property. For example:

hst:asyncmode: esi

Restrictions

Consider the following limitations when using asynchronous HST components and containers:

• All descendant components of an asynchronous component are rendered as part of the asynchronous request for the ancestor.
• Asynchronous components must be independent from synchronous components. For example, do not rely on synchronous components to populate request context needed by asynchronous components. While you can set hst:standalone = false, this is strongly discouraged. See HST component rendering for details.
• Asynchronous components cannot contribute hst:headContributions because they are rendered after the head contributions are written to the HTML.
• If the client does not have JavaScript enabled and the async mode is Ajax, asynchronous components will not render.

A key implication of the last restriction is that search engines will not index the output of asynchronous components, as bots do not execute Ajax calls. This is often desirable for personalized or externally dependent content.

Note:
Asynchronous components cannot contribute hst:headContributions.

Nested Asynchronous Components

If you mark a hst:component as hst:async = true and it has an ancestor component already marked as asynchronous, the ancestor's asynchronous request will render all descendants, regardless of their own hst:async setting. Nested asynchronous HST components are supported, but all descendants are rendered together with the ancestor.

How Asynchronous Rendering Works

When you set hst:async = true on a component, HST request processing skips the doBeforeRender and render methods for that component and all descendants. Instead, HST injects a placeholder <div> in the response, with an id containing a Component Rendering URL and a class with an obfuscated value. The aggregation valve also adds a head contribution for the asynchronous JavaScript loader, included via:

<hst:headContributions categoryIncludes="scripts" xhtml="true"/>
copy

Disable Asynchronous HST Component Rendering

Global Configuration

Asynchronous HST component rendering is enabled by default. To disable it globally, regardless of component-level configuration, set the following property to false in your HST-2 Container Configuration:

# Disable asynchronous component window rendering globally.
default.asynchronous.component.window.rendering.enabled = false

Automatic Disabling for Search Engine or Bot Requests

For requests from web crawlers or bots, asynchronous rendering is automatically disabled (since v13.4). This ensures that crawlers receive a complete page response, including content from asynchronous components, which is important for SEO.

By default, HST detects many known web crawlers by checking if the User-Agent HTTP header contains any of the following values:

abachobot, accoona-ai-agent, addsugarspiderbot, adsbot-google,\
anyapexbot, aolbuild, appengine-google, arachmo, baidu, bingpreview, b-l-i-t-z-b-o-t, becomebot, beslistbot,\
billybobbot, bimbot, bingbot, blitzbot, boitho.com-dc, boitho.com-robot, bot, btbot,\
catchbot, cerberian drtrs, charlotte, converacrawler, cosmos, covario-ids,\
dataparksearch, diamondbot, digg deeper, discobot, dotbot, duckduckgo, earthcom.info, emeraldshield.com webbot,\
envolk[its]spider, esperanzabot, exabot, fast enterprise crawler,\
fast-webcrawler, fdse robot, findlinks, furlbot, fyberspider, g2crawler, gaisbot,\
galaxybot, geniebot, gigabot, girafabot, googlebot, googlebot-image, googlebot-mobile,\
googlebot-news, googlebot-video, gurujibot, happyfunbot, hl_ftien_spider,\
holmes, htdig, iaskspider, ia_archiver, iccrawler, ichiro, igdespyder, irlbot,\
issuecrawler, jaxified bot, jyxobot, koepabot, l.webis, lapozzbot, larbin,\
ldspider, lexxebot, linguee bot, linkwalker, lmspider, lwp-trivial, mabontland,\
magpie-crawler, mediapartners-google, mj12bot, mlbot, mnogosearch, mogimogi,\
mojeekbot, moreoverbot, morning paper, msnbot, msrbot, mvaclient, mxbot, netresearchserver,\
netseer crawler, newsgator, ng-search, nicebot, noxtrumbot, nusearch spider,\
nutchcvs, nymesis, obot, oegp, omgilibot, omniexplorer_bot, oozbot, orbiter,\
pagebiteshyperbot, peew, polybot, pompos, postpost, psbot, pycurl, qseero,\
radian6, rampybot, rufusbot, sandcrawler, sbider, scoutjet, scrubby, searchsight,\
seekbot, semanticdiscovery, sensis web crawler, 'seochat::bot', seznambot,\
shim-crawler, shopwiki, shoula robot, silk, sitebot, snappy, sogou spider,\
sosospider, speedy spider, sqworm, stackrambler, suggybot, surveybot, synoobot,\
teoma, terrawizbot, thesubot, thumbnail.cz robot, tineye, truwogps, turnitinbot,\
tweetedtimes bot, twengabot, updated, urlfilebot, vagabondo, voilabot, vortex,\
voyager, vyu2, webcollage, websquash.com, wf84, wofindeich robot, womlpefactory,\
xaldon_webspider, yacy, yahoo! slurp, yahoo! slurp china, yahooseeker, yahooseeker-testing,\
yandex, yasaklibot, yeti, yodaobot, yooglifetchagent,\
youdaobot, zao, zealbot, zspider, zyborg, abilogicbot, link valet, link validity check,\
linkexaminer, linksmanager.com_bot, mojoo robot, notifixious, online link validator,\
ploetz + zeller, reciprocal link system pro, rel link checker lite, sitebar, vivante link checker,\
w3c-checklink, watchmouse, copperegg, revealuptime, bot/0.1, compspybot, feedfetcher-google, rogerbot,\
sogou web spider, twitterbot, xenu link sleuth, crawler, spider,\
scraper, bash, java, facebook, nutch, ruby, httpclient, optimizer, go 1.1 package,\
megaindex, brokenlinkcheck, dlvr.it, ltx71, qwantify, python-urllib,\
python-requests, google favicon, binlar, metauri, coccoc, disqus, xurl, netlyzer,\
gigablastopensource, panscient, hubspot marketing grader, kimengi, libwww-perl, trove, typhoeus

To check for additional User-Agent values in your project, add them to the search.engine.or.bot.user.agent.patterns.extra property in your HST-2 Container Configuration:

# Extra comma-separated search engine or bot User-Agent regex patterns (lowercase only).
search.engine.or.bot.user.agent.patterns.extra = mycompanybot, yetanotherbot

All values must be lowercase.

API Support

Since v13.4, you can use the HstRequestContext#isSearchEngineOrBotRequest() API to check if a request is from a detected web crawler.

Troubleshooting

If an asynchronous component does not appear on the page (and you have not set hst:asyncmode to esi or ssi), use browser development tools to check for an Ajax network request similar to:

?_hn:type=component-rendering&_hn:ref=r6_r1

The _hn:ref value is unique per component. The _hn:type=component-rendering parameter indicates that the Ajax call uses a Component Rendering URL.

If you do not see this request, inspect the HTML. The page should include a JavaScript element in the <head> similar to:

a425658310.b425658310.AsyncPage = { .....

At the end of the HTML, you should see:

<script type="text/javascript">
//<![CDATA[
a425658310.b425658310.AsyncPage.load();
//]]>
</script>

The a425658310.b425658310 part is project-specific and randomly generated. If the AsyncPage.load(); script appears in the <head>, asynchronous Ajax calls will not work. To resolve this, ensure that the base page template uses categoryExcludes="scripts" in the <head> and categoryIncludes="scripts" at the bottom of the page. A typical template looks like:

<!doctype html>
</html>
  <head>
    <snip/>
    <@hst.headContributions categoryExcludes="htmlBodyEnd, scripts" xhtml=true/>
  </head>
  <body>
    <snip/>
    <@hst.headContributions categoryIncludes="htmlBodyEnd, scripts" xhtml=true/>
  </body>
</html>
copy

Exclude the scripts category at the top and include it at the bottom to enable asynchronous components.

Customizing Asynchronous AJAX Component Loading

Info: This feature is available in brXM 12.4 and later.

When a page loads, HST generates a scriptlet to trigger asynchronous component loading in Ajax mode:

<script type="text/javascript">
//<![CDATA[
a425658310.b425658310.AsyncPage.load();
//]]>
</script>

You may need to customize this scriptlet, for example, to delay the JavaScript call until other AJAX calls complete or visitor state is established.

To customize the generated scriptlet, set the ajax.asynchronous.component.windows.load.js.fragment.template property in hst-config.properties:

# Example: wrap the default JavaScript fragment using java.text.MessageFormat format.
ajax.asynchronous.component.windows.load.js.fragment.template = registerAsyncComponentsRenderingCallback(function() '{' {0} '}');

With this configuration, HST generates:

<script type="text/javascript">
//<![CDATA[
registerAsyncComponentsRenderingCallback(function() { a425658310.b425658310.AsyncPage.load(); });
//]]>
</script>

Implement the custom JavaScript function, for example:

function registerAsyncComponentsRenderingCallback(callback) {
  // Perform actions to establish visitor state, such as AJAX calls to analytics.
  // ...
  callback();
}
copy

By configuring ajax.asynchronous.component.windows.load.js.fragment.template, you control how and when asynchronous components are loaded on page load.