Filtering

TDI allows modifying and extending its parser events before they reach the tree builder. Since these filters are applied once before the rendering process, they have no performance impact during the actual rendering.

Understanding The Build Process

From Template To Template Tree

First let’s take a look at the basic template tree building logic:

_images/treebuilder.png

Processflow: How a template becomes a template tree

It’s more or less standard compiler logic. The steps are as follows:

  1. The template is fed chunk by chunk to the parser (which contains a lexer and a “tokenizer”).
  2. The found tokens are just messages or events like “a starttag with this name and these attributes” or “an endtag with this name”. These events are passed to the tree builder.
  3. The tree builder creates relations between the events by maintaining a stack of them and thus determining the current nesting. This information is used to actually build the tree.
  4. The template tree is the final structure containing all nodes, prepared for rendering.

Adding Filters

Installing event filters into the process flow described above is easy. You just need to intercept the events between step 2 (events emitted by the parser) and step 3 (events received by the tree builder). The following figure visualizes the updated logic:

_images/treebuilder-filtered.png

Processflow: Creating a template tree with filters

And here is, how it’s implemented:

The tree builder is an object implementing two interfaces: → BuilderInterface and → BuildingListenerInterface. The former is used by a user of the builder object (the code expecting the result). The latter is used by the parser, which feeds the builder with events. Events are implemented as different methods of the BuildingListenerInterface, like handle_starttag() or handle_endtag().

A filter is an object, which “impersonates” a builder in every direction. It has to implement both interfaces and pass along or manipulate the method calls it receives. Sometimes filters may even generate their own. In other words, a filter object simply wraps the builder or the next filter. By stacking them that way you can easily combine as many filters as you want.

Using Filters

Filters can be hooked into different stages of the template loading process. But all of them follow the same principle. You need to specify a list of filter factories. A second parameter specifies, whether the default filters should be applied (usually a good idea). A filter factory is a callable returning the filter object (→ FilterFactoryInterface). Often the filter class itself can be passed as factory.

There’s only one default filter right now - the encoding detection filter - which generates handle_encoding events, whenever it sees a suitable declaration in the event stream.

TDI always adds a filter, which provides the filename of the template (which may or may be not something useful). But this filter cannot be turned off. (→ filters.FilterFilename)

The following sections describe the different places, where you can hook in your filters.

Load Filters

Load filters are invoked when initially loading a template into the memory. They are specified using the eventfilters and default_eventfilters arguments in → Factory.__init__ and → Factory.replace.

A typical load filter is the → MinifyFilter provided by the html tools:

from tdi.tools import html as html_tools
from tdi import html

tpl = html.replace(eventfilters=[
    html_tools.MinifyFilter
]).from_string("""
<html>
<head>
    <title>Hello World!</title>
    <style>/*<![CDATA[*/
        Some    style.
    /*]]>*/</style>
</head>
<body>
    <script>//<![CDATA[
        Some    script.
    //]]></script>
    <h1>Hello World!</h1>
</body>
""".lstrip())

tpl.render()

Note, how it minifies everything except the <script> and <style> elements:

<html><head><title>Hello World!</title><style>/*<![CDATA[*/
        Some    style.
    /*]]>*/</style></head><body><script>//<![CDATA[
        Some    script.
    //]]></script><h1>Hello World!</h1></body>

If you want to squash those, too, just add more filters:

from tdi.tools import html as html_tools
from tdi.tools import css as css_tools
from tdi.tools import javascript as javascript_tools
from tdi import html

tpl = html.replace(eventfilters=[
    html_tools.MinifyFilter,
    css_tools.MinifyFilter,
    javascript_tools.MinifyFilter,
]).from_string("""
<html>
<head>
    <title>Hello World!</title>
    <style>/*<![CDATA[*/
        Some    style.
    /*]]>*/</style>
</head>
<body>
    <script>//<![CDATA[
        Some    script.
    //]]></script>
    <h1>Hello World!</h1>
</body>
""".lstrip())

tpl.render()

<html><head><title>Hello World!</title><style>Some style.</style></head><body><script>Some script.</script><h1>Hello World!</h1></body>

Note

Filter order may be important. For example, the → javascript.CDATAFilter adds a CDATA wrapper around a script block, while the → javascript.MinifyFilter removes those. So, if you want to create CDATA containers around your minified script, the CDATA filter needs to run afterwards.

Filters are specified in reverse running order. That is, if you write them vertically (as in the examples above), the builder is “up” and the parser is “down”.

Overlay Filters

Overlay filters are applied after all overlaying is done. That’s right before pre-rendering (or rendering, if the template is not pre-rendered). The filters are specified using the overlay_eventfilters and overlay_default_eventfilters arguments in → Factory.__init__.

Filtering at this stage is mostly useful if you need to look at the final template (constructed by the overlays). For example, you might write a filter, which resolves script or style dependencies of the template.

Prerender Filters

Prerender filters are run after pre-rendering, but before actual rendering (well, as pre-rendering invokes the parser, it’s technically during the pre-rendering). The point about these filters is - they are specified per prerender-model and not per per factory. So you can apply different filters for different needs.

TDI looks if the prerender-model provides a prerender_filters() method. If it exists, it’s called without any arguments. The method is expected to return a dict, which may contain the well-known eventfilters and default_eventfilters keys. The specified filters are injected into the parser, which is set up to generate the final template tree.