Template Loading Revisited

The template loader provides methods to build template objects from any source. Additionally it provides caching and refreshing mechanisms.

From the user’s point of view template loading is a two-step process. The first step is to create a template factory which holds all parameters needed to eventually create the actual templates. The second step is to call one of the factory’s methods to pass in the template source and get the template object returned.

TDI provides a few default factory instances: → tdi.html (which is heavily used in the example scripts as well), → tdi.xml and → tdi.text. While these are not configured for advanced features like memoizing or autoreload, they serve as reasonable defaults for many cases.

Simple Template Loading

The template factory provides four methods to create a template from a single source:

→ from_string()
Takes a simple byte string as template source.
→ from_file()
Takes a file name as input. The file is opened from the disk and the contents of the file is used as template source.
→ from_stream()
This method expects an open stream object (that needs a read(size) method). The stream is read chunk by chunk until its end is reached. The read chunks are the template source. Note that the stream is not closed by the method. The stream can be anything with a proper read(size) method: an open file, a byte stream from a socket, a stream from a zipfile, and so on.
→ from_opener()
This is similar to from_stream(), but takes a stream opener. This is a function able to open the desired stream (possibly more than once). This is needed for auto-reloading on source changes.

The following script shows the basic usage in code form:

from tdi import html
from tdi import factory

template = html.from_string("""
<html>
<body tdi="body">
    some template
</body>
</html>
""")
print template.tree

template = html.from_file('loading.html')
print template.tree

stream = open('loading.html')
try:
    template = html.from_stream(stream)
finally:
    stream.close()
print template.tree

template = html.from_opener(factory.file_opener, 'loading.html')
print template.tree

Note that → from_file is a simple wrapper around → from_opener. So the last example is effectively the same as the from_file example. However, for non-file streams the from_opener() method becomes interesting.

Overlayed Template Loading

The factory offers two methods to load multiple templates at one go and overlay them immediately:

→ from_files()
Takes a list of file names as input. Each file name is passed to → from_file() to get a template.
→ from_streams()
This one is more complicated. It takes a list of opaque stream “tokens” and a streamopen function. The streamopen function is called with each of the stream tokens and returns either an open stream or a stream opener.

Here’s some code:

import tempfile
from tdi import html

file_1 = tempfile.NamedTemporaryFile()
try:
    file_2 = tempfile.NamedTemporaryFile()
    try:
        file_1.write("""<html lang="en"><body tdi:overlay="huh">yay.</body></html>""")
        file_1.flush()

        file_2.write("""<html><body tdi:overlay="huh">file 2!</body></html>""")
        file_2.flush()

        template = html.from_files([file_1.name, file_2.name])
    finally:
        file_2.close()
finally:
    file_1.close()

template.render()

This example creates two template files and loads them with the from_files() method. The template from the second file is overlayed over the template from the first one. This creates a resulting template. Now if there would be a third file, its template would be overlayed over the just created overlay result (creating a new overlay result) and so forth.

Here’s the result:

<html lang="en"><body>file 2!</body></html>

Automatic Reloading On Change

It’s often very convenient (for example, during development) if changes of the template sources are reflected by the template objects.

Template objects provide methods for recreating themselves from their sources (→ reload()) and checking themselves for up-to-dateness (→ update_available()).

The → Factory can be configured to wrap every template into a proxy object that invokes these methods on every access to the template object. That way the template is always up-to-date. Here’s how:

First of all you need to tell the factory. You can either create a new factory directly (by instanciating the → Factory class) or create a derivative factory from an existing one, using the factory’s → replace() method.

The further handling depends on the loading method. Autoupdate is only supported by the following methods:

Have a look at the following script (the wait() function is not included here for reasons of clarity):

import tempfile
from tdi import html

# 1) Tell the factory that we want automatic template updates
html = html.replace(autoupdate=True)

tfile = tempfile.NamedTemporaryFile()
try:
    tfile.write("""<html><body tdi="body">Yey</body></html>""")
    tfile.flush()

    # 2) Load the template from_file
    template = html.from_file(tfile.name)
    print template.tree

    # (... wait for low-timer-resolution systems ...)
    wait()

    # 3) Update the file
    tfile.seek(0)
    tfile.truncate()
    tfile.write("""<html><body tdi="nobody">Yup!</body></html>""")
    tfile.flush()

    # 4) Voila
    print template.tree

finally:
    tfile.close()

The script emits the current template tree as updated from modification:

/
  '<html>'
  body
    'Yey'
  '</html>'
\

...

/
  '<html>'
  nobody
    'Yup!'
  '</html>'
\

The mechanism works for combined templates, too. The whole pile of overlayed templates is checked then and changes bubble up to the final template object.

Memoizing Factory Calls

The factory can be configured to remember the results of the method calls depending on the input (i.e. they return the same template object if the arguments are the same). This technique is called memoizing. Of course, it’s only useful if it’s a longer-running script repeating to call the same methods again and again.

Usually this happens transparently. However, because of the complexity of the input parameters the memoizing needs to be triggered in a more explicit way here. In order to make it work there are two conditions to be met:

  1. The factory needs to be configured with a memoization storage container. Such a container is passed using the memoizer argument of either the factory’s → constructor or the factory’s → replace() method. The container simply needs to provide some of dict‘s methods plus an optional lock attribute.
  2. In order to actually store and remember stuff the methods themselves would need to determine a unique key from their input parameters. As said, that’s pretty complicated and next to impossible to do in a general way. The solution is to do it in a very specific way - the key is simply to be passed to the method (using the key argument).

Once the memoizer is configured and the particular key is present the methods remember their calls. Since it’s very inconvenient to pass in an extra key every time, you can wrap your factory into a proxy object, which provides the keys for you. TDI provides such a proxy class: → factory_memoize.MemoizedFactory. The keys here are basically derived from file names and stream tokens. If you need something different, take this class as an example or simply inherit from it.

Now finally, here’s some code visualizing the memoization usage:

from tdi import factory_memoize
from tdi import html

t1 = html.from_file('loading.html')
t2 = html.from_file('loading.html')

# t1 and t2 are different template objects here
print t1 is t2 # False

# 1) Tell the factory that we want memoized calls
html = html.replace(memoizer={})

# Wrap into the key provider
html = factory_memoize.MemoizedFactory(html)

t1 = html.from_file('loading.html')
t2 = html.from_file('loading.html')

# t1 and t2 are the same objects here
print t1 is t2 # True