Evan Lenz

I was reading in Chapter 6 of Practical Common Lisp about dynamic (a.k.a. “special”) variables and it hit me that this is something that I have longed for in XSLT.

I generally favor using template rules, and hence xsl:apply-templates, over xsl:for-each whenever possible. But sometimes it’s a pain—particularly when you have to pass parameters around.

An example using <xsl:for-each>

Take this contrived piece of code, for example:

<xsl:template match="foo">
  <xsl:variable name="x" select="string(@x)"/>
  <xsl:variable name="y" select="string(@y)"/>
  <xsl:for-each select="document('some-doc.xml')//bar">
    <my-element x="{$x}">
      <xsl:for-each select="document('another-doc.xml')//bat">
        <another-element y="{$y}"/>
      </xsl:for-each>
    </my-element>
  </xsl:for-each>
</xsl:template>

Okay, not so bad. But it’s inflexible.

The template rule version

If I wanted to convert this to a solution using template rules, I’d end up with this monstrosity:

<xsl:template match="foo">
  <xsl:variable name="x" select="string(@x)"/>
  <xsl:variable name="y" select="string(@y)"/>
  <xsl:apply-templates select="document('some-doc.xml')//bar">
    <xsl:with-param name="x" select="$x"/>
    <xsl:with-param name="y" select="$y"/>
  </xsl:apply-templates>
</xsl:template>

<xsl:template match="bar">
  <xsl:param name="x"/>
  <xsl:param name="y"/>
  <my-element x="{$x}">
    <xsl:apply-templates select="document('another-doc.xml')//bat">
      <xsl:with-param name="x" select="$x"/>
      <xsl:with-param name="y" select="$y"/>
    </xsl:apply-templates>
  </my-element>
</xsl:template>

<xsl:template match="bat">
  <xsl:param name="x"/>
  <xsl:param name="y"/>
  <another-element y="{$y}"/>
</xsl:template>

UGH! It’s bad enough to have to pass parameters around. Add in XSLT’s syntax for parameter-passing and it becomes excruciating. So… in this case, the solution using <xsl:for-each> has to be preferred, because the variables stay within lexical scope and require no parameter-passing. The only reason to go with template rules is when you really do need the flexibility afforded by the latter solution (e.g. enabling a template rule to be overridden, etc.).

Using a global variable is often the solution (and a perfectly acceptable one), but in this case, the values may vary for each <foo> element that is processed, and I don’t know upfront how many <foo> elements there will be. So that won’t work here.

Enter dynamic variables. As explained in the aforementioned chapter on Lisp, a dynamic variable is a global variable that can have multiple bindings. You have the option of shadowing it in a given lexical context and—here’s the kicker—for the full extent of any calls you make to other functions. For XSLT, just replace “functions” with “templates”.

The fanciful template rule version

XSLT doesn’t have such a thing, but if it did, it might look something like this:

<xsl:variable name="x" special="yes"/> <!-- made-up "special" attribute -->
<xsl:variable name="y" special="yes"/>

<xsl:template match="foo">
  <xsl:let name="x" select="string(@x)"> <!-- made-up "let" element -->
    <xsl:let name="y" select="string(@y)">
      <xsl:apply-templates select="document('some-doc.xml')//bar"/>
    </xsl:let>
  </xsl:let>
</xsl:template>

<xsl:template match="bar">
  <my-element x="{$x}">
    <xsl:apply-templates select="document('another-doc.xml')//bat"/>
  </my-element/>
</xsl:template>

<xsl:template match="bat">
  <another-element y="{$y}"/>
</xsl:template>

To me, this would be ideal. No parameter-passing required.

XSLT 2.0 offers a very good compromise

XSLT 2.0 introduces what are called “tunnel parameters”. I’ve known about these for a while, but my new acquaintance with dynamic variables in Lisp is helping me understand where these fall in the spectrum of programming language features. If the above example shows what a “global dynamic variable” looks like, then tunnel parameters represent what you would call a “local dynamic variable”. So here’s the bona fide XSLT 2.0 version of the above, without resorting to any made-up syntax:

<xsl:template match="foo">
  <xsl:apply-templates select="document('some-doc.xml')//bar">
    <xsl:with-param name="x" select="string(@x)" tunnel="yes"/>
    <xsl:with-param name="y" select="string(@y)" tunnel="yes"/>
  </xsl:apply-templates>
</xsl:template>

<xsl:template match="bar">
  <xsl:param name="x" tunnel="yes"/>
  <my-element x="{$x}">
    <xsl:apply-templates select="document('another-doc.xml')//bat"/>
  </my-element/>
</xsl:template>

<xsl:template match="bat">
  <xsl:param name="y" tunnel="yes"/>
  <another-element y="{$y}"/>
</xsl:template>

With tunnel parameters, you don’t have to declare a parameter unless you need access to it. And the only time you have to explicitly pass the parameters is when you initialize them. After that, you can just trust that they will be there for you when you do need them, later on down the call stack.

Since this is just a simple example (with only two template rules being invoked), it doesn’t do justice to how helpful tunnel parameters actually are. Imagine 10 more template rules that get invoked between the first and last one. Most of those won’t require any xsl:param declarations. In other words, the final example above is much better than it looks. Conversely, the first template rule example above (the XSLT 1.0 solution) is much, much worse than it looks. Imagine having to type out <xsl:param>or <xsl:with-param> 40 times in a row. That’s essentially what you’d have to do.

Conclusion

I may actually write some programs in Lisp at some point, but so far the insights I’m getting into XSLT alone are worth the effort of learning it.

Wouldn’t it be nice to be able to interact with Lisp sub-expressions? I should be able to, say, hover over a sub-expression (having perhaps set some default input data) and see the dynamic result of that sub-expression. That would make it easier to understand expressions, debug them, etc. Maybe the REPL is all that’s really necesary and I just haven’t learned it well enough yet. But it would be really handy to interactively explore the list transformations that happen as an expression (that I’ve already typed out) is composed. I couldn’t imagine a faster way to explore it than by hovering over sub-expressions and seeing the result in another window.

Has anyone built anything like this?

From the same Lisp tutorial I’m reading:

This “back-quoting” technique is a great feature in Lisp- it lets us write code that looks just like the data it creates. This happens frequently with code written in a functional style: By building functions that look like the data they create, we can make our code easier to understand and also build for longevity: As long as the data doesn’t change, the functions will probably not need to be refactored or otherwise changed, since they mirror the data so closely. Imagine how you’d write a function like this in VB or C: You would probably chop the path into pieces, then append the text snippets and the pieces together again- A more haphazard process that “looks” totally different from the data that is created and probably less likely to have longevity.

This is one of the things I’ve always liked about XSLT. The template rules directly show the XML that is being created. Attribute value templates (AVTs) are helpful in this regard too. In fact, I try to use them whenever possible. I try to avoid using xsl:attribute (unless there’s no other way to do it, i.e. when the attribute’s presence is conditional). For example, when you have to use an instruction to get an attribute’s value, this may seem the most natural approach:

<para>
  <xsl:attribute name="lang">
    <xsl:apply-templates mode="get-lang-value" select="."/>
  </xsl:attribute>
  ...
</para>

But I far prefer this approach even if it does take a few more characters:

<xsl:variable name="lang">
  <xsl:apply-templates mode="get-lang-value" select="."/>
</xsl:variable>
<para lang="{$lang}">
  ...
</para>

That way, when reading the code to figure out what it does, your eye can just skip by the variable definition (it’s just a variable; it doesn’t do anything). After that, it’s easy to see what we’re creating. The first example requires you to look more closely and to think about it for a second, ugh.

Now this is just a small version of a much bigger problem with understanding XSLT stylesheets, especially ones that use a lot of template rules. They can be so dynamic that you absolutely must see an example source document (input data) to understand what the stylesheet does. Integrating dynamic execution profiles into development via multiple code views and software visualization—those are some of my research interests…

I took a detour from Seibel’s book to try out this neat little Head First-like Lisp tutorial.

In it, there’s some sample data:

(setf *map* '((living-room (you are in the living-room of a wizards house. there is a wizard snoring loudly on the couch.)
                           (west door garden)  
                           (upstairs stairway attic))
              (garden (you are in a beautiful garden. there is a well in front of you.)
                      (east door living-room))
              (attic (you are in the attic of the wizards house. there is a giant welding torch in the corner.)
                     (downstairs stairway living-room))))

And a function for accessing part of that data:

(defun describe-location (location map)
  (second (assoc location map)))

The second function grabs the second item in the list returned by (assoc location map). What bothers me initially is that the order in which I have to write this seems to be the reverse of the order in which I would think through the problem. For example, if I wanted to get the description of the living room, I would think “First, get the *map*. Then, look inside it for the living room. Finally, grab the second item in the living-room list.” This is the way it works in XPath. If the sample data was XML, then an XPath expression for getting the living room description might look like this:

/map/*[@name=$location]/*[2]

Start with the map, then get location, then get the second item inside that. (Also, one nice thing about XPath 2.0 is that path expressions are generalized for function application. You can write get-map()/get-location()/get-second-item(), which works the same way in this case because each step returns a 1-item list.)

Anyway, I’m sure there’s probably another way to write such Lisp expressions that doesn’t require me to think backwards and—worse yet—keep all the steps in my head at once. Maybe an object-oriented approach is all that’s needed a la map.getLocation().getSecond() (like in Java). Short of that, I could always bind temporary results to variables and build up the function calls from that. But I really don’t like being forced to create variables just so I can write the code in the order that seems natural.

Maybe I just need to reverse my thinking. Maybe this has something to do with what Paul Graham refers to as “bottom-up programming”. I don’t know.

Speaking of the Head First series, Kathy Sierra just posted “The Clueless Manifesto”. In the spirit of that, I intend to ask even more dumb questions as I continue to learn Lisp. I may want to teach this stuff some day, and this will be a record of what it was like for me when I was a beginner.

In bed this morning, I was contemplating all I learned about Lisp last night. XSLT’s most obvious analog to Lisp macros is its ability to generate stylesheet code, since XSLT is in XML syntax.

But if data is really code, then XSLT template rules are also like macro definitions. An element in the source document is processed and thereby expanded, causing the code in the body of the corresponding template rule to be executed.

At one level, template rules seem more powerful than Lisp macros. A macro is defined by a name and resolved via name-lookup, whereas a template rule is defined by a pattern and resolved via pattern-matching. I don’t think I’ve encountered this level of indirection yet in my two days of learning Lisp. I’m curious what the best analog to this pattern-based function resolution is in Lisp. Maybe the best analog is to be found in particular Lisp functions, such as remove-if-not. Perhaps XSLT’s core processing model is like just another Lisp function.

An obvious difference is that Lisp macros are expanded recursively, whereas template rules are just applied once. I’ve wanted a way to apply them recursively sometimes (well, certain template rules in my stylesheet if not all of them). This isn’t built in to XSLT. Either you have to store a temporary result in a variable and then process the results again, or you have to define a pipeline process outside of XSLT to invoke the XSLT processor. I’ve contemplated how a more automatic recursive application of template rules might look or might be built into an XSLT-like language.

Anytime you learn something new, you’re going to compare it to what you already know. So far, it seems like having an XSLT background is going to be helpful in my learning of Lisp.

Got Aquamacs, Allegro CL, and SLIME up and running on my iMac at home. I can’t believe I’ve already been using Vim for 5 years now. Emacs is certainly another world. But I am motivated one way or another to learn Lisp, and learning Emacs seems to go hand in hand with that. So far, Viper mode is my friend. But I’ve already gone so far as to swap my Capslock and Control keys, as recommended in “Effective Emacs”.

So far, I’ve worked through chapter 3 of Peter Seibel’s book and am now reading chapter 4. That macros are a way of life in Lisp is exciting. I’ve gotten a glimpse of their power with XSLT, but I’m looking forward to taking it to the next level.