UseModWiki | RecentChanges | Preferences

This is a brief explanation of a powerful Perl idiom that is very confusing till you get it, and is then obvious. This page describes how CliffordAdams does DeferredCompile (in v0.92). Subpage /Alternatives considers several alternative approaches.

Quick test before we go any further. In the following code

$answer = "no";
$x = "";
$x = <<'#SomeComment';
$answer = "yes";
sub fred {
print 1, $answer, 2, $x
what now is the printed value of
  1. $answer ?
  2. $x ?
If you believe that $answer = "yes" then you still have a very clever Perl idiom to learn about.

Of course, if you already know why $answer is "no", you are still very welcome to stick around to add any other magic you know about...

Let's look at the line that assigns a value to $x. The << operator is one that makes a custom string delimiter, in this case we have said the #SomeComment? on a line all by itself will be the end of the string - the full delimiter is therefore really "\n#SomeComment?\n". Also notice that the string starts on the line following the << operator.

So we set $x = "$answer=\"no\"\nsub fred {\n xxx\n}". It may be valid Perl, but the compiler doesn't know that yet, all we told it was that it is a string.

It follows that the compiler does not yet know about sub fred - if we use it in an expression our script will probably die on us.

So what if we want to call fred? We pass the string to the compiler to evaluate, after which it has not only set the new value to answer, it has compiled fred and stored it away ready for our call.

There are just two more details to explain - when we got the file out of the tarball, deferred compile was switched off. This is done by commenting the $x= <<.. line out, having preceded it with an assignment to an empty string. Now the lines follwoing are not inside a string, they are real Perl code. Now also when we eval $x, there is nothing to do.

And the final detail, the funny delimiter starts with a #, so that when it is not seen as a delimiter it is understood as a comment and does not snarl up the code.

That is how wiki.pl works. If you run it in deferred compile mode, well over half the source ends up in two strings and is not immediately compiled at all - it only gets compiled when DoWikiRequest realises it will be needed.

Hope that helps fellow newcomers to Perl --RiVer

Thanks a bunch it's a really helpful note. It would be nice if someone wrote a doc describing how to set up DeferredCompile (I guess I need to install modperl right?). Well with my new 2Ghz/2GB Ram pc I have enough horsepowers for my 2hits/week site. :-) --ElMoro

If you use modperl you do not want deferred compile of any sort, because modperl compiles it once then runs the same compiled code for each user. By deferring any compilation you just make more overhead for no extra gain. One way of thinking of this is that modperl is the exact opposite - AdvancedCompile? so that no compilation is needed while the user is waiting.

Cliff's DeferredCompile (as described above) should work on any version of Perl, as it is simply an idiomatic use of standard Perl features. Try taking out the # before $BrowseCode= (~line 300) and before $OtherCode= (~line 2442) and set $UseCache=1. --RiVer

But then why would anyone use DeferredCompile rather than ModPerl? (presumably a lot more efficient)? I mean if your hardware is in such a bad shape to need DeferredCompile then why not go the extra mile and install modperl --ElMoro

Preliminary results from timing tests - on Apache server running on an elderly Pentium:

DeferredCompile seems to serve up cached pages about five times faster than Compile-Everything-without-caching, at a cost of taking a little longer to create new pages (due to the cache flush).

After a new page the next hit on any given page needs to rebuild the cache entry, so if people are creating new pages a lot then the advantage may well be lost.

The reason for the cache flush after page creation is to avoid problems with any cached pages showing blue ? links. Rather than check each page it throws the whole cache away, leaving it to be rebuilt as people subsequently read all the pages. This is sensible in the long run as it shares the grief over many users, and pages that are never read at all do not get time wasted on them.

Editing existing pages seems about the same with/without DeferredCompile. --RiVer

moved from misspelled page DefferedCompile:

Can someone explain about DeferredCompile? I have no idea. --TakuyaMurata

Essentially, you store most of the program in a $string variable. If this part of the program is not needed, you don't compile it, thereby saving time and memory. The moment you do need it, you compile it (via do), thereby not losing any functionality. The cost is some complexity and therefore some overhead.

I see. Indeed, I tried to remove comment-out at $OtherCode, then browsing speed increased somewhat if not quite. Thanks --TakuyaMurata

I believe you only save time with languages that compile when run, like Perl, and I doubt you save memory. The byte-code used by compiled code will be smaller than the source, or I would think. In the case of UseMod, it speeds things up, but only when not running under mod_perl or something of that sort.

--- This idiom falls solidly under the headings of "clever tricks", "marginal benefit" and "considered harmful". There's a reason it's disabled by default. If you actually use the code in the string, you've wasted a lot of time and memory building it up into a string and then reinvoking the parser. No matter what, you've wasted the memory building up the string -- and do recall that's per-request in a CGI. If you don't use the code, it belongs in a separate file that should be pulled in with 'require'. The sole reason it's used in wiki.pl is to keep all the source in a single file, a goal of many "drop-in" type engines like Usemod and bloxsom, but a goal one shouldn't be shooting for in most programs. The equivalent to this idiom in modern perl is AutoLoader?.pm (or SelfLoader for those that do prefer the source in the same file). Similar to using 'require', it also requires resources only when demanded. --ChuckAdams

UseModWiki | RecentChanges | Preferences
Edit text of this page | View other revisions | Search MetaWiki
Last edited April 23, 2007 1:56 pm by MarkusLude (diff)