1.0 Document id

Copyright © 1995-2007 Jari Aalto

License: This material may be distributed only subject to the terms and conditions set forth in GNU General Public License v2 or later; or, at your option, distributed under the terms of GNU Free Documentation License version 1.2 or later (GNU FDL).

This document contains Unix Emacs Lisp programming information. It addresses the coding style of Emacs lisp and also presents notes about Emacs lisp byte compiler. Emacs Lisp code profiling is also examined and some profiling result presented.

1.1 What this document is not

This document won't introduce you to lisp, you must have basic knowledge about lisp programming beforehand: functions, local, global variables and various forms used in lisp. Mainly, this contains no ready solutions, functions, that you could use. There may be case studies though.

1.2 What is this document?

This document contains some guidelines that were found handy. There have also appeared also very good articles in the Usenet Emacs newsgroups and many good articles may have passed by, but hope you find those included interesting. It is recommended that you first read some elementary lisp reference before reading this paper.

Read this document as recommendations, not as strict rules. Adapt ideas that seem reasonable to you, and discard the others you feel, don't serve your needs.

The Elp (lisp profiling tool) results in the page are mainly for curious reader, when he needs some reference how to write tight loops, time critical functions. But normally there is not much need for optimization in Emacs: you run into performance problems very rarely. Be very skeptical when reading the results and do not put your blind trust on them.

Used Abbreviations

[jari] Jari Aalto
[kai] Kai Grossjohann
[vladimir] Vladimir Alexiev


2.0 The basic coding suggestions

2.1 Major rule1 – comment as much as possible

Someone else reading your code will appreciate any extra explanation that you may have written. Someday the code may also be maintained by someone else than you, so bear in mind that the would-be-maintainer can take over your code, when you no longer are around.

2.2 Major rule2 – maintenance comes first

Maintenance and readability comes first, never write tight code. Functions are easier to read if they are "airy" instead. Your code doesn't run any faster, no matter how much you shrink it. Some people like to delete all white spaces from their functions so that the code lines are stuck together; but that is not necessarily the best practice.

Organize parts that belong together, into groups and add dashes or anything to make it visible that something important is happening (function or condition)

2.3 Major rule3 – don't spare variables

Don't be afraid of using many variables. Especially in functions, that need local variables. A variable can "self document" the code if named properly. Use XEmacs byte compiler to check is you have defined variables that you haven't actually used, so that the byte compilation results are clean. (Note: The XEmacs byte compiler can better catch programming errors than Emacs byte compiler.)

In most cases the possible minor performance penalty of using many variables doesn't matter. See the profiling results later in this document.

2.4 Major rule4 – document your functions and variables well

Document your variables and functions well. If functions sets globals, say it in the docs (use "References:" tag). Every function and variable should have DOC-STRING, because when you do describe-symbols, it'll print out the SYMBOL and DOC-STRING. And one can even search through the doc strings with super-apropos Don't forget that the first line of the doc string should be a complete sentence.

Don't you feel frustrated too if staring at this?

      (defconst foo-list-storage nil)    

Now you have to skim the code to understand how and where the variable is used. The original Emacs recommendation has been that you do need to document package private variables, like the one above. However this recommendation is old and dates back to 18.xx days where the doc-strings memory space was a limited resource. In new Emacs releases there is dynamic byte compilation option which reduces the docstring memory consumption. There is no need to "hand optimise" the doc-strings away.

2.5 Major rule5 – don't use tricks

Think about new comers when you program, who know nothing about lisp. Try to code clearly. Avoid tricks, which is not very frienly for readers of your code. At least document well why the code at that point looks so complex.

2.6 Major rule6 – use byte compiler to check leaks

Check your code for variable leaks in fresh emacs "emacs -q" by running M-x byte-compile-file XXX.el. If possible use XEmacs's for checking because it reports warnings better.

2.7 Use the error function.

Use error command if you cannot continue, or if you think that some other program may depend on your code, it is best that the other program cannot continue. Don't try unnecerrarily handle erorr conditions - it won't work in general with Emacs Lisp, like if you have got used to Java's or C++'s throw staements.

2.8 Make function as general as possible

But not so general that it can eat apples and cars. It's "good", when the function doesn't get excessive long: still long cond statements are ok. Sometimes you just can't split the task into smaller parts or it makes no sense to split the function, oh well...use your best judgement.

Still, a long function raises always thoughts about bad coding. Usually there may be reusable parts, which can be separated, but then, perhaps no. Just make sure you are convinced you need that long function, and that's it.


3.0 Coding style issues

3.1 Function's variable definitions

Unclear code:

       (defun xx (arg1 arg2 &optional arg3 )
         (let ((foo 1) (bar "xx") baz-flag point))
          ...    

Maybe better written as:

       (defun xx (arg1 arg2 &optional arg3 )
         (let* ((foo         1)                ;temporary counter
                (bar         "xx")             ;displayed value
                baz-flag
                point)
         ...    

Tip: To "line up" variables nicely in the let statement use some package that can do it, like tinytab.el, which is tab minor mode.

3.2 Let with initialised empty values

Let's start with example code:

      (let* ((foo '())    ;; A list
             (bar nil)    ;; A truth value
             test)        ;; A scratch variable.    

This effectively causes foo, bar, test to be nil. Don't give the extra stuff fool you. The programmers intention was to clarify, that the foo is a list and by initialising it with () would signify a list context... and so on..

But it can be done it more cleanly. The more symbols there are in the view, the less easily can human eye focus to important things. Let's try this instead:

      (let* (foo-list     ;; A list
             bar-flag     ;; A truth value
             test)        ;; A scratch variable.    

In this case, the variable names itself tell where they are used, and the missing symbols greatly improve the layout. You know, that variable is nil by default, so there is no point of assigning an empty list. Less is more, in most cases. In addition, when you use these variables inside function body, it's clear all the time what that stand for because the names tell them.

3.3 There are alternatives to `progn'

progn indents code right very fast and that makes writer to code in a tight space. All code examples below give exactly the same results.

      (if variable                    ;test here
          (progn
             ...it was non-nil , do something))    

Sometimes 'cond' statement can be used similarly. It has an implicit progn form:

      (cond
       (variable                      ;test here
        ...code..
        ...code..))    

And there is also and command, but it requires that all the statements you want to execute return non-nil. This may not be exactly usable every time.

      (and variable
           ..code..
           ..code..)    

Common lisp library, cl.el offers doing the same more cleanly. This is even more nicer. Prefer this one:

      (eval-when-compile (require 'cl))

      (when variable
        ...code..
        ...code..)    

[vladimir] ...There are other even worse cases. The worst I can think of is mapcar with an inlined function:

      (mapcar (function (lambda (e)
                          (do stuff)))
            '(1 2 3))    

This leaves too few columns for (do stuff). Especially if it contains another mapcar. This is much better:

      (mapcar
       (function
        (lambda (e)
          (do stuff)))
       '(1 2 3))    


4.0 Using global variables

4.1 Thoughts on globals

Because you will be using globals a lot in Emacs lisp packages, couple of words may be in place You probably are shocked by the fact that lisp programs use globals (actually prefixed or namespace globals) all the time, when you have learned that using globals is totally wrong and should be avoided at any costs.

The class variables behave quite alike to global variables especially if the class derivation chain is long. Hm, to be strict, the scope of the variable just gets larger.

                  derived classes
      BASE        -->C1 -->C2 -->C3
      public var1               sees BASE's var1    

var1 is not real global, because it ceases to exist if the class is deleted. But, When you see code like this, where variable is "seen" outside of function, the instincts say that we should treat var1 like global. It's temptating to think that local is something that is inside function or function block, and variables outside of a function, while they may be actually packaged within class, are all "globals". Admittedly this is not very accurate distinction, but a practical point of view. In Emacs lisp, the variable scope is whole package and the variables are true globals because any other packages can see them too.

In Emacs Lisp you can also abstract the use of globals with the degree you want:

4.2 Globals and emacs lisp packages

They are traditionally used in Emacs packages for

      (defvar my-global-var t
        "Some docs come here how to use it")    

      (defvar my-collect-function 'my-default-collect-function-1
        "*There are two default choices:
         'my-default-collect-function-1
         'my-default-collect-function-2")    

      (defvar my-hash-table nil
        "Private. List of hash elements")    

4.3 Aliasing to globals – summary

To clarify: the term aliasing used in next sections doesn't mean real aliasing. The variable is not actually referred through an alias. When you work with a alias variable, you can pretend that you actually work with the global variable. The alias term is used merely for purposes where you read the global when you use it in function. You do not write to a variable aliased like this. We're actually using a copy of variable.

The next sections will describe these benefits better, but the advantages of copying the global variables are listed here for quick reference:

4.4 Aliasing to globals – counter arguments

[vladimir] ...If alias is used, the reader has to remember that `foo-mode-switch` and `switch` are the same thing. Furthermore, when you read the body of the function, foo-mode-switch is clearly a global var (perhaps a user option), while you have to look back at the 'let' in order to see that 'switch' is one. Introducing a second name to the same entity doesn't necessarily make anything more clear. There are only a few valid reasons to

      (let ((local foo-global)))
          (setq local (car local))
      ;; end    

full name seems too long. Of course, dabbrev or PC-lisp-complete-symbol will help you to write the long names, but what will help the reader to read them? of different name.

      (let ((case-fold-search nil))
         (re-search-forward foo bar)   ; respect case
      ;; end    

4.5 Referring to global variable

If any global variables is used in a function, don't use them directly, instead put them into function body let* where any anyone can see at a glance what variables are used. It also make maintaining process much simpler, since the changes have to be done into let* only. Prefer putting globals first in let.

The other benefit for the maintainer is that if he ever decides to move that global to function call parameter, the task is easy: you just lift the value from let-form to the parameter list, and you never have to touch the function body, because it uses the local variables.

      (defun my-func ()                ; #1
        (let* ((list  my-mode-alist)   ;copy
               ..other variables..)
          ..BODY..))    

You may later find out that it's actually better to call the function with list argument, so that function becomes more general. Following is the lifted global version of the previous function. Notice that the function body doesn't change in any way.

      (defun my-mode-func (list)          ; #2 , global is now an argument
        (let* (
               ..other variables..)
          ..BODY..))    

[Vladimir] also suggested that you really don't need this kind of abstraction, because converting function from #1a into #2a with function that use globals directly is as easy.

      (defun my-mode-func ()                      ; #1a
          ..BODY..
          (if (memq match my-mode-alist)
              ...)))


      ;;  Now uses paramer, not global variable my-mode-alist
      (defun my-mode-func (my-mode-alist)         ; #2a
          ..BODY..
          (if (memq match my-mode-alist)
              ...)))    

Hm. What do you think? I'd say that this is equal to mine what comes to lisp. But by using the same name in function argument list as for global variable, may makes things confusing, because my-mode-alist is originally meant to be global variable and used in other functions directly. The key point here was that we intended to make function more general, implying that we are probably moving it away from this my-mode package and including it into some general lisp library. If we move this function in #2a format, we wouldn't want to keep symbol names(variables), which refer to specific package my-mode.

Detecting reusable functions from any package is easier if the globals are presented in the first let statement.

4.6 Maintenance and deferring to global

Someone may now think in his mind:

Doesn't that make program more slower, I can avoid those private variables and let* altogether if I use globals directly.

Hm, yes and no; program won't slow down remarkably by the extra let* statement. More important is the ease of maintenance and ability to add comments beside the let statement, since all variables may not be self explanatory. If function is very small, use could those global variables directly to gain some more speed.

But if there is anything more than 10 line long function, for clarity's sake, use the alias method to hide the global from the actual body of function.

The only case where you may try to bother to optimise let* out is, when function gets called many times. Do you know that the function is dangerous to my program's performance? Probably not, that's why you sometimes use some lisp profiler (elp.el) to bog down speed problems.

4.7 Deferring to global, caution

The only exception when alias cannot be made in let is presented here. We may have to introduce a control function to read global. Suppose we have following situation.

      (defun my-foo ()
        (let* ((table my-:passwd-entries-table) ;; copy global
               point)
      ..code..
      ..code..
      (my-change-passwd-table)                    ;; OOPPS!
         (while table                             ;; error!
           ..do, read content..
           ..code..)))    

Obviously it is not possible to read the global beforehand, if it will be changed by another function call during the execution of current function.

For small amount of globals, 5-10, there is no point to make separate control function for reading global, like in this following example.

      (defun my-foo ()
        (let* (table                      ;; no global copy
               point)
      ..code..
      ..code..
      (my-change-passwd-table)            ;; Watch out!
         (setq table (my-read-passwd))    ;; a macro to read global.

         (while table                     ;; okay now..
           ..do, read content..
           ..code..)))    

The my-read-passwd, is implemented as

      (defmacro my-read-passwd ()
        "Returns contents of my-:passwd-entries-table"
          (` my-passwd-entries-table))    

Using this single macro is overkill, but it may turn into more complex function later, if you decide to use many globals. See next:

      (defsubst my-read-variable (variable)
        "Returns contents of global variables"
        (cond
         ((eq variable 'passwd)
          my:passwd-entries-table)
         ((eq variable 'user)
          my-user)
         (t
          (error "No such variable '%s'" variable))))

      (my-read-variable 'passwd)    

The function is called with symbol describing the variable. This implementation totally hides the global variables from lisp calls and from other outside functions. You must decide how strong abstraction of data you want: For small programs, you probably don't need this kind of strong abstraction, but if the program gets bigger and you start having 20-50 globals, you may consider using similar global control functions.


5.0 The 'let' forms in function

5.1 Using only one let form inside function

Nowadays many books and many programmers teach that you should define variables inside block where you need them. This is perfectly good suggestion and you should follow it in natively compiled languages. The advise for Emacs Lisp is: "use when appropriate".

Note: There is slight difference in memory usage if you define a) all variables at the beginning of function b) along the execution of your program, where they are created and destroyed. While the A may take couple of bytes overall more memory, the important point is the content of the variables. If you put 100 cons cells immediately into the variable, that's hogging memory, not the variable definition itself.

In practice don't worry about this minor memory increase, because creating and destroying a variable increases overhead for functions too (multiple let statements), so how do you choose: a small memory increase at the beginning of file where you define all my variables or do you take chance on adding a slight overhead by defining variables while the function executes? In big, complex function this could be very important issue, but in short functions the choice is insignificant.

Most of the time you can use only one let* because it helps keeping function layout clearer, while admittedly that there are very good reasons when you could consider using multiple let* statements. you can arrange the inner body of function to self standing blocks by using many let statements and introduce new let statements where logically appropriate and many lisp programmers recommend that you do so.

In C++ using block local variables is pretty nice looking.

      ..FUNCTION START
      if (var == 1)
      {
          int temp = 0                    // local to block if
          if ( condition )
          {
              int tmp = 0                 // this is again local
              ..do something..
          }
      }    

But If we do the same in Emacs lisp, the count of added parentheses may be disturbing:

      ..FUNCTION START
      (if (= var 1)
          (let ((temp 0))
             (setq tmp (1+ tmp))
             (if condition
                 (let ((tmp 0))
                    (setq tmp (1+ tmp))
                    ..do something..))))    

If we were to write real lisp (not Emacs Lisp), the inner let variables could have been be optimized to registers and you should definitely use the multiple let statements. In Emacs lisp, this kind of optimizatrion does not happen because the code is not compiled to native machine code. That's why do not worry so much if all the variables are defined at the top-level let and not inside later let statements. You won't see any noticeable performance drop if you define couple of more variables at the beginning of function. That's why you see most of the time this format.

      ..FUNCTION START
      (let ((tmp1 0)      ;; Define all used variables
            (tmp2 0))
      (if (= var 1)
          (setq tmp1 (1+ tmp1))
          (if condition
              (setq tmp2 (1+ tmp2))
              ..do something..))))    

The idea for using only one let is that functions look like simple. In one let you can see what variables are used in a function and decide is some private variable is a candidate for global.

      FUNCTION
        VARIABLES
        BODY    

While they could look like this:

      FUNCTION
        VARIABLES
        BODY
          VARIABLE
          BODY
        VARIABLES
        BODY
          VARIABLE
          BODY    

5.2 Let form and initialising variables

But while the variables can be defined without performance penalty, postpone initialisation if it takes lot of time. They are initialised only just before they are actually used.

      (defun my-func (var)
        (let* ((re1  (get-re-1))     ; scans whole file.txt, 100Meg
               (re2  (get-re-2))     ;
               tmp)
          (cond
           ((eq 'this var)
            ;; okay we're in business
            (re-search-forward re1 nil t)
            ...    

Instead write code like this, which initialises variable only when condition goes into the branch.

      (defun my-func (var)
        (let* (re1                ; NOTE - No initialisation
               re2                ; NOTE - No initialisation
               tmp)
          (cond
           ((eq 'this var)
            ;; okay we're in business
            (setq re1 (get-re-1))
            (setq re2 (get-re-2)))
            ...    


6.0 Function call arguments and return values

6.1 Using separate return value 'ret'

Before going further, remember that all lisp forms return the last value which the form executed prior its ending. This is fundament of lisp language and whole lisp programming is based on it. The key here is that you can make the function's return value more visible: the point where the return value is set is obvious. If we use extra variable, say ret, over the implicit return value, the function is a) easier to debug: you can print the ret variable anywhere b) easier to follow: setting the return value is obvious 3) and one exit point is better than "hidden".

Of course if the function is very small or extremely simple, you don't have use 'ret': return value is already obvious. Use your common sense to determine when the extra return variable ret could clarify the function and when you decide to leave it out and use lisp form's side effects of returning value of last executed statement.

      (defun my-func ()
         (let* (..
                ..)
           ... many lines of code
           (if test
              (cond
                ((= 1 var)
                 ;; IMPLICIT RETURN ))     <---  I wouldn't do this
                (..other-test
                 ;; IMPLICIT RETURN ))     <---  I wouldn't do this
             ... many lines of code
             ...)) ;; let-defun end    

Alternative choice

      (defun my-func ()
         (let* (ret
                ..)
            ... many lines of code
            (if test
              (cond
                ((= 1 var)
                 (setq ret (point)))
                ((= 2 var)
                 (setq ret ..)))
              ... many lines of code
              ...)
         ret))  ;; You can put your debugger breakpoint here    

And here are some extremely simple functions, compared to above function that would have had many lines of code. In here, the return values are clear.

      (defun a (b)
        (if b
            3))    ; 'else' case returns nil.

      (defun a (b)
        (cond
         ((...)
          1)                            ;return value
         ((...)
          2)
         (t
          3)))    

Another advantage of using ret is, that it jumps into existence with default value nil. In function body, you just set it to another value if some condition is satisfied, otherwise called receives value nil by default.

6.2 Calling functions: passing non-nil

[Andrew Fitzgibbon andrewfg@oculus.aifh.ed.ac.uk] It's common to use a descriptive symbol instead of t when passing arguments to functions. E.g.

      (directory-files "~" 'absolute "^[^.#%]")    

It's a pain then that there's only one nil when you want to default an argument, meaning that you can't easily document it. It's just occurred to me however that you can write:

      (directory-files "~" (not 'absolute) "^[^.#%]")    


7.0 Interactive functions and message display control

How should message displaying be controlled in good manner? If you print any messages, you can add variable verb to the optional parameter list. This variable should be the last element there; unless you have the &rest list of course. Now, why such an recommendation? Suppose your function is quite time consuming; eg. if it does some file handling and it may be a good idea to print some messages to the user about the progress stages.

7.1 Take one, lack of verbosity control

      (defun my-do-files ()
        (let* (...)
           (while
            (message "reading files..."))
             ... do it for 10 secs)
            (message "reading files...done"))))    

This was the traditional way to code it, because the message is always printed, no matter how the function is called: interactively or by some top level function.

7.2 Take two, some verbosity control

      (defun my-do-files ()
        (let* (...)
           (while
             (if (interactive-p)
                 (message "reading files..."))
             ... do it for 10 secs)
           (if (interactive-p)
               (message "reading files...done"))))    

Thi may be better implementation. Messages are printed only if the user has called fucntion interactively. Do you see anything to make better here? If not, let's examine one more example.

7.3 Take three, full control of verbositiness

      (defun my-do-files (&optional verb)
        (let* (...)
           (setq verb (or verb (interactive-p)))
           (while
             (if verb
                 (message "reading files..."))
             ... do it for 10 secs)
           (if verb
               (message "reading files...done"))))    

There are couple of interesting points in this solution. First, it provides verbosity to the user. Second it provides verbosity to the caller too. The idea is, that by default the function is verbose when user calls it, but it also gives the verbose messages whenever someone else calls it..

The function can now be called like this and it keeps the used nicely aware of progress:

      M-x my-do-files         --> verbose    

But the functions is recalled with the command with C-x ESC ESC followed by re-run with RET, the verbose messages are nnot printed.

This actualy makes user function easier to call, because you don't have to call them via M-x (or key binding) to get the verbose messages (like returning status, state of mode on/off). Developers can now turn on particular verbosity of some function if they think would be good to display messages to the user while function is executing.

7.4 Using call-interactively isn't always the solution.

Aha, now I hear someone claiming that the example 3 drains into this simple lisp call if verbosity is required by lisp call

      (call-interactively 'my-do-files)    

Yes, it turns on the (interactive-p) test in function, but by using this it also activates interactive part of the function. If function had the interactive part like this, it would be executed:

      (defun my-do-files (&optional verb)
        (interactive "sWhat's up doc? ")
        ..code..
        (if (interactive-p)
            (message "this"))    

Then the "What's up doc?" prompt would have been popped onto screen. The variable 'verb' is needed if the decision of printing message is given to the calling function.


8.0 Overriding functions

8.1 What is that?

Overriding means that the function exists already, but it doesn't do exactly what you want --> you want to write your own implementation which replaces the function. Sometimes there is Here are instructions how you override functions properly. If you just want to have some minor modification, then you should look at the advice.el (std emacs distribution), but to complete replace a function, you can fllow steps above.

First, make separate file, where you gather overridden functions. You use this file in next sections.

      ~/elisp/my/emacs-rc-override.el    

The body of the file looks something like this

      ;;; emacs-rc-override.el --- My implementations
      ;; Override settings of functions for xxxx

      ;; ................................................ forms ...

      <code here>

      ;; ................................................ funcs ...

      <code here>

      (provide 'emacs-rc-override)

      ;;; End of file emacs-rc-override.el    

8.2 Using the eval-after-load hook trick

Let]s start by defining our own mail-signature function which is defined in sendmail.el. First the Emacs startup file must be modified by adding this code to it:

      (eval-after-load "sendmail" '(load "~/elisp/my/emacs-rc-override"))    

Next, a function is added to replace the original. Add this code to emacs-rc-override.el after the "funcs" section:

      (defun mail-signature (atpoint)
        "My. Sign letter with contents"
         ^^^
        ...code)    

or

      (defun mail-signature (atpoint)
        "Overridden. Sign letter with contents"
        ^^^^^^^^^^^^
       ...code)    

Make sure you add some word like "My" or "Overridden" in front of the documentation string, so that when you look up the function description with M-x describe-function <func> or C-h f <func>, you don't mistakenly believe that is is standard emacs function If you overridden 1-2 functions, you may remember which ones you have rewritten, but when you start modifying emacs for your taste (I have 20-30 overridden functions), you can't remember which ones are "true" emacs functions.

Besides, if you post the solution to emacs newsgroups, people will appreciate the comment so that they get the describe-function information too. Inexperienced user's typically just copy the function from the post and if the word my is not there they may never know later that the function whether it was emacs's default or not that they're using...

Now you have the file ready and only thing left is to put one statement into your .emacs init file:

      (load "~/elisp/my/emacs-rc-override")    

This loads the file and hooks everything for you. If you later want to override some other function, you just open the ~/.emacs.o again and (say we override some Gnus functions) add this to the forms section and write the function to the funcs section in emacs-rc-override.el

          (eval-after-load "gnus" '(load "~/elisp/my/emacs-rc-override"))
                            ^^^^^    

8.3 Using advice.el to replace functions

Note: When you use advice, make sure that the original behaviour of function is preserved. You don't want to break any existing packages that may use the advised function.

This is much better way than previously presented eval-after-load method. This time you need advice.el from standard emacs distribution. Why is this better? Because advice doesn't wipe out functions permanently, you can turn them on and off when needed.

The advice has flag around that lets you do things around then function: before and after calling it. But if you don't call ad-do-it inside advice, then you have effectively replaced the function. This is what you need

      (defadvice mail-signature (around my act)
        "Replaces function."
        ...code
        (setq ad-return-value something))    

The important point here is that you say around and do not include advice macro ad-do-it in the body of function (which would call the original function). The advice is put into category my to refer to your definitions and finally it's put into immediate use: act means activate now.


9.0 Using macros

9.1 Macros introduction

Dewey M. Sasser dewey@newvision.com

Macros are (probably) the most difficult thing in LISP to understand, especially coming from a background in C or assembly. The big key in lisp is that a macro is just a function invoked by the evaluator to find out what it should really evaluate. This has two big implications:

It is not necessary (and because of feature #1, somewhat brain twisting) to call a macro from another macro.

When you write a macro, don't think of it as writing a macro, but as a function that will be called to translate the arguments (as you've specified) from the way they are to some other form. Your return value is the form to be executed instead.

For example:

      (defmacro my-setq (var value)
        (list 'setq var value))

      (macroexpand '(my-setq x y))
      ==>(setq x y)

      (defmacro msf (symbol)
        `(symbol-function (quote ,symbol)))    

But is a bit less obvious.

If you really want to hurt your brain, think about situation where you might want to do ',',form (which is valid code and I've seen it used, but never had to use it myself). You do this kind of thing when you write macros which produce other macros.

9.2 A note on lisp and "forward declarations"

Dewey M. Sasser dewey@newvision.com

Lisp does not have "forward declarations", as in some other languages. In using Lisp, you should make sure that the definition has been seen before it is used.

If you define function A using function B, but before function B has been defined, it will work, but the byte compiler may not be able to check your call to function B. Also, if B is really a macro rather than a function, it's definition must have been seen before it is used. Remember that macros are expanded by the byte-compiler and do not actually get compiled into your code. Only the results are compiled in.

Anyone who programs in Lisp a lot (and you definitely do) should have a copy of CommonLisp:_the_Language_, 2nd Edition, by Guy L. Steel. Emacs Lisp is not strictly compatible with the language it defines, but Steel's book (commonly referred to as CLtL2) is a very good reference and description of how and while. It's not a tutorial, but an annotated standard.

Whenever possible, have your macros expand to normal lisp code, the way you'd write it if you weren't using macros. Since you wouldn't write a normal function like:

      (setf (symbol-function 'my-func)
              (function (lambda (x) (do-something x))))    

don't make your macro expand to that unless there's some very good reason. If you go look at my modefn.el, where modefn::define-mode-specific-function does the real work behind a "defmodemethod" call, you'll see that what it's doing is just building the proper defun!

This has the advantage of avoiding all of the nasty byte compiler tricks necessary to have something compiled as a function (like quoting with function, for example) or other things. Also, there's really no simple work-around for defvar. You pretty much have to use a defvar form. (OK, you could work around it, but it's a lot more work.)

I think that if you forget about the code you've written so far (I know, that's difficult to do), and rewrite it using what you now know, you'll save yourself a lot of work and get better results.

9.3 Macros and autoloading

One important thing that you must remember when using macros is, that you must tell in autoload statement explicitly, that that defined symbol is a macro. Suppose following.

          library X: has 100 functions and macros
          library Y: has 100 functions only    

Now user builds his package using code from library Y and X. Sophisticated user doesn't want to slurp whole library, immediately, but he wants to instruct emacs to load functions on demand by adding autoload statements into the code.

Here is simple way to load packages

          (require 'X)
          (require 'Y)    

Slightly different way is presented below. The function y-function-this is loaded from package Y only when is is needed somewhere in the code.

          (require 'X)
          (autoload 'y-function-this "Y")    

And the bogus way would be

          (autoload 'x-macro-this    "X")   ;; Wrong
          (autoload 'y-function-this "Y")    

The last example fails, not during the byte compilation phase – it passes with flying colours, but in the middle of run time function it breaks. That's because the user forgot to mention that the the x-macro-this is a macro. What actually happened, was that in byte compiled file there is function call

          (x-macro-this)    

But the macro should have been open coded and expanded! The right way to use the autoloads is:

          (autoload 'x-macro-this    "X" 'macro)   ;; okay now
          (autoload 'y-function-this "Y")    

Tip: See tinylisp.el and command '$ A' in tinylisp-mode which creates right autoload statements from any lisp package file.

9.4 Coding defmacros and name space problem

This topic is throughly explained in the '(XEmacs lispref) Surprising Local Variables' and It is adviced that you read that section for better reference. If you have been using macros, you probably know about the dynamic scoping problem that may be occurring.

      (defmacro my-macro (&rest body)
        (let* ((counter 0))
           (while (< counter 9)
             (inc counter)
             ;; BODY sees _counter_
             (,@ body))))    

In the code above, macro's counter is visible to body and if there is also user defined "counter", then there is a serious name conflict.

One possible way to avoid this clashing is to use mangled variable names in local macros. Because the lisp is case sensitive, you can mix upper and lowercase letters to make unique variable name; the chance that body would have similar name is astronomically small. A non-clashing name could have been made from mixing the first and last characters:

      CounteR    

Another way to use unique names I learned from a post by wbrodie@panix.com (Bill Brodie), gnu.emacs.help, 23 Aug 1996. He quoted my post where I wondered where I could use make-symbol command.

> In fact I don't know any use of the command
> make-symbol...

Probably its most common use is in writing macros, to make sure that a temporary variable introduced into the macro's expansion doesn't conflict with any user variables. For example:

      (defmacro m (x)
        (let ((x-var (make-symbol "x")))
          (` (let (((, x-var) (, x)))
               ...))))    

9.5 Macro or defun

      | looking thru the advice.el code I notice this definition:
      |
      | (defmacro ad-xemacs-p ()
      |   ;;  Expands into Non-nil constant if we run XEmacs.
      |   ;;  Unselected conditional code will be optimized
      |   ;;  away during compilation.
      |   (string-match "XEmacs" emacs-version))
      |
      | and was wondering what the difference is between using `defmacro'
      | instead of `defun' when no args are used.    

[Vladimir]

The above is equivalent to nil on non-xemacs and 6 (or whatever) on emacs. The bytecompiler will compile (if nil (foo)) to nothing. If you used a function instead (or a variable), the bytecompiler would generate code to call it (fetch its value) and accordingly it will include both the emacs and xemacs variants of the code. This is slower and produces more code, however there's one significant shortcoming of the macro variant: code compiled with emacs won't work on xemacs and vice versa This makes it impossible to share .elc's one a site that has both emacsen installed.

9.6 Expanding macros

It sometimes usefull to expand the macro to really see what happend in there. Evaluate these and be amazed

      (macroexpand      '(dolist (i '(1 2)) i))
      (cl-prettyexpand  '(dolist (i '(1 2)) i))

      ;;  XEmacs 19.15 only
      (prettyexpand-sexp '(block nil))    

9.7 Macro demonstration – tutorial

Dewey M. Sasser dewey@newvision.com

As an example, here's my start at the "minor-mode" wizard (you know, ever since Microsoft started using that term, I've hated it.) This code defines a macro "make-minor-mode", which can be invoked like this:

      (make-minor-mode dewey
         "\C-cd" 'insert-dewey
         "\C-cs" 'insert-sasser)    

The above call expands to this:

      (progn
        (defvar dewey-minor-mode nil
          "Variable which controls if dewey-minor-mode is active.")

        (defun dewey-minor-mode (&optional arg)
          "Function which toggles the dewey-minor-mode"
          (setq dewey-minor-mode
              (if (null arg)
                  (not dewey-minor-mode)
                (> (prefix-nume ric-value arg) 0))))

        (setq minor-mode-alist
              (cons (cons (quote dewey-minor-mode) name)
              minor-mode-alist))

        (defvar dewey-minor-mode-keymap nil
          "The keymap for dewey-minor-mode")

        (if keymap-symbol nil
          (setq dewey-minor-mode-keymap (make-sparse-keymap))
          (define-key dewey-minor-mode-keymap "^Cd" 'insert-dewey)
          (define-key dewey-minor-mode-keymap "^Cs" 'insert-sasser')))    

9.8 Macro demonstration – code

Dewey M. Sasser dewey@newvision.com

Here is the complete macros that are used. Study them carefully.

      (defun minor-mode-variable-symbol (mode)
        "Return the symbol of the minor mode controlling variable.
      Arguement MODE is a symbol"
        (intern (concat (symbol-name mode) "-minor-mode")))

      (defun minor-mode-make-keymap-symbol (mode)
        "Return the symbol of the minor mode controlling variable.
      Arguement MODE is a symbol"
        (intern (concat (symbol-name mode) "-minor-mode-keymap")))

      (defun minor-mode-function-name (mode)
        "Return the symbol naming the minor mode function.
      PREFIX can be used to determine which function"
        (intern (concat
                 (symbol-name mode)
                 "-minor-mode")))

      (defun make-minor-mode-keymap (mode bindings)
        "Define the appropriate keymap"
        (let ((name (symbol-name mode))
              (keymap-symbol (minor-mode-make-keymap-symbol mode)))
        (list
         `(defvar ,keymap-symbol nil
            (concat "The keymap for " name "-minor-mode"))
         `(if keymap-symbol nil
            @(let (results key binding)
                (if (oddp (length bindings))
                    (error "Odd number of keys and bindings"))
                (push `(setq ,keymap-symbol (make-sparse-keymap))
                      results)
                (while bindings
                  (setq key (pop bindings))
                  (setq binding (pop bindings))
                  (push
                   `(define-key ,keymap-symbol ,key ,binding)
                   results))
                (nreverse results))))))

      (defun make-minor-mode-add-to-alist (mode)
        "Add appropriate thing to minor-mode-alist"
        (let ((name (symbol-name mode))
              (variable-symbol (minor-mode-variable-symbol mode)))
          `(setq minor-mode-alist
                 (cons
                  (cons ',variable-symbol name)
                  minor-mode-alist))))

      (defun make-minor-mode-variable (mode)
        (let* ((variable-symbol (minor-mode-variable-symbol mode)))
          `(defvar ,variable-symbol nil
             (concat "Variable which controls if " (symbol-name mode)
                      "-minor-mode is active."))))

      (defun make-minor-toggle-mode-function (mode)
        "Return the defun form to define the minor mode"
        (let* ((mode-name (symbol-name mode))
               (variable-symbol (minor-mode-variable-symbol mode))
               (function-name (minor-mode-function-name mode)))
          `(defun ,function-name (&optional arg)
             (concat "Function which toggles the "
                      mode-name "-minor-mode")
                (setq ,variable-symbol
                      (if (null arg) (not ,variable-symbol)
                        (> (prefix-numeric-value arg) 0))))))

      (defmacro make-minor-mode (mode &rest bindings)
        "Define the minor mode functions, etc"
        `(progn
           (make-minor-mode-variable mode)
           (make-minor-toggle-mode-function mode)
           (make-minor-mode-add-to-alist mode)
           @(make-minor-mode-keymap mode bindings)))    

9.9 Nesting macros

Below is a very simple demonstaration how you use toplevel to call other macros that need symbols as arguments. The toplevel expects that the variables are know by name before hand.

      ;;  some predefined variables

      (defvar my-variable1)
      (defvar my-variable2)

      (defmacro my-internal (sym)
        ;;  Example macro that needs symbol as input argument
        ;;
        (` (symbol-value (, sym))))

      (defmacro my-toplevel (variable-prefix)
        ;;  toplevel uses my-internal macro; Create symbols for calls
        ;;
        (let ((sym1 (intern
                      (concat (` (, variable-prefix)) "-variable1")))
              (sym2 (intern
                      (concat (` (, variable-prefix)) "-variable2"))))
          (`
           (,@
            (let* (ret)
              ;;  Really, nothing magic here. Since the return value
              ;;  of macro must be a list, we build up list with
              ;;  push command. To return the list in proper order
              ;;  we finally use nreverse.
              ;;
              (push 'progn ret)
              (push (` (my-internal (, sym1))) ret)
              (push (` (my-internal (, sym2))) ret)

              (nreverse ret))))))

      ;;  To check what happens when the macro is expanded

      (macroexpand ' (my-toplevel "my"))

      ;;  And this is the result:
      ;; --> (progn
      ;;       (my-internal my-variable1)
      ;;       (my-internal my-variable2))    

9.10 Code: nesting macros, complex example

[vladimir] Here's a macro to define toggle commands.

      (defmacro v/deftoggle
        (sym &optional get set comment before after message)
        "Define a function v/toggle-SYM to toggle SYM on and off.
      GET and SET are either nil in which case SYM and (setq SYM)
      are used, functions (eg default-value and set-default)
      called with SYM and SYM VAL,
      or (macro lambda (SYM) ...) and
      (macro lambda (SYM VAL) ...) respectively.
      COMMENT is additional comment for v/toggle-SYM,
      BEFORE and AFTER are lists of additional
      forms around the toggle code,
      MESSAGE is a (macro lambda (SYM VAL) ...) replacing the normal
      \"SYM is VAL.\""
        (cond ((null get) (setq get sym))
              ((symbolp get) (setq get `(,get (quote ,sym))))
              ((setq get (macroexpand (list get sym)))))
        (let ((val `(if arg (> (prefix-numeric-value arg) 0)
                            (not ,get))))
          (cond ((null set) (setq set `(setq ,sym ,val)))
                ((symbolp set) (setq set `(,set (quote ,sym) ,val)))
                ((setq set (macroexpand (list set sym val)))))
          `(defun ,(intern (concat "v/toggle-" (symbol-name sym)))
                  (&optional arg)
             (concat "Toggle " (symbol-name sym)
              ". Return the new value. With positive ARG set it,
              with nonpositive ARG reset it."
                      (if comment (concat "\n" comment)))
             (interactive "P")
             @before
             set
             @after
             (if message (macroexpand (list message sym get))
                `(message "%s is %s" (quote ,sym) ,get))
             get)))

      A simple command to toggle a var is defined like this:

      (v/deftoggle bibtex-maintain-sorted-entries)    

This executes some code after toggling the var:

      (v/deftoggle debug-on-error nil nil
        "Require 'fdb (filter out trivial errors)." nil
        ((if debug-on-error (require 'fdb))))    

This uses default-value and set-default as the get and set functions because url-be-asynchronous is buffer-local, and we need to manipulate its global value.

      (v/deftoggle url-be-asynchronous default-value set-default)    

This goes wild: it uses special get/set functions and a special message

      (v/deftoggle indented-text-mode
        (macro lambda (sym)
               '(eq major-mode 'indented-text-mode))
        (macro lambda (sym val)
               `(if ,val
                    (progn
                      (make-variable-buffer-local
                       'before-indented-text-mode)
                      (put 'before-indented-text-mode
                           'permanent-local t)
                      ;; so that kill-all-local-variables won't touch it
                      ;;
                      (setq before-indented-text-mode major-mode)
                      (indented-text-mode))
                  (if (boundp 'before-indented-text-mode)
                      (funcall before-indented-text-mode)
                    (normal-mode)
                    (if (eq major-mode 'indented-text-mode)
                        (text-mode)))))
        "Toggle the major mode between indented-text-mode
         and the normal-mode."
        nil nil
        (macro lambda (sym val) `(message "%s" major-mode)))    

9.11 Using macros to create functions

      ;; example presented by [kai]

      (defun make-multiplier (n) `(lambda (x) (* ,n x)))
      (fset 'double (make-multiplier 2))    

[Bill Dubuque wgd@martigny.ai.mit.edu] The above technique does not work to create a closure. The point of a closure is that it 'closes' over (captures) some lexically apparent bindings. The exact same binding may be shared by many different closures created in the same lexical context. If one of the closures alters the value of a closed variable, all the other closures will see the change.

E.g. one can use closures to implement data abstractions where the closed bindings essentially are state that is hidden by the abstraction. Here is a toy example that implements a counter with READ and INCREMENT methods:

      (defun make-counter (value)
        (values
          #'(lambda ()                ; READ method
              value)
          #'(lambda (increment)       ; INCREMENT method
              (setq value (+ value increment)))))

      (multiple-value-bind (counter-read counter-incf)
                           (make-counter 1)   ; value <- 1
        (funcall counter-incf 2)              ; value <- value + 2
        (funcall counter-read))               ; read value

      => 3    

Note how the same lexical binding of 'value' was captured in both the READ and INCREMENT closures returned by make-counter.

Dewey M. Sasser dewey@newvision.com comments:

Actually, I found when experimenting that the fset line is byte compiled. I suppose what that means is that byte-compiler is smart enough to treat the argument to "fset" as a function.

      (defmacro make-multiplier (n)
         (` (lambda (x) (* (, n) x))))

      (macroexpand ' (make-multiplier 2))
      ;; --> (function (lambda (x) (* 2 x)))

      (fset 'double (make-multiplier 2))    

However, if you do a

      (setq some-var (make-multiplier 2))
      (fset 'double some-var)    

I don't think it will be compiled; maybe you have to write

      (setq some-var (make-multiplier 2))
      (fset 'double (byte-compile-sexp some-var))    

Here is another possibility

      (defun make-multiplier (func-sym n)
         (let ((name (intern (symbol-name func-sym))))
           (` (defun (, name) (x) (* (, n) x)))))

       ;;;###autoload
       (` (,@ (make-multiplier 'double 2)))
       ;; --> (defun double (x) (* 2 x))    

[*Dewey* comments more]

However, while the real function is installed there, the autoload won't notice it. Autoload is a magic text thing. When the ;;;###autoload token is read, the autoload library uses (read) to read the next form. Read does not expand macros (well, only reader macros like #', and evidently ` is a reader macro that expands to the old-style (` (,a)) syntax). It the above example you get nothing (autoload should really be rewritten to be extensible).

If you know the form will expand into "blah-func", which is a function, you can use:

      ;;;###autoload (autoload 'blah-func "this-file" "docs")    

or whatever the actual call to autoload that you want.

9.12 How to read the macros

When you see some exiting new styled macro, you can convert it back to old format with trick presented by [dewey]

      (setq  foo (read (current-buffer)))
      (print foo (current-buffer)) C-u C-x C-e    


10.0 Using lambda notation – some thoughts

10.1 Clarification

lambda is same as function, it is just "anonymous" function. So everything you can do with real function, you can do with lambda.

Lisp programmers use lambda functions very often, but many times it would be better to see real functions instead. The lambdas have their places in lisp, eg. for mapcar and inside macros they are often used. But overall, lambdas are not that good.

[Vladimir] also comments: There are several important things about using anon functions:

10.2 Motivation to not to use lambdas all the time

Let's see an example. Suppose we want to add some more regexps to error identification regexp list when the compile.el gets loaded.

Bad choice:

      (eval-after-load
        "compile"
        '(progn
           ;; SGI's cc warning message
           (setq compilation-error-regexp-alist
                 (cons
                  ;; IAR C Compiler: "can.c":390  G
                  '("\n\"\\(.*\\)\":\\([0-9]+\\) +.*$" 1 2)
                   compilation-error-regexp-alist))))    

While looking perfectly valid, it has some problems. How do you post this answer to someone else? Maybe he already has used some other way and he doesn't like this approach. How do you change this setup afterwards, especially when your're experimenting with right regexps. Gosh! How do I remove the entry from the variable eval-after-load ?

In here things are simple. Easily modifiable. And easily handed to anyone else.

Possibly better choice:

      (defvar my-compile-eval-after-form
        '(progn (my-compile-setup))
       "*Form executed when file is loaded.")

      ;;  Install it

      (eval-after-load "compile" my-compile-eval-after-form)

      ;;  Define my function to handle this

      (defun my-compile-setup ()
       "Installs new regexps to compilation-error-regexp-alist"

        ;;  first save the original, defvar executes only once

        (defvar my-compilation-error-regexp-alist
                 compilation-error-regexp-alist
           "Copy.")

        ;;  Reset to default, we modify this later

        (setq compilation-error-regexp-alist
              my-compilation-error-regexp-alist)

        ;;  now we can experiment as much as we like by changing
        ;;  contents of these statements

        (setq compilation-error-regexp-alist ; SGI's cc warning message
              (cons
               ;; IAR C Compiler: "can.c":390  G
               '("\n\"\\(.*\\)\":\\([0-9]+\\) +.*$" 1 2)
               compilation-error-regexp-alist)))    

Now, there is much more code involved here, but it is more portable. Remember the rule: space is cheap, ease of use comes first. Now you can also delete the entry easily from the eval-after-form.

      (defun my-delete-eval-after-form (file form)
        "Deletes FORM for FILE form `eval-after-load-alist'"
        (delete form (assoc file after-load-alist)))

      ;; Remove my installation

      (my-delete-eval-after-form "compile" my-compile-eval-after-form)    

10.3 Putting lambdas into hooks

The same lambda talk applies to global-set-key and add-hook cases It's much cleaner to have function than the lambda. If you post the solution, people will appreciate function more than lambda solution. Let's try it this way first:

      (add-hook 'write-file-hooks
                   '(lambda ()
                     "My checkings"
                     (save-excursion
                       (goto-char (point-min))
                       (if (re-search-forward ....)
                           .. do something fancy
                          .. else))))    

Two obvious notes raise immediately: a) the indentation is disturbing, and limits more complex programming b) how do you use remove-hook for this ? Not very nice job... Turn this into a function and you're back to clear waters.

      (add-hook 'write-file-hooks 'my-write-file-hooks)

      (defun my-write-file-hooks ()
        .. whatever)    

Advantages: no more lambda, no more indentation problems, you can use remove-hook easily, and you can print the hook contents nicely with the following. If there were lambdas, the output wouldn't be so nice.

(Be in scratch buffer, make sure lisp-mode is on, write the variable and Hit the C-u .. keys behind a variable)

      write-file-hooks C-u C-x C-e    

[Vladimir] For short functions to put in hooks/define-keys, I prefer to put the function like below. Then I can remove-hook it if I need, or re-eval the above to redefine the function, and whatnot.

      ;; Defun returns the symbol just defined: the function name
      ;;
      (add-hook 'write-file-hooks
          (defun my-write-file-hooks ()
           .. whatever))
      ;; End    


11.0 Keeping your code organised

11.1 Use function separation dash lines

If you have loaded lisp packages from the net, you probably see many functions that are there "as is". Instead of just coding them there, you can ease the visibility of the functions by adding separator line before every function.

Traditional:

      (defun my-func1 ()
        (let* (...)
          (save-excursion
           ..)))

      (defun my-func2 ()
        (let* (...)
          (save-excursion
           ..)))    

More visible choice:

        ;;; ---------------------------------------------------------
        ;;;
        (defun my-func1 ()
          (let* (...)
            (save-excursion
             ..)))


        ;;; ---------------------------------------------------------
        ;;;
        (defun my-func2 ()
          (let* (...)
            (save-excursion
             ..)))    

The comments use ";;;", although ";;" would suffice when the comment is outside of function body. According to lisp commenting rules the ";;" would have parked to the left hand too. The reason is that, when every comment outside of function has ";;;", I can grep my files for "outside" comments. The ";;" style I leave to function bodies.

Few packages which might interest you, they all keep your code better organised.

folding.el
Included in latest XEmacs
uses folders {{{ }}}

tinybookmark.el
(b)ook(m)ark package "straight lines with names"
provides also X-popup for bookmarks

imenu.el
finding specific funtion, more detailed control.
Included in Emacs and XEmacs

11.2 Add autoload statements

When you're making a package, don't forget to include those important autoload directives to key functions. If your package is expected to be put through the emacs build process when update-file-autoloads will add your autoloads to loaddefs.el, and subsequent dumping of that file with emacs will make them a permanent part of the emacs executable. (Usually loaddefs.el is dumped, so simply updating it and bytecompiling it won't cause it to be loaded at startup time.) Some sysadm may decide to keep your packages permanently in his emacs installation and he can rip the autoloads from your file with M-x generate-file-autoloads (function is defined in autoload.el).

      ;;;###autoload
      (defun my-func ()    


12.0 About lisp symbol naming

Common idiom in lisp programs has been that the names contain only [-a-zA-Z] characters and so case chosen isn't generally mixed: My-Var is bad variable name. Also the traditional package definition convention has been:

      (defun  csh-mode-yyy ...
      (defvar csh-mode-xxx ...    

Here the first 'words' always specifies the package that is using the name space bucket, here csh-mode. Remember that symbol names are put to global name space, so each function and each variable must be unique.

12.1 Don't use your initials

In comp.lang.emacs, comp.emacs.xemacs and gnu.emacs.help where people are likely to post their own solution to other peoples, it seems that only few are aware of how they should name their symbols properly. The problem is that if you post code that has function name:

      fill-matched    

How do you know afterwards (when you just grab the code and save it somewhere in your .emacs or personal "snippet" lib) when you start writing code using that function, that it wasn't a Emacs distributed function or variable?

The problem arises too, if you name the functions so, that they start with your initials:

      joe-fill-matched    

Now, what's wrong with that? Well, if you're going to post such code that have lot's of functions and variables starting with prefix joe- , people get upset when they save the functions and notice that there is someone other's initials involved. They just wanted to have some general function to solve current task.

Now, when they ask help again, someone else posts his own functions and they end up gathering functions:

      joe-funcs ..
      mike-funcs ..
      bill-funcs ..    

Putting those into .emacs doesn't look pretty.

12.2 Use prefix my for private symbols

It becomes obvious that it would be nice if everybody would use common naming convention, so that the code can be handed to anybody without changes. The best way to achieve this is that people use prefix:

          my-    

To denote everything that they own: own variables own functions, maps.. Now it's very simple to post that code to someone else, and believe me everybody is happy when they receive good and clean code without someone else's initial involved. They feel that it's "my" code too, to solve "my" problems.

To extend this naming more, people should also use convention:

      my-csh-mode-do-this...    

If it has anything to do with the csh-mode.el, so in general add the word my- + possible LIB-ID if you write some special functions for elisp packages. This way you can easily find all functions related to "csh-" package with describe-symbol function (available in tinyliby.el ), including your own.

12.3 Naming variables differently?

There is still a matter of style with the variable naming. While it is possible to program like the "lisp" way, that may not be the best bet. In Emacs Lisp, variable and function names do not need to differ in any way, so it's perfectly legal to have same name for a variable and for a function and for a keymap and .. you name it.

      (defun  csh-mode ()
        ...)

      (defvar csh-mode nil
        "Mode on/off variable")    

This is both a good and a bad idea. The good part of it is, that when you're working with MODES or KEYMAPS, it's very desirable to have same name, so that you know what's going on in the code.

But in the other hand, if you're not using modes, the naming convention is .. hmpf, confusing. In practical terms it's lot more easier to look at the code if the symbol itself denotes the CLASS where it belongs. If everything looks the same, as lisp does due to its nature, it would be welcomed if there were something that separate variables from function elements.

12.4 Separate naming of variables

In Tiny Tools you have seen another convention. Some has said that it "looks ugly", "I don't like it", and admittedly it can give that impression to the reader of the code.

But managing lisp code gets complicated and hard to maintain if you don't develop some aids. A different symbol naming according to their CLASSES does help to read my code better and help maintainer to see where the variables are and where the functions are. Here is one possibility:

      (defun my-function () ..
      (defvar my-:variable 100)    

There is another benefit from this: it is now possible to grep all symbols referring to variables, and there is no false hits, It is also posisble to run a program to do the name replacement and it succeeds 100%. Variables fromcan be searched from the buffer by give my-: prefix to search engine. All in all, navigating in lisp code is much more easier.

Have you ever tried to complete the lisp symbols? It's lot nicer when you car write my-: prefix and hit lisp-complete-symbol command to give you listing of all variables. No false hits concerning functions.

Why ":" ? Well, that is familiar to a C++ and Perl programmer and The ':' character seems neutral and visible enough to be used in the code.

There are also alternative choices, like using "--", double dash to denote variables:

      (defconst my--var1 "some" "*tmp var")
      (defconst my--var2 "some" "*tmp var")
      (defconst my--var3 "some" "*tmp var")    

Note: The colon character is by default in the same syntax class as the dash, so your lisp commands like backward-sexp work as usual. You can verify this with commands

      (char-to-string (char-syntax ?:)) and
      (char-to-string (char-syntax ?-)) in lisp-mode.    


13.0 Lisp code notes

13.1 Call-interactively

      > If I define a kbd macro, and then name it `say-hi', and I
      > make the kbd
      > macro map to the letters "HI", then that macro is a command.
      >
      > (defalias 'say-hi (read-kbd-macro "HI"))
      >
      > should end up being interactive. In fact, the expression:
      > (commandp 'say-hi)
      >
      > evals to TRUE.    

Hrvoje Niksic hniksic@srce.hr, comp.emacs.xemacs, 13 Apr 1997

It is, but for a different reason. commandp returns t for interactive compiled functions, interactive lambda expressions, autoloads with fourth argument non-nil and *string and vectors*.

      (commandp [some vector])    

yields t not because [some vector] is a valid command, but because it can be callable through execute-kbd-macro or such. Documentation for commandp never guarantees that you'll be able to call-interactively the objects it blesses with t.

      > The error is when I do this:
      > (call-interactively 'say-hi)
      > I get :
      > wrong type of argument: commandp, say-hi    

That is only a lousy-stated error message. You can call macros with execute-kbd-macro.

      (defun maybe-macro-call-interactively (def &rest junk)
        "If DEF is a keyboard macro, execute it, else execute
         it as a command."
        (if (and (symbolp def)
                 (or (vectorp (symbol-function def))
                     (stringp (symbol-function def))))
            ;; looks like a macro
            (execute-kbd-macro def)
          ;; else just proceed to call-interactively
          (call-interactively def)))
      ;; End    

13.2 Condition-case vs. unwind-protect ?

Steven L Baur steve@miranova.com

unwind-protect executes the clean up forms whenever the stack is unwound by either a throw (non-local exit), or by a signal (error condition). Condition-case handles only the error condition and can be bypassed by a non-local exit.

Here's some sample code that illustrates the differences: (tested on Emacs 19.34 and XEmacs 19.15)

      (defun test-func (foo)
        "Test Function."
        (cond (foo (throw 'some-random-condition "Return Result"))
              (t (signal 'error "some-data"))))
      ;; end

      (defun wrapper-1 (foo)
        "Wrapper for test function."
        (catch 'some-random-condition
            (condition-case err
                (test-func foo)
              (error (message "Caught Error Condition")))))
      ;; End

      (defun wrapper-2 (foo)
        "Wrapper for test function."
        (catch 'some-random-condition
            (unwind-protect
                (test-func foo)
              (message "Caught Error Condition"))))
      ;; End    

If you call (wrapper-1 t), the "Caught Error..." message is never executed, but if you call (wrapper-2 t) it will be.

In the error signaling case, (wrapper-1 nil) will cause the error to be caught and never signalled up. Unwind-protect (wrapper-2 nil), the error condition does get propagated up. Since this appears to be what you want anyway, use unwind-protect.

I hope that makes it a little clearer.

13.3 Dolist

The dolist command loops through a list and it is defined int the cl package; you can stop the loop with return command. Below you see example and the expansion with cl-prettyexpand.

      (dolist (elt '(1 2))
        (if (eq elt 1)
            (return)))        ;Stop the list loop now

      (block nil
        (let ((--dolist-temp--1090818 '(1 2))
              elt)
          (while --dolist-temp--1090818
            (setq elt (car --dolist-temp--1090818))
            (if (eq elt 1) (cl-block-throw '--cl-block-nil-- nil))
            (setq --dolist-temp--1090818 (cdr --dolist-temp--1090818)))
          nil))    

macroexpand to find out the real expansion

      (cl-block-wrapper
       (catch (quote --cl-block-nil--)
         (let ((--dolist-temp--1090818 (quote (1 2))) elt)
           (while --dolist-temp--1090818
             (setq elt (car --dolist-temp--1090818))
             (if (eq elt 1)
                 (cl-block-throw (quote --cl-block-nil--) nil))
             (setq --dolist-temp--1090818 (cdr --dolist-temp--1090818)))
           nil)))    

Dave Gillespie daveg@thymus.synaptics.com comments:

Common Lisp loops use the Common Lisp block mechanism, not the catch mechanism. The Emacs CL package implements block in terms of catch, but there is a catch, so to speak.

The CL package treats block specially in order to optimize it. Catch blocks are expensive at run-time, so I wanted to make sure the compiler could eliminate them when the body code didn't actually call return. (This is especially important since many Common Lisp constructs include implicit blocks whether you use those blocks or not.)

There were technical reasons, which I don't remember exactly, why the optimization was best done in the compiler itself instead of in the block macro. Therefore, the CL package has some hacks to modify or delay the expansion of block under certain circumstances. But this will always be invisible unless you deliberately peek at the macro expansions. If you actually try using return or return-from in your code, you will find that it works properly.

13.4 Narrow-to-region – don't use it always

The narrow-to-region lisp form is great if you create functions that should do their job in restricted area. Say:

      (defun my-find-a-region-1 (beg end)
        "Find something from region BEG and END"
        (interactive "r")
        (let* ((i  0))
          (save-restriction
            (narrow-to-region beg end)
            (PMIN)
            (while (re-search-forward "a" nil t)
              (inc i)))
          (message "%d"  i)))
      ;; End    

There is also another way to write this function by not using the narrow at all. I would prefer this another alternative and avoid the narrow, because you can take advantage of the END parameter of re-search-forward.

      (defun my-find-a-region-2 (beg end)
        "Find something from region BEG and END"
        (interactive "r")
        (let* ((i  0))
          (save-excursion
            (goto-char beg)
            (while (re-search-forward "a" end t)
              (inc i)))
          (message "%d"  i)))
      ;; End    

13.5 Obarray: length and efficiency

[Jamie Zawinski jwz@netscape.com] ...A vector of length 0 can't be used as an obarray. And for performance reasons, an obarray should have a length which is prime, and which is roughly the size of the number of elements you're going to put into it; The larger the ratio of elements/length, the more time lookups will take.


14.0 Optimizing and byte compiler tips

14.1 Use eq instead of =

[24 Jan 1996, terra@diku.dk (Morten Welinder)] If you're not an Emacs wizzard you should skip these patches for now. You may benefit later. I have discovered that lots of Emacs Lisp code uses equal and = where they could have used eq or even null.

      `equal' is slow and uses a function call.
      `='     is fair but does unnecessary check if we can
              decide in advance that the arguments are integers.
      `eq'    is almost as fast as...
      `null'  ...which is best.    

Examples. Often you see something like these expressions:

      (= (point) (point-min))
      (equal 'foo bar)
      (assoc 'foo bar)
      (equal (current-buffer) buf)
      (eq arg nil)    

which from a functional (and style) point of view are perfectly ok. But they're not as efficient as they could be. The ones below are better because they use the available type information about the arguments.

      (eq (point) (point-min))
      (eq 'foo bar)
      (assq 'foo bar)
      (eq (current-buffer) buf)
      (null arg)    

14.2 About setq and set

Simon Marshall Simon.Marshall@esrin.esa.it Jan 1997 in gnu.emacs.help mentined that

...One difference not mentioned is that they are byte-compiled differently. I think

      (setq fubar foo)    

results in faster byte-code than

      (set 'fubar bar).    

14.3 About let in Emacs

[Hrvoje Niksic hniksic@srce.hr 1998-03-13 XE-L]

      (setq global 2)
      (setq real-global 3)
      (let (global)
        (setq global 4)
        (setq real-global 5))
      global
      ==> 3    

...`let' sets up an unwind-protect that remembers the old value (2), and places the new value to the symbol value slot (nil in this case). When you assign 4 to global, it's written to its value slot, overriding nil. When let is left, the internal unwind-protect restores the old value (2).

This is one of the reasons why let is extremely slow in Emacs Lisp.

14.4 Make lambda to run code faster

This all is actually explained well in the Emacs lisp pages, but let's refresh memory a bit. Let's start with the traditional example:

          (mapcar '(lambda (x) ... )   list)    

The lisp manual page in says that "(elisp, Node: Anonymous Functions) ...Lisp compiler cannot assume this list is a function, even though it looks like one". So, we have to help byte compiler by adding function directive.

          (mapcar (function (lambda (x) ... )   list)    

Which, when compiled probably speeds the code by factor 2 or more. One compatibility not to this: in Emacs 19.29 and up you can actually write like this, which is exactly the same as the using the function syntax.

          (mapcar (lambda (x) ... )   list)    

14.5 Deleting and inserting in buffer is slow

[Jerry Quinn jquinn@nortel.ca] ...I used to dump data to buffer and then moving to a column, making various changes with insert and delete-char and moving on to the next change. It would take about 22 seconds on my system

I now collect the message data into lists with regexps erase the buffer and dump the new results in with format. This is MUCH faster. (3sec compared to previous 22sec)

14.6 Byte compiler options

18 Sep 1996, Andreas Schwab schwab@issan.informatik.uni-dortmund.de answered to question below

> (defalias 'pair (symbol-function 'cons))
> (defalias 'pairp (symbol-function 'consp))
>
> The trouble is that the byte-compiler doesn't optimize a
> call to e.g. pair as it would do with a call to cons
> because it doesn't recognize pair as an alias for cons.
>
> Is there a way to tell the byte-compiler to treat
> pair the same way as cons?

      (byte-defop-compiler '(pair byte-cons) 2)
      (byte-defop-compiler '(pairp byte-consp) 1)    

14.7 Byte compiler warnings – how to get rid of them

19 Feb 1996, andersl@csd.uu.se (Anders Lindgren)

> If you have code that depends on a library that is not
> always included in a program (be it Emacs Lisp or other
> Lisp), the correct way to insure that it's compiled
> properly is to do the require. It's not overkill; after
> all, a user presumably will only compile it once. And >
> it may save you from interactions that you cannot predict
> now, e.g., when at some future time when you change your
> package or font-lock changes in a future revision of
> Emacs.

Genrally this is a good idea. Unfortualtely, when it comes to font-lock it's not. It contains a check that it is runed under a window system, and barfs at load-time if it's not. This makes it impossibel to require the package when compiling in batch mode or on a system without a window system.

I have been using a (very ugly) method where I replace statements by equivalent statements which doesn't raise the anger of the compiler:

      foo              == (symbol-value 'foo)
      (setq foo ...)   == (set 'foo ...)
                         == (funcall (symbol-funtion 'set) 'foo ...)
              (The former fools the Emacs compile but not
               the XEmacs'. The latter fools both.)
      (foo ...)        == (funcall (symbol-function 'foo) ...)    

This type of coding is specially useful when writing programs which should be able to run (and compile) under both under Emacs and XEmacs. – Anders

14.8 Inlining and byte compiler

The byte compiler is quite powerfull, but there is only handfull of people who really understand how its features can be exploited in full. Here is couple of suggestions how you could force some function to be inlined and thus save the function call, which in emacs is quite expensive (see the profiling results later and examine eg. mapcar)

Notice that

      defsubst --> Byte Compiler inlines the function automatically.    

But in case of func beeing a regular 'defun' you want to use special form inline to force inlining the code.

      (defun func (arg)
        (if arg t nil))

      (defun my (x)
        (inline (func xx)))    

See what we got:

      ;; You do not need this: (byte-compile 'my)
      ;; because disassemble does it for you
      ;;
      (disassemble  'my)

      byte code for my:
      args: (x)
      0       varref    xx
      1       dup
      2       varbind   arg
      3       goto-if-nil 1
      6       constant  t
      7       goto      2
      10:1    constant  nil
      11:2    unbind    1
      12      return    

As you saw; the func was open coded inside function my. Here is reminder from the byte compiler page:

You can also open-code one particular call to a function without open-coding all calls. Use the 'inline' form to do this, like so:

      (inline (foo 1 2 3))    ;; `foo' will be open-coded
      (inline                 ;;  `foo' and `baz' will be
        (foo 1 2 3 (bar 5))    ;; open-coded, but `bar' will not.
        (baz 0))    

You can make a given function be inline even if it has already been defined with defun by using the proclaim-inline form like so:

            (proclaim-inline my-function)    

This is, in fact, exactly what defsubst does. To make a function no longer be inline, you must use proclaim-notinline. Beware that if you define a function with defsubst and later redefine it with defun, it will still be open-coded until you use proclaim-notinline.

14.9 Inlining an interactive function, danger ahead

[Moral: do not make interactive functions defsubst] [Sample test file available: test-defsubst.el]

When I was converting some very small functions from defun to defsubst, I run in to this observation. I was wondering what inlining would do to functions that had interactive spec. Below the terms IACT refers to functon that has interactive spec; Here is th epseudo code for two functions.

      defsubst fun1
        IACT
        iact-fun1-body

      defun fun2
        IACT
        call fun1
        body
      ;; end    

Now, there is conflict, because when I byte compile fun2, we see

      defun fun2
        iact-fun1-body
        body
      ;; end    

Where the iact-fun1-body is copied "as is". And that was what I was afraid of. Because iact-fun1-body had (interactive-p) test, it gets inserted into wrong place and the whole construction isn't what I intended. Here are the results in case you're interested.

      (defsubst test (&optional arg)
        (interactive "P")
        (if (interactive-p) (message "Gotchya")))

      (defun test2 (arg)
        (interactive "P")
        (test))

      (test2 1)
      --> nothing, this is okay
      (call-interactively 'test2)
      --> "Gotchya"        << SUPRISE! That wasn't meant to happen!    

byte code for test2 reveals how the inlining happened.

        args: (arg)
       interactive: "P"
      0       constant  nil
      1       varbind   arg
      2       interactive-p
      3       goto-if-nil-else-pop 1
      6       constant  message
      7       constant  "Gotchya"
      8       call      1
      9:1     unbind    1
      10      return    


15.0 Profiling

15.1 Test envinronment

Here is collection of tests and results I made out of curiosity which way is better to code.

15.2 Elp foreword

Note, that if you time the same functions you will get different absolute timings. Nevertheless, you should get same results about the fact that which one feels fastest. The values have been taken from the Elapsed row: IT DOES NOT REPRESENT EXACT TIME SPENT in the function, because time spent depends on of operating system and current load of the Unix machine.

Strong Note: [From elp.el, Barry Warsaw] Note that there are plenty of factors that could make the times reported unreliable, including the accuracy and granularity of your system clock, and the overhead spent in lisp calculating and recording the intervals. I figure the latter is pretty constant, so while the times may not be entirely accurate, I think they'll give you a good feel for the relative amount of work spent in the various lisp routines you are profiling. Note further that times are calculated using wall-clock time, so other system load will affect accuracy too.

Keep in mind that some of the tests may be very stupid or misleading to experienced lisp programmer or to person who knows Emacs internals very well. My sincere intention has been pure curiosity. Please feel free to send any comments or corrections for the used tests cases if they are not representative enough. It is unfortunate if some test case presented here is totally bogus and someone reads it with good intention.

15.3 Using elp for timing – repeat test several times

The elp.el is great, but don't trust the first results. Sometimes the timings are totally different if you clear the list and run the tests again. Repeat your test cases at least 3 times before you derive conclusions about the performance.

In here, the harness count is mentioned; that means that the test has been repeated N times and that the most representative time values has been selected(usually average). Using elp, say 10 times to repeat the test and record the timing, should give you solid estimate what timings are right.

You can use the elp very easily via minor mode if you ftp lisp helper module: tinylisp.el. All the tests have been executed with that package in the following manner:

After the tili-elp-harness function (where you can give the prefix how many times to repeat the test set; defualt is 3) has finished the elp results are shown in separate buffer from where the average of the results can determined.

15.4 Byte compilation note

If you byte compile files, the generated code is much faster thnt what the non-byte compiled one. During byte compiling, some structures are also optimized so that while they may look different in the code, the byte code is exactly the same. This means that if you should pay attention to tests that show considerable timing differencies that probably are not optimized away.

Here are som examples where you see the effect of byte compiling Pay attention to cases 1a and 1d which show you fine example how byte compilation optimizes structs.

[_1a_] Using let in function.

Here is one long way to read byte code. If you want to byte compile expressions withing functions, you probably want to be aware of this method too.

      (setq bcode     ;; Simple let with 2 variables
            (byte-compile-sexp
              (defun foo () (let ((a 1) (b 2)) (some-call))) ))
      (disassemble bcode)    

Here shorter way to read byte code; which produces exactly the same byte code as previous one. The disassemble compiles the sexp automatically.

      (disassemble '(lambda () (let ((a 1) (b 2)) (some-call) )))

      byte code for foo:
        args: nil
      0   constant  1
      1   constant  2
      2   varbind   b
      3   varbind   a
      4   constant  some-call
      5   call      0
      6   unbind    2
      7   return    

[_1b_] Same as previous one, but using the call let*. Notice, that the only difference to previous one is the order how the variables are pushed into stack. In 1a case all the values were pushed there first and then popped in varbind. Internal stack depth is thus bigger in 1a and according to experts, that makes big let statements slightly slower than if one used let* for the same purpose.

      (disassemble '(lambda () (let* ((a 1) (b 2)) (some-call) )))

      byte code:
        args: nil
      0   constant  1
      1   varbind   a
      2   constant  2
      3   varbind   b
      4   constant  some-call
      5   call      0
      6   unbind    2
      7   return    

[_1c_] Example, where let* binds previous variables. This has same byte code as 1b.

      (disassemble '(lambda () (let* ((a 1) (b a)) (some-call) )))

      byte code for foo:
        args: nil
      0   constant  1
      1   varbind   a
      2   constant  1
      3   varbind   b
      4   constant  some-call
      5   call      0
      6   unbind    2
      7   return    

[_1d_] In the following we use multiple let stetments and the byte compiling reports that the byte code is equal to 1a. A fine example how byte compiler optimizes statements.

      (disassemble
        '(lambda ()
           (let ((a 1))
             (let ((b 2))
               (some-call) ))))

      byte code:
        args: nil
      0   constant  1
      1   varbind   a
      2   constant  2
      3   varbind   b
      4   constant  some-call
      5   call      0
      6   unbind    2
      7   return    

[_4_] Things change if there is some call between the let stetments

      (disassemble
        '(lambda ()
           (let ((a 1))
             (call1)
             (let ((b 2))
               (call2) ))))

      byte code:
        args: nil
      0   constant  1
      1   varbind   a
      2   constant  call1
      3   call      0
      4   discard
      5   constant  2
      6   varbind   b
      7   constant  call2
      8   call      0
      9   unbind    2
      10  return    

15.5 Byte compiler can optimise smartly

Let me start by and example. I was not sure what the impact of callf would be if I used it my code, so I pulled out byte compiler and dissassempled some of test defun.

The call (callf or var 0) expand to statement (let* nil (setq var (or var 0))), so I wrote three function and compared their dissassemble results: They were identical. Generated empty let statement was optimised away. This is a good sign that you can safely use cl macros.

      [_1_] The cl way

          (defun my1 () (callf or var 0))

      [_2_] The regular code writing way

          (defun my2 () (setq var (or var 0)))

      [_3_] The `callf' like it would macroexpand to

          (defun my3 () (let* nil (setq var (or var 0))))

          byte code for my[1-3 are identical:
            args: nil
          0       varref    var
          1       goto-if-not-nil-else-pop 1
          4       constant  0
          5:1     dup
          6       varset    var
          7       return    


16.0 Profiling results

16.1 Refence functions

The format of the test function was presented by [Vladimir] and from the timing you can see how much the this wrapper affects the timings measured. Because the timing is measured from the Elapsed(accumulted time) row, here are the reference times for different loop-for values: 5 and 10 that are normally used in test.

       0.32 (10)
       |      |
       |      how many times function is called (loop-for count)
       Elapsed time


      ;;  Reference function, without any extra calls
      ;;
      (defun t01 ()                   ;; 0.16(5) 0.32(10)
        (let ((i    0))
          (while (< i 1000)
              ;;
              ;;  TEST CODE IS PUT HERE
              ;;
              (setq i (1+ i)))))

      ;; function with one parameter
      ;;
      (defun t02 (list)               ;; 0.16(5) 0.32(10)
        (let ((i    0))
          (while (< i 1000)
              ;;
              ;;  TEST CODE IS PUT HERE
              ;;
              (setq i (1+ i)))))

      (when HARNESS                                   ;; 10 times
        (setq list (make-list 200 nil))
        (loop-for 0 5
          (t01) (t02 list) ))    

16.2 How to get first element?

As you can see; there is no difference between the element retrieval functions.

      (defun t1 (list)                            ;; 0.4, car
        (let ((i    0))
          (while (< i 1000)
              ;;
              (car list)
              ;;
              (setq i (1+ i)))))

      (defun t2 (list)                            ;; 0.4, nth
        (let ((i    0))
          (while (< i 1000)
              ;;
              (nth 0 list)
              ;;
              (setq i (1+ i)))))


      (defun t3 (list)                            ;; 0.4,elt
        (let ((i    0))
          (while (< i 1000)
              ;;
              (elt list 0)
              ;;
              (setq i (1+ i)))))

      (when HARNESS                               ;;10 times
        (setq list (make-list 200 nil))
        (loop-for 0 10
          (t1 list) (t2 list) (t3 list)))    

16.3 How to access last element?

The results were quite impressive. Naturally using the reverse command is slower, because it has to access each elemnt, where addressing last element directly is the fastest possible way.

      ;;  Reading the last element by counting the position.
      ;;
      (defun t1 (list)
        (let ((i    0))
          (while (< i 1000)
              ;;
              (nth (1- (length list)) list)       ;; 1.3
              ;;
              (setq i (1+ i)))))

      ;; Using the reverse command
      ;;
      (defun t2 (list)
        (let ((i   0))
          (while (< i 1000)
              ;;
              (car (reverse list))                ;; 8.0
              ;;
              (setq i (1+ i)))))

      (when HARNESS                               ;;3 times
        (setq list (make-list 200 nil))
        (loop-for 0 5
          (t1 list) (t2 list)  ))    

16.4 Which loop method is faster?

There seems to be huge diffrence between while and mapcar. probably due to function call the mapcar does every time when passing element to lambda function.

      (defun t1 (list)                            ;; 28, mapcar
        (let ((i    0))
          (while (< i 1000)
              ;;
              (mapcar '(lambda (x) nil) list)
              ;;
              (setq i (1+ i)))))

      (defun t2 (list)                            ;; 0.2, while
        (let ((i    0))
          (while (< i 1000)
              ;;
              (while list nil (setq list (cdr list)))
              ;;
              (setq i (1+ i)))))

      (when HARNESS                               ;;3 times
       (setq list (make-list 200 nil))
        (loop-for 0 5
         (t1 list) (t2 list) ))    

16.5 Adding to list fast

If I want to append things to a list, should I do it with append or with nconc or cons? So that results are comparable to each other, every function must return the list in the same order and that's why you see nreverse calls prior returning the list in some functions.

      (defun t11 ()                           ;;3.3, nconc 1
        (let ((i    0)
              list)
          (while (< i 1000)
              ;;
              (setq list (nconc list (list i)))
              ;;
              (setq i (1+ i)))
          list))   ;; (0 1 2 3 ..)


      ;;  Traditional nconc
      ;;
      (defun t12 ()                           ;; 3.3, nconc 2
        (let ((i    0)
              list)
          (while (< i 1000)
              ;;
              (if (null list)
                  (setq list (list i))
                (nconc list (list i)))
              ;;
              (setq i (1+ i)))
           list))   ;; (0 1 2 3 ..)

      (defun t21 ()                           ;; 24.0, append to end
        (let ((i    0)
              list)
          (while (< i 1000)
              ;;
              (setq list (append list (list i)))
              ;;
              (setq i (1+ i)))))

      (defun t22 ()                           ;; 0.5, append to beg
        (let ((i    0)
              list)
          (while (< i 1000)
              ;;
              (setq list (append (list i) list))
              ;;
              (setq i (1+ i)))
           (nreverse list)))  ;; (0 1 2 3 ..)

      (defun t3 ()                            ;; 0.7, list*
        (let ((i    0)
              list)
          (while (< i 1000)
              ;;
              (setq list (list* 1 list))
              ;;
              (setq i (1+ i)))
           (nreverse list)))   ;; (0 1 2 3 ..)


      (defun t4 ()                            ;; 1.0, push
        (let ((i    0)
              list)
          (while (< i 1000)
              ;;
              (push i list)
              ;;
              (setq i (1+ i)))
           (nreverse list)))    ;; (0 1 2 3 ..)

      (defun t5 ()                            ;; 0.3, cons
        (let ((i    0)
              list)
          (while (< i 1000)
              ;;
              (setq list (cons i list))
              ;;
              (setq i (1+ i)))
           (nreverse list)))    ;; (0 1 2 3 ..)


      (when HARNESS                               ;; 3 times
        (loop-for 0 5
          (t11) (t12) (t21) (t22) (t3) (t4) (t5) ))    

Wow. using append to add to the end of list is enermously slower than when compared to fastest way cons. You should only use append to add to the beginning of list.

[Vladimir]

This is expected. For every call, append traverses to the end of the list, making a copy along the way, then adds a new element at the end, then discards the old list. This may even lead to garbage collection, which can take unpredictably long.

nconc is better in that it doesn't copy the list ("doesn't cons", which means that doesn't create new conses. Cons creation is quick when the new cons is taken from the free cons list, but if that is exhausted, memory allocation should be done). However, nconc still traverses the list at every iteration.

cons just adds a new cell at the beginning. append and nconc take O(n^2/2): when the list length is l they perform O(l) operations to traverse the list. cons has amortized cost O(1) (ie constant). "Amortized" means that it may cause memory allocation and/or garbage collection every once in a while, but most of the time it won't.

16.6 How to copy a alist fast

Idea by Morten Welinder terra@diku.dk (copy-sequence minor-mode-alist) only copies the cdr structure of the list (mapcar 'copy-sequence minor-mode-alist) ought to copy the pairs in the alist `copy-alist copies' list structure and pairs: it does slightly more than we need but it is much faster.

      (defun t1 (list)                                ;; 3.5
        (let ((i    0))
          (while (< i 100)
              ;;
              (mapcar 'copy-sequence list)
              ;;
              (setq i (1+ i)))))

      (defun t2 (list)                                ;; 2.5
        (let ((i    0))
          (while (< i 100)
              ;;
              (copy-list list)
              ;;
              (setq i (1+ i)))))

      (when HARNESS                                   ;;10 times
        ;; Make '((t 1) (t 1) ..) list first.
        (setq list (mapcar '(lambda (x) (list x 1)) (make-list 100 t)))
        (loop-for 0 10
          (t1 list) (t2 list) ))    

16.7 Let: is using let* slower than let

See explanation in (benchmarks) which explains the unexpected result where let* is marginally faster.

[Vladimir]

From common sense, it wouldn't matter how you arrange your lets and how you init the vars, even if your function is called in a long loop. The function call time will still dominate the lets. If fc=100 and let=1, a second let will only add 1% to the overall time. The only time it matters is when the inner let is inside a loop, in which case it will probably pay to take it outside.

We'll find that using let inside loop (defining variable j again and again) slightly decreases the performance. Yes, only slightly, because you don't normally use 1000 let statements in your function. This would also suggest that even if you put several let statements into the function, that wouldn't be be very much slower that using just one let statement at the beginning of file.

      (t01)                               ;; 0.32, without let

      (defun t1 ()                        ;; 0.7, let
        (let ((i 0))
          (while (< i 1000)
             ;;
             (let (j) )
             ;;
             (setq i (1+ i)))))

      (defun t2 ()                        ;; 0.6 let*
        (let ((i 0))
          (while (< i 1000)
             ;;
             (let* (j) )
             ;;
             (setq i (1+ i)))))

      (when HARNESS                       ;;10 times
        (loop-for 0 10
          (t1) (t2)  ))    

It seems that there is not much difference in tested emacs. I wouldn't be that thrilled of the results, but I'd guess that let* would have been definitely slower that let. Let try with variation where let* is used for the purpose is it meant to: binding previous values's content.

      (defun t1 ()                        ;; 1.2, let
        (let ((i 0))
          (while (< i 1000)
             ;;
             (let ((a 0) (b 1) (c 1) (d 1) (e 1))   )
             ;;
             (setq i (1+ i)))))

      (defun t2 ()                        ;; 1.1 let*
        (let ((i 0))
          (while (< i 1000)
             ;;
             (let* ((a 0) (b 1) (c b) (d c) (e d)) )
             ;;
             (setq i (1+ i)))))

      (when HARNESS                       ;;10 times
        (loop-for 0 10
          (t1) (t2)  ))    

Hm. While the let* binds previous variables values to successive ones, there still doesn't seem to be a big difference. Don't pay attention to marginal 0.1 advantage which let* seems to have gained.

16.8 Let: or function arg list

Some times I only need one variable and I have a bad habbit of defining it in the function call argument list to save typing and indentation of let call. Like following.

      (setq xxx-function
        '(lambda (&optional ignore)
           (if (setq ignore (my-call-someone))
               (symbol-value ignore))))    

Above I only needed one variable, that I named ignore, and used it to record the return status of function. But does this buy anything for me? lets find out.

      (defun  t1 (&optional a) (setq a (ignore)))     ;; 0.11
      (defun  t2 () (let (a) (setq a (ignore))))      ;; 0.12

      (defun t11 ()
        (let ((i    0))
          (while (< i 100)
              ;;
              (t1)
              ;;
              (setq i (1+ i)))))

      (defun t22 ()
        (let ((i    0))
          (while (< i 100)
              ;;
              (t2)
              ;;
              (setq i (1+ i)))))


      (when HARNESS                                   ;; 3 times
        (setq hook nil)
        (loop-for 0 10
          (t11) (t22) ))    

If doesn't seem to matter much. I have just had a bad habbit and I should get rid of it.

16.9 Variables: count of them

The count of variables starts gradually affecting the performance. Decide yourself how big threath using many variables is to your function: usually there are other statements that affect the overall perfomance of the function much more. The function call alone takes considerable amount of time when compared to sole let statement.

      (defun t1 ()                        ;; 0.5
        (let ((i 0))
          (while (< i 1000)
            ;;
            (let* (a1 a2 a3) )
            ;;
            (setq i (1+ i)))))

      (defun t2 ()                        ;; 0.7 2x more variables
        (let ((i 0))
          (while (< i 1000)
            ;;
            (let* (a1 a2 a3 a4 a5 a6) )
            ;;
            (setq i (1+ i)))))

      (defun t3 ()                        ;; 0.9 3x more variables
        (let ((i 0))
          (while (< i 1000)
            ;;
            (let* (a1 a2 a3 a4 a5 a6 a7 a8 a9) )
            ;;
            (setq i (1+ i)))))

       (when HARNESS                                   ;; 3 times
        (loop-for 0 10
          (t1) (t2) (t3) ))    

16.10 Variables: using let or setq

I always question myself, does it make difference shere I set the variables value. Some times If I complex initializations I would like to declare variable (not set it) in let stament and leave the initializing after the let. This seems to indicate that using the let to set the variables is better.

      (defun t1 ()                                    ;; 2.7
        (let ((i 0))
          (while (< i 1000)
            ;;
            (let ((str1 (format "%s" "1"))
                  (str2 (format "%s" "1"))
                  (str3 (format "%s" "1"))
                  (str4 (format "%s" "1")))
            ;;
            (setq i (1+ i))))))

      (defun t2 ()                                    ;; 3.5
        (let ((i 0))
          (while (< i 1000)
            ;;
            (let (str1 str2 str3 str4)
              (setq str1 (format "%s" "1")
                    str2 (format "%s" "1")
                    str3 (format "%s" "1")
                    str4 (format "%s" "1")))
            ;;
            (setq i (1+ i)))))

      (when HARNESS                                   ;;10 times
        (loop-for 0 10
          (t1) (t2) ))    

16.11 Variables: many *setq* commands

Yes they do. Using one setq command is naturally faster than many of them. For comparision there is t0 function which does the same, but does not use setq at all.

      ;;  Reference function
      ;;
      (defun t0 ()                                    ;; 1.1
        (let ((i 0))
          (while (< i 1000)
            ;;
            (let ((a 1) (b 1) (c 1) (d 1) (e 1) (f 1) (g 1) ))
            ;;
            (setq i (1+ i)))))

      (defun t1 ()                                    ;; 1.4
        (let ((i 0))
          (while (< i 1000)
            ;;
            (let (a b c d e f g)
              (setq a 1  b 1 c 1 d 1 e 1 f 1 g 1))
            ;;
            (setq i (1+ i)))))

      (defun t2 ()                                    ;; 1.9
        (let ((i 0))
          (while (< i 1000)
            ;;
            (let (a b c d e f g)
              (setq a 1) (setq b 1 ) (setq c 1 ) (setq d 1 )
              (setq e 1) (setq f 1 ) (setq g 1 ))
            ;;
            (setq i (1+ i)))))

      (when HARNESS                                   ;;3 times
        (loop-for 0 10
          (t0) (t1) (t2)  ))    

16.12 If and cond, which is faster one?

Neither. Common sense tells you that too: this is actually a stupid test, but I was curious what elp says. From here you can see that elp.el isn't that bad if you use it for timing.

      (defun t1 ()                                    ;; 0.4
        (let ((i 0))
          (while (< i 1000)
            ;;
            (if t nil)
            ;;
            (setq i (1+ i)))))


      (defun t2 ()                                    ;; 0.4
        (let ((i 0))
          (while (< i 1000)
            ;;
            (cond (t nil))
            ;;
            (setq i (1+ i)))))

      (when HARNESS                                   ;;10 times
        (loop-for 0 10
          (t1) (t2) ))    

16.13 Concat and format commands

I have very hard time to determine which elp results would describe the average timing difference. I ran the elp test several times, but the deviation between the results were too big to give any reliable estimate. Be very skeptical.

      (defun t1 ()                                    ;; 1.2
        (let ((i 0))
          (while (< i 1000)
            ;;
            (concat "1" "2" "3" "4" "5" "6" "7" "8")
            ;;
            (setq i (1+ i)))))


      (defun t2 ()                                    ;; 1.0
        (let ((i 0))
          (while (< i 1000)
            ;;
            (format "%s%s%s%s%s%s%s%s"
                    "1" "2" "3" "4" "5" "6" "7" "8")
            ;;
            (setq i (1+ i)))))

      (when HARNESS                                   ;; 5 times
        (loop-for 0 10
          (t1) (t2) ))    

16.14 Using *if* before running run-hooks

I just wonder if it makes sense to test hook contents before running it. Why should I call function run-hooks if there is nothing in a hook? From the results point of view, the there is small time difference: we prevent a function call to run-hooks.

      (defun t1 ()
        (let ((i 0))
          (while (< i 1000)
            ;;
            (run-hooks 'hook)                         ;;0.83
            ;;
            (setq i (1+ i)))))


      (defun t2 ()
        (let ((i 0))
          (while (< i 1000)
            ;;
            (if hook (run-hooks 'hook))
            ;;
            (setq i (1+ i)))))

      (when HARNESS                                   ;; 0.41
        (defconst hook nil "temp hook variable")
        (loop-for 0 10
          (t1) (t2) ))    

16.15 Returning boolean or *data* from function

Supposes you have some data in some variable, but you wonder does ot make a difference to return that data to calling program or just plain boolean t or nil

Consider that we have some string data that we could return to mean True value, or success. The t2 changes the final return value to boolean.

      ;; Remark: this is actually not a very good test set.
      ;;
      (defun  t1 () (let ((ret a)) (setq ret a) ret))
      (defun  t2 () (let ((ret a)) (setq ret a) (setq ret t) ret))

      (when HARNESS                               ;; 3 times
        (setq a (make-string (* 2 80) ?a))
        (loop-for 0 500
         (t1)                                     ;; 0.13
         (t2)))                                   ;; 0.13    

No, it doesn't seem to make any difference, so we just return anything we have already in the variable.

[Vladimir] ...There is nothing that could slow down the t1 function, because returning the variable does not make copy of it, it only delays garbage collection of that structure for a while.

16.16 Calling length or using len variable

If you use the length of list in many places, calculating it every time with length function decreases performance considerably.

      (defun t1 ()                                   ;; 2.3
        (let ((i 0))
          (while (< i 1000)
            ;;
            (progn  (length list) (length list) (length list))
            ;;
            (setq i (1+ i)))))

      (defun t2 ()
        (let ((i 0)
              (len (length list)))                    ;; 0.5
          (while (< i 1000)
            ;;
            (progn  len len len)
            ;;
            (setq i (1+ i)))))

      (when HARNESS                                   ;; 3 times
        (setq list (make-list 100 nil))
        (loop-for 0 10
          (t1) (t2) ))    


17.0 Xemacs and Emacs compatibility

17.1 About compatibility

During development of my packages I run into many incompatibities not only between Emacs and XEmacs, but also between Emacs version. If you care to write XEmacs and Emacs compatible code without hashless, I'd recommend using fucntions from my main library: they offer transparent interface to certain Emacs and XEmacs specific features. See these libraries and funcktions

      tinylib.el   -- functions ti::xe-*
      tinylibxe.el -- XEmacs and Emacs emulation library    

17.2 Overlay and extent compatibility

Good news! XEmacs 19.15 now has package overlay.el which mimics the calls of Emacs overlay functions. This means, that you no longer need to try to accomodate both Emacs(overlay) and XEmacs(extent) commands into your code. Following is enough to make your overlay code work in XEmacs.

      (eval-and-compile
        (if (xemacs-p)
            (load "overlay")))    

17.3 Portable Menus

Don't use Emacs specific menus, but see easymenu.el and compose your menus with it. Below you see a very simple minor mode and it's menu definition. The menu appears when the minor mode is turned on and disappears when the minor mode is turned off (at least in Emacs). Pay attention to the Selection 3 that can be enabled and disabled on the fly.

Note: The easymenu's enable/disable choice is buggy in Emacs 19.28 - 19.34 (in non-windowed mode), so if the progn tests at the end of file fail, don't mind that. Newer Emacs releases have fixed the problems.

      ;; test.el -- Just sample .el file

      (require 'easymenu)
      (require 'cl)


      (defconst my-map (make-sparse-keymap))
      (defconst my-menu nil)
      (defconst my-mode nil)
      (defvar   my-flag nil)

      (define-key my-map "\ez1" 'my-1)
      (define-key my-map "\ez2" 'my-2)
      (define-key my-map "\ez3" 'my-3)

      (easy-menu-define
         my-menu
         (if (boundp 'xemacs-logo) nil (list my-map))
         "My test menu"
         (list
          "My Menu"
          ["Selection 1" my-1 t]
          ;;
          ;;  X window note:
          ;;  Works in 19.14, but not in Emacs.
          ;;
          ;;  This is a bug in 19.34 but will be
          ;;  corrected to later release
          ;;
          ["Selection 2" my-2 nil]
          ;;
          ;;  let's try something more fancier here.
          ;;
          ["Selection 3" my-3 (get 'my-menu 'menu-flag) ]))


      ;;  Add mode to minor mode list
      ;;
      (unless my-flag                         ;Add only once
        (setq my-flag t)
        (push (cons 'my-mode my-map)  minor-mode-map-alist))

      (defun my-1 () (interactive) (message "1"))
      (defun my-2 () (interactive) (message "2"))
      (defun my-3 () (interactive) (message "3"))


      ;;   Toggle mode and add the menu, not the menu is available
      ;;
      (setq my-mode nil)
      (setq my-mode t)
      (easy-menu-add my-menu)

      ;; X window note:
      ;; Trying this does not enable choice "3" in XEmacs 19.14
      ;; In Emacs 19.30 it works ok.
      ;;
      ;; (progn (put 'my-menu 'menu-flag t) (force-mode-line-update))
      ;; (progn (put 'my-menu 'menu-flag nil) (force-mode-line-update))


      ;; Another Test, you need this in XEmacs, but not actually
      ;; in Emacs. --> Use it for portability.
      ;;
      ;; (easy-menu-remove my-menu)

      ;; end of code    

17.4 List of simple incompatibilities.

Here is a small list of functions that do not work in both emacs versions.

      (defvar XXX-package-load-hook nil "")
      ..code..
      (run-hooks 'XXX-package-load-hook)
      ;; End of package XXX.el

      XEmacs 19.15 and 20.1 does have this form though.    

17.5 Hash table compatibility

Hrvoje Niksic hniksic@srce.hr 17 May 1997 comp.emacs.xemacs

Use CL package's hash function which are compatible with Common Lisp and GNU Emacs. They use XEmacs hashtables on XEmacs and emulate CL hashtables on GNU Emacs.

      (let ((foo (make-hash-table :test 'equal)))
        (setf (gethash "David" foo) 'cool)
        (setf (gethash "Hrvoje" foo) 'wow)
        ...

        foo
          => #<hashtable 2/29 0x1ef7>    

Now, if you want to dump the hash-table anywhere, the simplest thing to do is dump it to a list. For example, your program crunches data in and out of hashtable for

      (let (alist)
        (maphash (lambda (key val)
                   (push (cons key val) alist))
                 foo)
        alist)
        => (("Hrvoje" . wow) ("David" . cool))    

There you have all your entries in alist, which you can print, save to file etc. All of this is, of course, much faster than if you had used an alist all the time, since the search time would have been O(n) instead of much better hashtable characteristics.

17.6 Character handling changes

If you had any character tests in your code, it will likely break in XEmacs20 and Emacs20, where a single integer does no longer present a charcter code. Beware especially contructs where you read characters directly and test the input:

      (setq ch (read-char))
      (if (memq ch '(?y ?Y))
          ...    

That will no longer work as expected. Also if you have test like this

      (if (eq (following-char) ?.)
          ...    

Those will fail also because you can't use old operators like eq. In my latest 'm' library there is emulation for some of the following functions that are from XEmacs20's documentation. The above example can now be converted into

      (require 'tinylibm)
      (if (char-in-list-case ch '(?y ?Y))

      (if (char= (following-char) ?.)    

And the code will work in every Emacs 19.28+, XEmacs 19.14+.

17.6.1 Characterp: (object), XEmacs20

t if OBJECT is a character. Unlike in FSF Emacs, a character is its own primitive type. Any character can be converted into an equivalent integer using char-to-int. To convert the other way, use int-to-char; however, only some integers can be converted into characters. Such an integer is called a char-to-int; see char-int-p.

Some functions that work on integers (e.g. the comparison functions <, <=, =, /=, etc. and the arithmetic functions +, -, *, etc.) accept characters and implicitly convert them into integers. In general, functions that work on characters also accept char-ints and implicitly convert them into characters. WARNING: Neither of these behaviors is very desirable, and they are maintained for backward compatibility with old E-Lisp programs that confounded characters and integers willy-nilly. These behaviors may change in the future; therefore, do not rely on them. Instead, use the character-specific functions such as char=.

17.6.2 Char-int (ch) XEmacs20

– a built-in function. Convert a character into an equivalent integer. The resulting integer will always be non-negative. The integers in the range 0 - 255 map to characters as follows:

      0 - 31      Control set 0
      32 - 127    ASCII
      128 - 159   Control set 1
      160 - 255   Right half of ISO-8859-1    

If support for Mule does not exist, these are the only valid character values. When Mule support exists, the values assigned to other characters may vary depending on the particular version of XEmacs, the order in which character sets were loaded, etc., and you should not depend on them.

17.6.3 Char-to-int: (ch) XEmacs20

– a built-in function. Convert a character into an equivalent integer. The resulting integer will always be non-negative. The integers in the range 0 - 255 map to characters as follows:

      0 - 31          Control set 0
      32 - 127        ASCII
      128 - 159       Control set 1
      160 - 255       Right half of ISO-8859-1    

If support for Mule does not exist, these are the only valid character values. When Mule support exists, the values assigned to other characters may vary depending on the particular version of XEmacs, the order in which character sets were loaded, etc., and you should not depend on them.

17.6.4 Int-to-char: (integer) XEmacs20

– a built-in function. Convert an integer into the equivalent character. Not all integers correspond to valid characters; use char-int-p to determine whether this is the case. If the integer cannot be converted, nil is returned.

17.6.5 Char-int-p: (object) XEmacs20

– a built-in function. t if OBJECT is an integer that can be converted into a character. See char-to-int.

17.6.6 Char-equal function (c1 c2 &optional buffer) XEmacs20.0

– a built-in function. Return t if two characters match, optionally ignoring case. Both arguments must be characters (i.e. NOT integers). Case is ignored if case-fold-search is non-nil in BUFFER. If BUFFER is nil, the current buffer is assumed.

17.6.7 Char= (c1 c2 &optional buffer) XEmacs20.1

– a built-in function. Return t if two characters match, case is significant. Both arguments must be characters (i.e. NOT integers). The optional buffer argument is for symmetry and is ignored.


This file has been automatically generated from plain text file with t2html
Last updated: 2008-08-09 18:07