http://www.pmsf.de/resources/lisp/expat.html

I have written a basic C wrapper for libmxl2, which parses an XML
document, and prints a list containing the parsed data.  At the moment,
I save the printed sexp to a file, and READ it from Lisp.  I am thinking
it may be better to avoid the intermediate printing and READing, and
instead save the parsed data to a C struct; I can then try to use Lisp
to get the sexp from the C struct, using an FFI.  What is a good way to
store a sexp in C?  Is it common practice to pass through an
intermediate file like I am doing now?

TIA,
Raghavendra.

PS: In case it's relevant, the C wrapper takes an XML document like this

--8<---------------cut here---------------start------------->8---
<?xml version="1.0" encoding="utf-8"?>
<?draft omit="author"?>
<doc xmlns="urn:example:ns:doc"
     xmlns:kw="urn:example:ns:keyword"
     xmlns:xi="http://www.w3.org/2001/XInclude"
     id="sexp" lang="en-GB">
  <?latex class="article" fontsize="12pt"?>
  <docinfo>
    <author>
      <firstname>Schöne</firstname>
      <surname>Grüße</surname>
    </author>
    <title>An Example Article</title>
    <titleabbrev role="running-title">Example art</titleabbrev>
    <keywordset>
      <keyword kw:type="ai">an articleinfo keyword</keyword>
      <keyword kw:type="ai">another articleinfo keyword</keyword>
    </keywordset>
  </docinfo>
  <xi:include href="urn:publicid:-:EXAMPLE:DOCUMENT+Section:EN"/>
</doc>
--8<---------------cut here---------------end--------------->8---

and prints a sexp, which when READ by CLISP looks like this

--8<---------------cut here---------------start------------->8---
((DRAFT (PROCESSING-INSTRUCTION "omit=\"author\""))
 (DOC (XML-NAMESPACE "urn:example:ns:doc") (ID "sexp") (LANG "en-GB")
  (LATEX (PROCESSING-INSTRUCTION "class=\"article\" fontsize=\"12pt\""))
  (DOCINFO (XML-NAMESPACE "urn:example:ns:doc")
   (AUTHOR (XML-NAMESPACE "urn:example:ns:doc")
    (FIRSTNAME (XML-NAMESPACE "urn:example:ns:doc") "Schöne")
    (SURNAME (XML-NAMESPACE "urn:example:ns:doc") "Grüße"))
   (TITLE (XML-NAMESPACE "urn:example:ns:doc") "An Example Article")
   (TITLEABBREV (XML-NAMESPACE "urn:example:ns:doc") (ROLE "running-title")
    "Example art")
   (KEYWORDSET (XML-NAMESPACE "urn:example:ns:doc")
    (KEYWORD (XML-NAMESPACE "urn:example:ns:doc")
     (TYPE (XML-NAMESPACE "urn:example:ns:keyword") "ai")
     "an articleinfo keyword")
    (KEYWORD (XML-NAMESPACE "urn:example:ns:doc")
     (TYPE (XML-NAMESPACE "urn:example:ns:keyword") "ai")
     "another articleinfo keyword")))
  (SECTION
   (SECTIONINFO
    (KEYWORDSET
     (KEYWORD (TYPE (XML-NAMESPACE "urn:example:ns:keyword") "si")
      "a sectioninfo keyword")))
   (TITLE "A Section")
   (PARA
    "This is a paragraph in a section.  It does not quote any one,
  so we don't need any `\"' characters.")
   (PARA "The canonical URI for this document is "
    (ULINK (URL "http://www.example.org/sexp"))
    ".  Please check there for the
  latest updates.  Meanwhile, here is Äñ example of
  character entity references."))))
--8<---------------cut here---------------end--------------->8---

* Kaz Kylheku <20090615105137....@gmail.com> :
Wrote on Wed, 3 Jun 2009 20:09:22 +0000 (UTC):

| On 2009-06-03, N. Raghavendra <ra...@mri.ernet.in> wrote:

|> I have written a basic C wrapper for libmxl2, which parses an XML
|> document, and prints a list containing the parsed data.  At the moment,
|> I save the printed sexp to a file, and READ it from Lisp.  I am thinking
|> it may be better to avoid the intermediate printing and READing, and
|> instead save the parsed data to a C struct; I can then try to use Lisp
|> to get the sexp from the C struct, using an FFI.  What is a good way to
|> store a sexp in C?  Is it common practice to pass through an
|> intermediate file like I am doing now?
|
| Why don't you just use an XML processing library for Common Lisp?

I'm sure there would be valid reasons to avoid depending on the recent
offerings of the free-common-lisp-library vendors in this department.

| It seems like your wrapper is a waste of effort.

If you looked at his posted example carefully. libxml2 has done extra
processing on his document: libxml2 has expanded this directive

  <xi:include href="urn:publicid:-:EXAMPLE:DOCUMENT+Section:EN"/>

and the results were included in the sexp output.

I'd imagine the OP would prefer to use libxml2 for its features.

To answer the OP, I'd expect that it may not be worth the effort to
construct lisp sexps directly from C code.

The next easiest thing would be to prepare a stream in C which lisp
could call READ on directly. But CL:READ would be likely be the slowest
step; I noticed you are dealing with UTF-8 strings and so any clever
speedups from munging 8bit-clean character arrays between C and lisp
will not be possible and you'd go through the extra copying/stream
layers in your lisp.

Are there any other characterestics of your XML data that you want to
avoid a file layer?

Regards,
Raghavendra.

So, I decided to use libxml2 for what it seems to be good at, namely
parsing XML, and to then use Lisp for working with the parsed tree.

Regards,
Raghavendra.

Regards,
Raghavendra.

An alternative would be to use cxml, and to patch it to make it pass
the 1800+ tests from the OASIS XML Tests Suite.

The problem with this is that, I suspect, you can't do this with a
dynamic FFI module. This technique needs access to symbols within the CLISP
image that are not available for dynamic linkage.  I.e. you have to make a
statically linked CLISP C module via the linking set mechanism.  

Here is an example.

Firstly, since we won't be using the FFI compiler at all, but writing our own
intrinsic function, we need an alternate way to generate the C module
boilerplate required for CLISP module linking. That alternate way is
a utility called modprep.lisp which is in the linkkit directory.

With modprep, we can write in a preprocessed C. Let's use the suffix "mp"
for this language:

  $ cat > hello.mp
  #include "clisp.h"
  #include <stdio.h>

  DEFMODULE(hello, "COMMON-LISP-USER")

  DEFUN(COMMON-LISP-USER::LISP-HELLO, &optional dummy)
  {
    object new_cons = allocate_cons();
    printf("allocated cons\n");
    Car(new_cons) = NIL;
    Cdr(new_cons) = NIL;
    printf("initialized cons\n");
    popSTACK();
    VALUES1(new_cons);
  }

Modprep provides considerable conveniences for writing C that interfaces
with CLISP. (Good to read the modprep.lisp source code in the linkkit
directory).

The #include "clisp.h" is mandatory.  The &optional dummy
is a workaround for the fact that DEFUN requires a second argument.

Note how I'm ``cheating'' by using the CL-USER package.  Also note how the
DEFUN syntax is very helpful. We can use Lisp symbol syntax, which is
automatically mapped to some C friendly name that we don't have to care about.

Because our function takes an argument, we need to clean it up before
returning, otherwise there will be a crash. That is the purpose
of the popSTACK() call.

Let's continue. The next step is to know where your CLISP linkkit directory is
and install this as an environment variable:

  $ export CLISP_LIB=/home/kaz/lib/clisp
  $ export CLISP_LINKKIT=$CLISP_LIB/linkkit

Use modprep to process hello.mp to hello.c:

  $ clisp $CLISP_LINKKIT/modprep hello.mp hello.c

Now, create a module set called hellomod, and combine it with CLISP's full
module set, to create a linking set called full_hellomod:

  $ $CLISP_LIB/clisp-link create-module-set hellomod ./hello.c

  $ $CLISP_LIB/clisp-link add-module-set hellomod $CLISP_LIB/full full_hellomod

Test:

  $ ./full_hellomod/lisp.run -q -M ./full_hellomod/lispinit.mem
  [1]> (lisp-hello)
  allocated cons
  initialized cons
  (NIL)

You have to watch out for GC safety when writing intrinsic code.
Objects that are only referenced by your C variables are not reachable
by the collector. If you call any function within CLISP that may trigger GC,
you have to park all your live objects somewhere where they are reachable;
usually the stack is used for this purpose.

It's nice to have features [and maybe libxml2 has error-recovery], but
unless you are really using them, being able to refine or correct the
code seems a better option.  S-XML's event interface was a little
funny, but worked out fine.

Back when I interfaced to a C program, I used a stream to feed lisp.
That was faster and stayed away from the R/W speed of the disk
subsystem, etc.  For simple files, using the file system is simple and
OK, but sooner or later the stream solution will prove to be better
and using it on a simple problem is a good step.

Regards,

  http://xmlsoft.org/xmlreader.html

Regards,