Subject: [FEniCS-dev] Logging in Python

From: "Martin Sandve Alnæs" <[email-masked]>
Date: Sun, 16 Mar 2008 16:50:07 +0100

I highly recommend that all Python projects in FEniCS use the module
"logging" that comes with Python from version 2.3. It is very simple
and very flexible, in particular for creating cross-module logs. For
the most basic usage, look at output.py and the tests in UFL.

Basic tutorial (good site for handy python modules):
http://blog.doughellmann.com/2007/05/pymotw-logging.html

Reference:
http://docs.python.org/lib/module-logging.html

--
Martin

---

From: Anders Logg <[email-masked]>
Date: Tue, 18 Mar 2008 16:34:12 +0100

> http://docs.python.org/lib/module-logging.html

Sounds good, but the big problem is more likely to coordinate logging
from C++ (DOLFIN) and Python. Is there a nice solution for this?

-- 
Anders

---

From: "Matthew Knepley" <[email-masked]>
Date: Tue, 18 Mar 2008 10:37:38 -0500

>  from C++ (DOLFIN) and Python. Is there a nice solution for this?

As long as whole lines are always written, and the file is line buffered, I
think writing from two different file descriptors should be fine. The nice
thing would be to replicate the format in C++.

   Matt

>  --
>  Anders
>  _______________________________________________
>  FEniCS-dev mailing list
>  [email-masked]
>  http://www.fenics.org/mailman/listinfo/fenics-dev
>

-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which
their experiments lead.
-- Norbert Wiener

---

From: Anders Logg <[email-masked]>
Date: Tue, 18 Mar 2008 16:46:01 +0100

> thing would be to replicate the format in C++.

Even if the file is stdout?

The format looks surprisingly similar to what we have implemented:

  logging.debug    --> dolfin::dolfin_debug
  logging.info     --> dolfin::message
  logging.warning  --> dolfin::warning
  logging.error    --> dolfin::error

(Maybe not so surprising since these are fairly obvious names.)

Anyway, it would be easy for us to move to something else (or modify
the behavior so it's identical to the logging module).

-- 
Anders

---

From: "Matthew Knepley" <[email-masked]>
Date: Tue, 18 Mar 2008 10:49:29 -0500

>  Even if the file is stdout?

stdout is usually character buffered. You have to change the flag on
your descriptor I believe.

  Matt

---

From: Andy Ray Terrel <[email-masked]>
Date: Tue, 18 Mar 2008 10:54:09 -0500

> thing would be to replicate the format in C++.

Or you could just write a C api that calls into python.  Is there any 
reason you would be needing to log from C++ and python at the same time?

-- Andy

-- 

====================
Andy Terrel
Computer Science Dept
University of Chicago
[email-masked]
---------------------

We make a living by what we get, 
but we make a life by what we give.
                 -Winston Churchill

---

From: Anders Logg <[email-masked]>
Date: Tue, 18 Mar 2008 22:05:37 +0100

> reason you would be needing to log from C++ and python at the same time?

Yes, when running DOLFIN from Python. We then call DOLFIN C++ code,
DOLFIN Python code (not everything is done in SWIG), FFC Python code
and FIAT Python code (which is quiet most of the time).

-- 
Anders

---

From: "Martin Sandve Alnæs" <[email-masked]>
Date: Wed, 19 Mar 2008 21:00:55 +0100

> and FIAT Python code (which is quiet most of the time).

You can subclass the "logging.Logger" class (or something similar) in
PyDOLFIN, simply passing the log messages to the C++ DOLFIN log
system. So nothing (or very little) needs to change in C++ DOLFIN.

-- 
Martin

---

From: Anders Logg <[email-masked]>
Date: Wed, 19 Mar 2008 21:21:20 +0100

> system. So nothing (or very little) needs to change in C++ DOLFIN.

Then what's the point of using the logging module in PyDOLFIN, instead
of just calling the DOLFIN log system directly?

-- 
Anders

---

From: "Martin Sandve Alnæs" <[email-masked]>
Date: Thu, 20 Mar 2008 15:45:56 +0100

> > system. So nothing (or very little) needs to change in C++ DOLFIN.

(Maybe we want something more though, I might have oversimplified things here.)

> of just calling the DOLFIN log system directly?

Python libraries can't depend on the DOLFIN log system.

If UFL, FFC, SyFi, Instant, PyDOLFIN, and other non-fenics Python
projects all use the Python logging module, the end application (in
Python) can combine log output from multiple libraries into a single
log in whatever combination is wanted, and there is a single standard
way to redirect log output to screen and/or file and set logging
levels for all libraries (and submodules of libraries) independently.

--
Martin

---

From: Anders Logg <[email-masked]>
Date: Thu, 20 Mar 2008 18:05:07 +0100

> Python libraries can't depend on the DOLFIN log system.

I agree, but didn't you suggest that we pass the log messages to the
C++ DOLFIN log system?

> levels for all libraries (and submodules of libraries) independently.

Good, but the problem still remains what to do about the messages
issued from DOLFIN in C++. Maybe there's a way to "attach" these to the
Python logging module? It's possible to specify the output destination
as a std::ostream. This might not be enough, but perhaps something
similar could work.

-- 
Anders

---

From: "Martin Sandve Alnæs" <[email-masked]>
Date: Sun, 23 Mar 2008 13:42:40 +0100

> I agree, but didn't you suggest that we pass the log messages to the

A python application that uses ufl, ffc and pydolfin should be able to
merge the logging output from ufl and ffc into the dolfin log. This
can be done by subclassing logging.Handler to pass log messages to the
dolfin system, and attaching this handler object to the ufl and ffc
Logger objects. Something like

h = dolfin.getLoggingHandler()
ufllog = logging.getLogger("ufl")
ffclog = logging.getLogger("ffc")
ufllog.addHandler(h)
ffclog.addHandler(h)

This should handle piping everything through the C++ dolfin log system
without tying the pure python projects to dolfin.

However, this is one of those designs that make dolfin "less a
library" and more an "application backend", which it would be nice to

>  as a std::ostream. This might not be enough, but perhaps something

What we need to make this work is to subclass logging.Logger in
pydolfin (lets call the subclass dolfin.Logger), such that messages in
the C++ dolfin log system can be passed through a dolfin.Logger object
if PyDOLFIN is in use, but not if it's a C++ application.

So we need to subclass std::ostream in the swig wrappers for pydolfin
such that they pipe their output to suitable python calls on the
global dolfin.Logger object. This should be possible, and all the ugly
bits can be confined to the .i files.

--
Martin

---

From: Anders Logg <[email-masked]>
Date: Thu, 27 Mar 2008 14:12:15 +0100

> bits can be confined to the .i files.

This sounds good to me, but it will be far down on *my* TODO list. ;-)

-- 
Anders

---

From: "Martin Sandve Alnæs" <[email-masked]>
Date: Thu, 27 Mar 2008 14:18:30 +0100

> This sounds good to me, but it will be far down on *my* TODO list. ;-)

Of course... This became more complicated than I first thought.
Anyway, dolfin can wait until all Python projects use it "some time in
the future"(tm).

-- 
Martin