Subject: Re: [Deb-dev] [FEniCS-dev] Build time bugs in syfi and segfaults in dolfin -- Partially Fixed

From: "Johannes Ring" <[email-masked]>
Date: Thu, 12 Feb 2009 08:19:59 +0100 (CET)

Hi Balbir,

I am the maintainer of the FEniCS packages for Debian/Ubuntu. If you want
to build Debian packages for Lenny I will recommend you to use the Debian
control files in the pkg-scicomp Subversion repository [1]. If you want
and there is a demand for it I could easily add packages for Lenny to the
FEniCS repository.

The reason the DOLFIN package doesn't build depend on some implementation
of MPI is because it already depend on PETSc and PETSc again is built with
OpenMPI in Debian.

If you have any patches, please forward them to the pkg-scicomp mailing
list [2]. Please continue the discussion at [email-masked].

[1] svn://svn.debian.org/svn/pkg-scicomp/fenics/
[2] http://lists.alioth.debian.org/mailman/listinfo/pkg-scicomp-devel

Johannes

---

From: "Johannes Ring" <[email-masked]>
Date: Thu, 12 Feb 2009 11:33:58 +0100 (CET)

On Thu, February 12, 2009 02:23, Balbir Thomas wrote:
> Hi,
>
> Looks like Harish was right. Rebuilding after update alternatives does
> seem
> to solve the problem with C++ files. Of course I had to edit the scons
> file
> and remove "-Werror" to build dolfin. I am now able to build and
> successfully run the C++ demos (have not tested them all). This does
> indicate Fenics has problems with mpich. Perhaps the build scripts and
> dependencies in the packaging system need to take this into consideration.
> Also  a note in big bold letters would help :-) .
>
> However the python demos fail (at least all the ones I checked). For
> instance demo/pde/poisson/python/demo.py produces the following error :
>
> bt@ps:$ python demo.py
> [darwin:04934] mca: base: component_find: unable to open osc pt2pt: file
> not
> found (ignored)
> Checking mesh ordering (finished).
> Calling just-in-time (JIT) compiler, this may take some time... WARNING:
> Didn't find dolfin.i in include paths returned from pkg-config.
> In instant.recompile: The module did not compile, see
> '/tmp/tmp0FoES72009-2-12-01-13_instant/dolfin_compile_function_c2c6ca28e8e29b2498da8bbcedb6ff5e/compile.log'
> Traceback (most recent call last):
>   File "demo.py", line 31, in <module>
>     u0 = Constant(mesh, 0.0)
>   File "/usr/lib/python2.5/site-packages/dolfin/constant.py", line 38, in
> Constant
>     return Function(V, cppexpr)
>   File "/usr/lib/python2.5/site-packages/dolfin/function.py", line 498, in
> __new__
>     cpp_base = compile_function([cpparg],[defaults])[0]
>   File "/usr/lib/python2.5/site-packages/dolfin/compile_function.py", line
> 335, in compile_function
>     function_classes =
> compile_function_code("\n\n".join(code_snippets),classnames)
>   File "/usr/lib/python2.5/site-packages/dolfin/compile_function.py", line
> 272, in compile_function_code
>     signature      = module_name
>   File "/usr/lib/python2.5/site-packages/instant/build.py", line 423, in
> build_module
>     recompile(modulename, module_path, setup_name,
> new_compilation_checksum)
>   File "/usr/lib/python2.5/site-packages/instant/build.py", line 101, in
> recompile
>     "compile, see '%s'" % compile_log_filename)
>   File "/usr/lib/python2.5/site-packages/instant/output.py", line 49, in
> instant_error
>     raise RuntimeError(text)
> RuntimeError: In instant.recompile: The module did not compile, see
> '/tmp/tmp0FoES72009-2-12-01-13_instant/dolfin_compile_function_c2c6ca28e8e29b2498da8bbcedb6ff5e/compile.log'
>
>
> The contents of
> /tmp/tmp0FoES72009-2-12-01-13_instant/dolfin_compile_function_c2c6ca28e8e29b2498da8bbcedb6ff5e/compile.log
> show
>
> bt@ps:$ more
> /tmp/tmp0FoES72009-2-12-01-13_instant/dolfin_compile_function_c2c6ca28e8e29b2498da8bbcedb6ff5e/compile.log
> dolfin_compile_function_c2c6ca28e8e29b2498da8bbcedb6ff5e.i:97: Error:
> Unable
> to find 'dolfin.i'
> running build_ext
> building '_dolfin_compile_function_c2c6ca28e8e29b2498da8bbcedb6ff5e'
> extension
> creating build
> creating build/temp.linux-i686-2.5
> gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall
> -Wstrict-prototypes -fPIC
> -I/home/bt/debian/sources/dolfin/dolfin-0.9.0/debian/tmp/include -I/
> usr/lib/python2.5/site-packages/numpy/core/include
> -I/usr/include/suitesparse -I/usr/include/python2.5
> -I/usr/lib/petsc/bmake/linux-gnu-c-opt -I/usr/lib/petsc
> /include -I/usr/lib/openmpi/include -I/usr/lib/openmpi/lib -I.
> -I/usr/include/scotch -I/usr/include/libxml2 -I/usr/include/glib-2.0
> -I/usr/lib/glib-2.0/includ
> e -I/usr/lib/python2.5/site-packages/numpy/core/include
> -I/usr/include/python2.5 -c
> dolfin_compile_function_c2c6ca28e8e29b2498da8bbcedb6ff5e_wrap.cxx -o build
> /temp.linux-i686-2.5/dolfin_compile_function_c2c6ca28e8e29b2498da8bbcedb6ff5e_wrap.o
> gcc: dolfin_compile_function_c2c6ca28e8e29b2498da8bbcedb6ff5e_wrap.cxx: No
> such file or directory
> gcc: no input files
> error: command 'gcc' failed with exit status 1
>
> The contents of the temporary folder are
>
> bt@ps:$ tree
> /tmp/tmp0FoES72009-2-12-01-13_instant/dolfin_compile_function_c2c6ca28e8e29b2498da8bbcedb6ff5e/
> /tmp/tmp0FoES72009-2-12-01-13_instant/dolfin_compile_function_c2c6ca28e8e29b2498da8bbcedb6ff5e/
> |-- __init__.py
> |-- build
> |   `-- temp.linux-i686-2.5
> |-- compile.log
> |-- dolfin_compile_function_c2c6ca28e8e29b2498da8bbcedb6ff5e.i
> `-- setup.py
>
> So I note dolfin.i and the cxx file could not be found. Where are these
> supposed to be and or what is supposed to generate them ? I would be
> grateful for any other suggestions to help fix these.

The file dolfin.i is installed by the libdolfin0-dev package and should be
in /usr/include/dolfin/swig. If not, please install the libdolfin0-dev
package. If it is, then there is probably something wrong with the
setup.py file. How is 'swig_cmd' defined in setup.py? It should include
'-I/usr/include/dolfin/swig'.

Johannes

---

From: Johan Hake <[email-masked]>
Date: Thu, 12 Feb 2009 11:52:40 +0100

On Thursday 12 February 2009 11:33:58 Johannes Ring wrote:
> > dependencies in the packaging system need to take this into
> > consideration. Also  a note in big bold letters would help :-) .
> >
> > '/tmp/tmp0FoES72009-2-12-01-13_instant/dolfin_compile_function_c2c6ca28e8
> >e29b2498da8bbcedb6ff5e/compile.log' Traceback (most recent call last):
> >   File "/usr/lib/python2.5/site-packages/dolfin/function.py", line 498,
> > in __new__
> >   File "/usr/lib/python2.5/site-packages/dolfin/compile_function.py",
> > line 335, in compile_function
> >     function_classes =
> >   File "/usr/lib/python2.5/site-packages/dolfin/compile_function.py",
> > line 272, in compile_function_code
> >     signature      = module_name
> > /tmp/tmp0FoES72009-2-12-01-13_instant/dolfin_compile_function_c2c6ca28e8e
> >29b2498da8bbcedb6ff5e/compile.log show
> >
> > bt@ps:$ more
> > /temp.linux-i686-2.5/dolfin_compile_function_c2c6ca28e8e29b2498da8bbcedb6
> >ff5e_wrap.o gcc:
> > dolfin_compile_function_c2c6ca28e8e29b2498da8bbcedb6ff5e_wrap.cxx: No
> > such file or directory
> > gcc: no input files

This shows that python-dolfin should depend on dolfin-dev. Is this the case 
now?

Also, related to previous discussion about swig dep, the reason we need to 
have a dependency on swig 1.3.37 is that python-dolfin is compiled using swig 
1.3.37. To be able to run any demos on your system you need to have the same 
version of swig installed.

But as mentioned previously DOLFIN is dependent on swig version >= 1.3.35. 
This meens that if you build DOLFIN from source it should be enough to have 
swig version 1.3.36.

Johan

---

From: "Johannes Ring" <[email-masked]>
Date: Thu, 12 Feb 2009 12:04:40 +0100 (CET)

> This shows that python-dolfin should depend on dolfin-dev. Is this the

No, it's not. Should I add this or is it better to include the .i files in
the python-pydolfin0 package instead of dolfin-dev?

Johannes

---

From: Johan Hake <[email-masked]>
Date: Thu, 12 Feb 2009 12:14:15 +0100

> the python-pydolfin0 package instead of dolfin-dev?

There are no standards for swig interface files. We meet the same problem with 
pkg-config files. Here we have chosen to install them in 
prefix/include/dolfin/swig, together with the header files. 

We also need dolfin.h when JIT compiling dolfin-functions so it seems natural 
to include dolfin-dev as dependency for python-dolfin.

Johan

---

From: "Johannes Ring" <[email-masked]>
Date: Thu, 12 Feb 2009 12:21:42 +0100 (CET)

> to include dolfin-dev as dependency for python-dolfin.

Ok, I will do that.

Johannes

---

From: "Johannes Ring" <[email-masked]>
Date: Thu, 12 Feb 2009 12:35:00 +0100 (CET)

On Thu, February 12, 2009 12:20, Balbir Thomas wrote:
> Hi,
>
> I think the only problem remaining now is why is the pkg-config cflags are
> wrong. The include directory points to the build directory not the install
> directory.
>
>
> bt@~:$ pkg-config --cflags dolfin
> -Wall -pipe -ansi -DDEBUG -g -DNDEBUG -O2 -DHAS_MPI=1
> -DMPICH_IGNORE_CXX_SEEK -DPACKAGE_VERSION="0.9.0" -DHAS_PETSC=1
> -DHAS_SCOTCH=1 -DHAS_UMFPACK=1 -DHAS_GTS=1 -DHAS_CHOLMOD=1
> -fno-strict-aliasing -pthread
> -I/home/bt/debian/sources/dolfin/dolfin-0.9.0/debian/tmp/include
> -I/usr/lib/python2.5/site-packages/numpy/core/include
> -I/usr/lib/petsc/bmake/linux-gnu-c-opt -I/usr/lib/petsc/include
> -I/usr/lib/openmpi/include -I/usr/lib/openmpi/lib -I.
> -I/usr/include/scotch
> -I/usr/include/libxml2 -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include
>
> /home/bt/debian/source/dolfin is the directory in which I unpack the
> debian
> source package and build it resulting in
> "-I/home/bt/debian/sources/dolfin/dolfin-0.9.0/debian/tmp/include"

I don't think that is the reason for it to fail because it should have
been replaced by /usr/include which is included anyway, and because there
is a similar line in dolfin.pc in the Ubuntu package
(-I/tmp/buildd/dolfin-0.9.0/debian/tmp/include) and this works without any
problems. However, we should still try to fix this. I think we need to add
some sort of DESTDIR support in the build system.

> As regards to "swig_cmd" in setup.py I am not sure what you mean by this.
> The only setup.py in the source directory is

I was referring to the setup.py file from the temporary Instant directory
which you posted earlier.

Johannes

---

From: "Johannes Ring" <[email-masked]>
Date: Thu, 12 Feb 2009 14:08:29 +0100 (CET)

On Thu, February 12, 2009 13:50, Balbir Thomas wrote:
> Here are the contents of setup.py in the temporary instant directory
> -------------------------------------------------------------------------------------------------
>
> [email-masked]:$ more
> /tmp/tmpCMJ4kh2009-2-12-11-48_instant/dolfin_compile_function_df46e19d3b7dfd0a28ed8c435a22313b/setup.py
>
> import os
> from distutils.core import setup, Extension
> name = 'dolfin_compile_function_df46e19d3b7dfd0a28ed8c435a22313b'
> swig_cmd ='swig -python  -c++
> dolfin_compile_function_df46e19d3b7dfd0a28ed8c435a22313b.i'

And there is your problem: The flags -I/usr/include and
-I/usr/include/dolfin/swig is missing from swig_cmd. Here is what the line
should look like:

swig_cmd ='swig -python  -c++ -I/usr/include -I/usr/include/dolfin/swig
dolfin_compile_function_c2c6ca28e8e29b2498da8bbcedb6ff5e.i

I am not sure why it is not included in your line.

BTW, I am about to create packages for lenny. They should be out during
the day.

And please keep the discussion of the Debian/Ubuntu packages in
[email-masked].

Johannes

---

From: Johan Hake <[email-masked]>
Date: Thu, 12 Feb 2009 14:31:46 +0100

> dolfin_compile_function_c2c6ca28e8e29b2498da8bbcedb6ff5e.i

The default behaviour of pkg-config is to not include /usr for the --cflags 
argument and swig compared to g++ does _not_ include /usr/include when 
compiling. So therefore we manually must say to pkg-config that we want the 
system flags too.

Try setting the following environment variable:

  PKG_CONFIG_ALLOW_SYSTEM_CFLAGS=1

and then rerun the demo. 

Btw, this is also fixed in the development version of instant. 

Johan

> _______________________________________________
> Deb-dev mailing list
> [email-masked]
> http://fenics.org/mailman/listinfo/deb-dev

---

From: "Johannes Ring" <[email-masked]>
Date: Thu, 12 Feb 2009 15:00:07 +0100 (CET)

> Btw, this is also fixed in the development version of instant.

Yes, and now I remember that added a patch for this in the Hardy and
Intrepid packages. From Instant's debian/changelog:

instant (0.9.5-1~intrepid2) intrepid; urgency=low

  * debian/rules: now including simple-patchsys.
  * debian/control: added build-essential as dependency for binary
    package python-instant.
  * Added patch for allowing system cflags to be returned by pkg-config.

 -- Johannes Ring <[email-masked]>  Tue, 13 Jan 2009 09:45:14 +0100

I will add the same when building Instant for Lenny.

Johannes

---

From: "Johannes Ring" <[email-masked]>
Date: Thu, 12 Feb 2009 16:19:58 +0100 (CET)

Hi Balbir,

There is now packages for lenny in the repository and I have updated the
installation instructions [1]. I would recommend to remove/purge your own
packages before installing and perhaps clean up the APT cache. Let me know
if the packages works or if you have any problems.

[1] http://www.fenics.org/wiki/Download

Johannes

On Thu, February 12, 2009 14:19, Balbir Thomas wrote:
> Yes I can confirm that does solve the problem. At least manually
> editing "swig_cmd" and doing a "python setup.py build" does build the
> dynamically linked libraries .
>
> Thank you very much for your efforts. I look forward to your packages.
> I shall direct further comment to [email-masked]. More than likely
> i will get busy learning dolfin so will stay in the user lists rather
> than the developer ones :-).
>
> regards
> bt
>

---

From: "Johannes Ring" <[email-masked]>
Date: Fri, 13 Feb 2009 08:12:21 +0100 (CET)

Balbir,

On Thu, February 12, 2009 16:59, Balbir Thomas wrote:
> Hi
>
> I just built and installed the lenny package you uploaded. It does
> have a single build failure which can be fixed by removing "-Werror"
> from line 164 of the  "SConstruct" file, in the top level source
> directory.

I can't see how this can happen when CXXFLAGS is set to "-Wall -pipe
-ansi" in debian/rules? It doesn't happen when I build the package. I get
warnings but no errors. Are you sure it's not just warnings?

> However the 'swig_cmd' still does not include, system include
> directories so dolfin python scripts fail as before. But setting
>
> as suggested by Johann does solve the problem.

Ok, I see what went wrong in this case. I forgot to include
simple-patchsys.mk in Instant's debian/rules and the patch was therefore
not applied. It should be fixed now. Please update your python-instant
package.

> In summary with these two changes I find that dolfin/fenics works just
> fine.

Good.

Johannes