Subject: [Fenics] Release deadline

From: Anders Logg <[email-masked]>
Date: Fri, 12 Feb 2010 00:18:19 +0100

Here's an update on the status for getting packages ready for
inclusion in the next Ubuntu LTS release.

1. Johannes needs all releases to be ready by Monday. That will give
him a couple of days to prepare the packages.

Which releases do we want to make? There have been JIT compiler fixes
in FFC and UFL that would be good to get in, and also updates in
DOLFIN.

We have also made changes to Instant. The buildbot is failing for
Instant but I don't know why.

2. Unfortunately the DOLFIN package needs to be build without CGAL and
ParMETIS.

The reason is that both CGAL and ParMETIS are in the non-free section
of Debian. I knew from before that ParMETIS has a non-free license
(which makes an even stronger case for SCOTCH) but I don't know why
CGAL is in non-free. Anyone else knows why?

If DOLFIN is compiled against CGAL or ParMETIS then the DOLFIN package
would depend on a non-free package, which would place DOLFIN in the
contrib section, which for technical Debian reasons places DOLFIN in
the "NEW" queue, which means Johannes cannot upload but must get
someone else with higher "Debian security clearance" to upload and
that will take a while. There would also be a similar problem for the
FEniCS meta package.

So DOLFIN will be built without CGAL and ParMETIS in Ubuntu, but will
be available with these dependencies in Launchpad PPA.

We should sort out our dependencies for future releases.

--
Anders

---

From: Johan Hake <[email-masked]>
Date: Thu, 11 Feb 2010 16:22:56 -0800

> CGAL is in non-free. Anyone else knows why?

Quote from <http://www.cgal.org/license.html>:

"Some parts of CGAL are available under the LGPL, other parts are under the 
QPL"

It looks like higher level algorithms are released under QPL. I do not know if 
we use such parts. 

Johan

---

From: Andre Massing <[email-masked]>
Date: Fri, 12 Feb 2010 01:28:40 +0100

> CGAL is in non-free. Anyone else knows why?

I still struggling with this license jungle but there is a rather long 
thread why Debian consider the QPL library as not compliant with the 
DSFG  (Debian Free Sofware Guidlines), but to cite from

http://lists.debian.org/debian-legal/2004/05/msg00992.html

"The QPL requires that all changes are sent to the original
author upon request, and that all license disputes are settled in
Amsterdam City Court by the laws of the Netherlands.  Both of these
restrictions are non-DFSG-free.
"

or see the general discussion on QPL

http://lists.debian.org/debian-legal/2004/04/msg00233.html
.
> be available with these dependencies in Launchpad PPA.

That's a bit of a pity that CGAL has this kind of strange license mix.

> We should sort out our dependencies for future releases.

I am also wondering whether using a dynamically loadable library instead 
of shared library would influence the decision which category a package 
will be put in...

> 
> --
> Anders
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> Mailing list: https://launchpad.net/~fenics
> Post to     : [email-masked]
> Unsubscribe : https://launchpad.net/~fenics
> More help   : https://help.launchpad.net/ListHelp

---

From: Anders Logg <[email-masked]>
Date: Fri, 12 Feb 2010 09:33:03 +0100

On Fri, Feb 12, 2010 at 01:28:40AM +0100, Andre Massing wrote:
> I still struggling with this license jungle but there is a rather
> long thread why Debian consider the QPL library as not compliant
> with the DSFG  (Debian Free Sofware Guidlines), but to cite from
>

I thought this was settled since Qt and KDE are in Debian main but
apparently not. Here's what Debian say about the QPL:

 The DFSG-freeness of this license has been called into question. Some
 people appear to believe that because the Qt library is in Debian
 main, that the QPL is DFSG-free. That is a hasty conclusion, however,
 because the Qt library is also licensed under the GNU GPL (see
 http://www.trolltech.com/newsroom/announcements/00000043.html).

 The QPL is not GPL-compatible, which, regardless of one's opinion
 about the license's DFSG-freeness, poses a major practical problem for
 any code licensed under the QPL that is reused elsewhere in
 conjunction with code under the GNU GPL. This makes the QPL alone a
 particularly poor choice of license for a library.

 Furthermore, it is not clear that the Trolltech corporation (the
 author of the Qt library and the QPL itself) believes the QPL to be a
 free software license. Trolltech's website describes how their
 dual-license approach is intended to be "open source-friendly" (see
 http://www.trolltech.com/company/model.html). If Trolltech felt that
 the QPL alone were friendly enough to open-source, why do they have a
 dual-licensing policy?

 Copyright holders in QPL-licensed works should be encouraged to follow
 Trolltech's example, and dual-license their work under the GNU GPL or
 another clearly DFSG-free license.

See http://wiki.debian.org/DFSGLicenses#QPublicLicense.28QPL.29.2CVersion1.0
So the QPL should be avoided.

Perhaps we should ask the CGAL people why they are still using QPL and
ask them to use something else. If enough people ask them, perhaps
they will care.

--
Anders

> I am also wondering whether using a dynamically loadable library
> instead of shared library would influence the decision which
> category a package will be put in...
>
> >
>

---

From: Anders Logg <[email-masked]>
Date: Fri, 12 Feb 2010 10:28:16 +0100

> in FFC and UFL that would be good to get in, and also updates in

As far as I can see, we are ready to make the following releases:

  UFL    0.5.2
  FErari 0.2.0
  FFC    0.9.1

I can just push the button on these if no one has anything to add.

The following remains:

  Instant 0.9.8: Why is the buildbot failing?
  DOLFIN  0.9.7: SCOTCH problems

It would be good to sort out the Instant problem today and the DOLFIN
problem needs to be fixed before Monday.

Anything else that needs to be released? As far as I can see, nothing
is needed for UFC, FIAT and Viper.

--
Anders

---

From: Johannes Ring <[email-masked]>
Date: Fri, 12 Feb 2010 10:40:54 +0100

>  Instant 0.9.8: Why is the buildbot failing?

It is failing on line 120 in tests/test19.py. I have tried to run the
test several times now and most often it fails but sometimes it works.

Johannes

fenics-slave@386:tests$ python test19.py
--- Instant: compiling ---
t = 0.654919  ((1) With modulename)
t = 0.001182  ((2) With modulename)
python -c "import test19_ext"
--- Instant: compiling ---
t = 1.214849  ((1) With signature)
t = 0.000259  ((2) With signature)
t = 0.000011  ((3) import_module)
python -c "import instant; assert
instant.import_module('test19_signature_module', 'test19_cache') is
not None"
--- Instant: compiling ---
t = 1.202170  ((1) With signature)
t = 0.000259  ((2) With signature)
t = 0.000011  ((3) import_module)
python -c "import instant; assert
instant.import_module('((test19_signature_module))', 'test19_cache')
is not None"
t = 1.499505  ((1) With signature)
t = 1.499883  ((2) With signature)
Traceback (most recent call last):
  File "test19.py", line 120, in <module>
    assert t1 > t2
AssertionError
fenics-slave@386:tests$ python test19.py
--- Instant: compiling ---
t = 0.793814  ((1) With modulename)
t = 0.001382  ((2) With modulename)
python -c "import test19_ext"
--- Instant: compiling ---
t = 1.200176  ((1) With signature)
t = 0.000264  ((2) With signature)
t = 0.000011  ((3) import_module)
python -c "import instant; assert
instant.import_module('test19_signature_module', 'test19_cache') is
not None"
--- Instant: compiling ---
t = 1.209449  ((1) With signature)
t = 0.000260  ((2) With signature)
t = 0.000012  ((3) import_module)
python -c "import instant; assert
instant.import_module('((test19_signature_module))', 'test19_cache')
is not None"
t = 1.500331  ((1) With signature)
t = 1.499962  ((2) With signature)
t = 1.499850  ((3) import_module)
--- Instant: compiling ---
t = 1.208890  ((1) Without modulename or signature)
t = 0.000458  ((2) Without modulename or signature)
fenics-slave@386:tests$

---

From: Anders Logg <[email-masked]>
Date: Fri, 12 Feb 2010 10:52:10 +0100

> test several times now and most often it fails but sometimes it works.

ok. I have opened a new thread on the Instant mailing list. All
Instant experts (Kent (and Ilmar, Martin if they are still reading
this list)), please have a look.

For technical reasons (uploading permissions) we need to get a new
release of Instant (and FErari) ready before the other packages
(today).

--
Anders

---

From: "Garth N. Wells" <[email-masked]>
Date: Fri, 12 Feb 2010 10:27:07 +0000

>   DOLFIN  0.9.7: SCOTCH problems

DOLFIN should be straightened out today. There are no problems per se,
just the efficiency needs to be improved. I've done this for one
function, and just now need to do it for another.

I've found some erratic behaviour with SCOTCH and ParMETIS with larger
numbers of partitions. Sometimes partitioning works, sometimes it hangs.
Can anyone run the elasticity cpp demo with 5 processes?

Garth

---

From: Andre Massing <[email-masked]>
Date: Fri, 12 Feb 2010 17:11:13 +0100

> ask them to use something else. If enough people ask them, perhaps

We can try, but I think they that there is a fairly little chance, since 
a lot of non-basic packages in CGAL are licensed under the QPL. But I 
will investigate it.

I also figured out that we nowadays only need to link against libCGAL 
and not libCGAL_Core (was a relict from old testing programs)
Johannes mentioned something that libCGAL is licensed unter LGPL ?
So I am wondering whether something can be debain/main if it *only* 
depends on a LPGL compatible lib which is distributed together with some 
  non QPL libs in non-free?

And there is there the issue that we using some template headers, which 
are under QPL.
Complicated, we probably have to figure it out after the next release 
due to time-constraints...?

---

From: Anders Logg <[email-masked]>
Date: Sun, 14 Feb 2010 23:32:32 +0100

> function, and just now need to do it for another.

Everything except DOLFIN should be ready for release now. It's
currently failing on the buildbot with

  In file included from dolfin/fem/DofMap.cpp:24:
  dolfin/fem/DofMapBuilder.h:13:35: error: boost/unordered_set.hpp: No
  such file or directory

> Can anyone run the elasticity cpp demo with 5 processes?

I haven't installed SCOTCH yet, so it fails for me. Is SCOTCH now
default? Does it work with the regular SCOTCH in Debian? How does it
compare to ParMETIS? (Speed, memory usage.) I remember the first time
we tried SCOTCH that it was very intensive on memory usage.

When I change to ParMETIS, assembling works (but is slow) and then
UMFPACK reports a singular matrix.

--
Anders

---

From: "Garth N. Wells" <[email-masked]>
Date: Sun, 14 Feb 2010 22:45:09 +0000

>   In file included from dolfin/fem/DofMap.cpp:24:

Where is Boost installed on the buildbot?

> default? Does it work with the regular SCOTCH in Debian? 

Yes, but that it's easily changed (just a parameter).

> How does it

That was in serial, right?

>From having run demos, speed seems comparable to ParMETIS. I haven't
checked memory use. What I hope that SCOTCH can partition meshes on
which ParMETIS fails since there is much greater scope for tuning
SCOTCH. Also, for the demos that I ran, SCOTCH produced better partitions.

The parts of the code which I added to build the mesh dual graph can
probably be reused to compute element neighbours across process
boundaries which we need for interior facet integrals.

> UMFPACK reports a singular matrix.

We need more people running in parallel to track performance!

Garth

---

From: Anders Logg <[email-masked]>
Date: Sun, 14 Feb 2010 23:50:23 +0100

> Where is Boost installed on the buildbot?

Don't know but I assume it's the regular Debian/Ubuntu packages.

Johannes knows.

> That was in serial, right?

Yes.

> boundaries which we need for interior facet integrals.

ok, sounds good.

> We need more people running in parallel to track performance!

Yes, definitely.

And don't forget the blueprint on benchmarking:

  https://blueprints.launchpad.net/dolfin/+spec/benchmarks

I don't remember how many cores the buildbot has but I assume it is 4,
so we would be able to track some regressions.

--
Anders

---

From: "Garth N. Wells" <[email-masked]>
Date: Sun, 14 Feb 2010 22:55:11 +0000

> default? Does it work with the regular SCOTCH in Debian? 

No. The Debian package contains only the serial libs. Not sure why.

Garth

How does it

---

From: Anders Logg <[email-masked]>
Date: Mon, 15 Feb 2010 00:03:57 +0100

> No. The Debian package contains only the serial libs. Not sure why.

ok, then we should probably make ParMETIS the default for now since
the DOLFIN package will depend on the Debian version.

We can set ParMETIS as default now, make the release and then change
back after the release.

--
Anders

---

From: "Garth N. Wells" <[email-masked]>
Date: Sun, 14 Feb 2010 23:04:52 +0000

> we tried SCOTCH that it was very intensive on memory usage.

I forgot to add that I observe some 'stochastic' results for SCOCTH with
the elasticity demo. All other demos I've looked at I compute the same
solution on any number of processes. I still need to look at this.

Garth

---

From: Johannes Ring <[email-masked]>
Date: Mon, 15 Feb 2010 10:36:06 +0100

> Don't know but I assume it's the regular Debian/Ubuntu packages.

Yes, it's the Ubuntu packages, however, the file
boost/unordered_set.hpp is not in the Boost packages on Hardy
(linux64-exp is also currently Hardy). If a newer version of Boost is
required I guess we should drop support for Hardy and upgrade the
buildbots.

Johannes

> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.9 (GNU/Linux)
>
> iEYEARECAAYFAkt4fi8ACgkQTuwUCDsYZdHBugCdGqjY0jByTFKy2dKh+wMh1kyZ
> D9EAn1aaFs8BWrhD5E7EjBizoiLEgww7
> =Jf5H
> -----END PGP SIGNATURE-----
>

---

From: Johannes Ring <[email-masked]>
Date: Mon, 15 Feb 2010 10:42:11 +0100

> No. The Debian package contains only the serial libs. Not sure why.

PT-SCOTCH support was recently added in the package in Debian unstable:

http://packages.debian.org/changelogs/pool/main/s/scotch/scotch_5.1.7.dfsg-1/changelog

Johannes

---

From: Anders Logg <[email-masked]>
Date: Mon, 15 Feb 2010 15:49:05 +0100

> required I guess we should drop support for Hardy and upgrade the

Yes, let's drop support for Hardy. It's close to two years old now and
I think there's a very limited number of people still using Hardy.

(Actually, I know of someone (@KVS), but his upgrade is long overdue...)

--
Anders

---

From: Anders Logg <[email-masked]>
Date: Mon, 15 Feb 2010 18:06:50 +0100

> > >> >>>   FFC    0.9.1

Now released.

> > >> >>>   Instant 0.9.8: Why is the buildbot failing?

Now released.

> > >> >>>   DOLFIN  0.9.7: SCOTCH problems

Remains to fix:

  - Drop Hardy support (upgrade buildbots)
  - Memory leaks (looks ok to me)
  - PETSc/Hypre link error

--
Anders

---

From: Johannes Ring <[email-masked]>
Date: Tue, 16 Feb 2010 08:45:33 +0100

> I think there's a very limited number of people still using Hardy.

Well, it is LTS and still a very good release. It is also used on
several computers at Simula, there among Bigblue.

> (Actually, I know of someone (@KVS), but his upgrade is long overdue...)

There is at least one more, me :-) (but my upgrade is long overdue...)

Johannes