Subject: Re: [Fenics] UFR - The Unified Fenics Repository

From: David Ham <[email-masked]>
Date: Wed, 6 Feb 2013 16:39:47 +0000

Hi Martin,

We're not amazingly happy about that idea.

The design of FEniCS as I understand it provides a series of tools
which *can* be used in conjunction but need not be. In particular, our
toolchain (which currently revels in the name flop.py) employs UFL,
FFC, FIAT and Instant, but not UFC or Dolfin (since we employ PyOP2
and Fluidity in those roles). Merging the repositories would give us a
de facto Dolfin dependency which we don't need. It also more generally
undermines the stand-alone nature of these tools and undermines the
idea that these are generic tools which might usefully be used by
solvers other than Dolfin (eg the UFL Dune project we heard about a
while back).

I am also concerned that having them all in the same repository would
tempt people to break tool orthogonality by introducing
cross-dependencies which are not necessary but might appear convenient
to the coder at the time. This would potentially turn a de facto
dependency into a real one and could really make our life hard.

Regards,

David

On 6 February 2013 15:43, Martin Sandve Alnæs <[email-masked]> wrote:
> Whether to merge all fenics projects into one repository was discussed
> offline in january, my tests of today show that it is definitely feasible.
>
> Before we eventually do that, we should allow some time for people to
> foresee eventual issues, e.g. we'll need changes to the buildbot, which
> projects should be included, etc.
>
> The blueprint interface is not really a good place to discuss, as it doesn't
> handle concurrent access by multiple users, so chip in on this thread if you
> have anything to say.
>
> Here's a summary of pros and cons:
> https://blueprints.launchpad.net/fenics/+spec/combine-projects-into-one-repository
> including a link to an already merged repository prototype and a script to
> create it.
>
> Martin

-- 
Dr David Ham
Department of Computing
Imperial College London

http://www.imperial.ac.uk/people/david.ham

---

From: Martin Sandve Alnæs <[email-masked]>
Date: Wed, 6 Feb 2013 18:27:38 +0100

Hi David. I agree those are real concerns, and that good intentions about
logical project separation now may easily be forgotten down the line. One
alternative is to keep dolfin out of it. Having two repositories instead of
six would still help a lot with atomic pushes of syncronized changes. It is
a real cost to the development having to handle version control of in
particular ufc,ufl,ffc separately.

Martin
Den 6. feb. 2013 17:39 skrev "David Ham" <[email-masked]>

---

From: Anders Logg <[email-masked]>
Date: Wed, 6 Feb 2013 18:28:38 +0100

> solvers other than Dolfin (eg the UFL Dune project we heard about a

We could make installation targets for specific subpackages (those you
mention) and add tests run by the buildbot that those can actually be
installed and used (for a set of test problems of your choice) without
needing to build DOLFIN.

The reason I suggested this in the first place was after sitting 5
people for one whole day making the releases of DOLFIN, FFC, FIAT,
Instant, UFC, UFL. It sounds ridiculous but there is something like 20
different steps to go through for each release (including pressing
buttons on Launchpad) so it's a big hassle for anyone wanting to make
a release. Then multiply that by a factor 6 and realize why we don't
make releases that often...

Another issue is working on new features. The same factor of 6 (or
more commonly only just 3-4) applies there.

We started out with 2 projects (DOLFIN + FIAT) which naturally were
separate (one C++ library and one Python library). Then we added FFC
and SyFi and UFL and UFC were added as a glue between the different
form compilers. At the time, this was an efficient way to collaborate
and allow different groups to work independently on interoperable
codes. The situation has since changed. Now most core developers touch
all the codes all the time so a common repository would be more
convenient.

The reason for the proposed change is all about making life more
convenient for developers and cutting down on administrative overhead.

> dependency into a real one and could really make our life hard.

We will likely see code being moved from one place to the other, but
cross-dependencies could likely be avoided by introducing new tests.
One particular test would be to check that the UFL unit tests and the
FIAT unit tests run without installation of any of the other packages.

--
Anders

---

From: David Ham <[email-masked]>
Date: Wed, 6 Feb 2013 17:42:26 +0000

I think Martin's suggestion of Dolfin + the rest as the split would
largely meet my concerns, and it sounds like it mostly meets your
concerns below as well. It also essentially takes the project back to
being one almost Python only project with limited dependencies (except
UFC but that's small) and Dolfin which is big and has more
dependencies. It also preserves the solver-independent aspect of the
first set of packages.

I'm not going to attempt to tell you how to run your project so if you
really need to go to one repository then we'll have to cope, but my
preference would be the two way split above.

Regards,

David

-- 
Dr David Ham
Department of Computing
Imperial College London

---

From: David Ham <[email-masked]>
Date: Wed, 6 Feb 2013 17:45:09 +0000

As a different issue,  we have a fairly persistent ffc branch which is
unlikely to be merged on the timescales we're talking about here, so
if there is a clean way for us to end up with a branch of the new
trunk which has our ffc branch (preferably with its history), that
would help. I imagine we're not the only people sitting on branches we
don't want to die.

David

-- 
Dr David Ham
Department of Computing
Imperial College London

---

From: Anders Logg <[email-masked]>
Date: Wed, 6 Feb 2013 18:47:47 +0100

> dependencies. It also preserves the solver-independent aspect of the

The problem here is the dependency from FFC to UFC and from UFC to
DOLFIN (the DOLFIN wrapper code). The form compiler needs to generate
DOLFIN-specific code for DOLFIN C++ users so they don't need to work
with raw UFC objects. We already have a cross-dependency between UFC
and DOLFIN (the DOLFIN utils module). We should probably sort that one
out as well.

> preference would be the two way split above.

Your input is very valuable and we definitely want to find a solution
that doesn't completely suck for you.

--
Anders

---

From: Martin Sandve Alnæs <[email-masked]>
Date: Wed, 6 Feb 2013 19:36:24 +0100

We definitely need to take some time to figure out such branching issues
before we are in too deep. One way to reduce the hit could be to merge ufl
and ufc into ffc. I dont know how well bzr will handle reverting and
merging files that are moved around. Also, the dolfin utils and ufc design
could get some overhaul for cleaner separation as Anders mentioned.

Martin
Den 6. feb. 2013 18:42 skrev "David Ham" <[email-masked]>

---

From: "Johan Hoffman" <[email-masked]>
Date: Thu, 7 Feb 2013 14:10:21 +0100

For future development I agree that ufl+ufc+ffc make sense, but I think it
is a good idea to keep Dolfin separate.

Best
Johan

>> >>> handle concurrent access by multiple users, so chip in on this
>> thread
>> if you
>> >>> have anything to say.
>> >>>
> _______________________________________________
> Mailing list: https://launchpad.net/~fenics
> Post to     : [email-masked]
> Unsubscribe : https://launchpad.net/~fenics
> More help   : https://help.launchpad.net/ListHelp
>

---

From: "Garth N. Wells" <[email-masked]>
Date: Sun, 10 Feb 2013 17:57:30 +0000

I don't have a strong opinion either way. The problems as present are:

1. Making releases is pain and time consuming. End result is that we
don't release often enough.
2. Versioning (which components are compatible with which) is
confusing for users.
3. Tracking down regressions is really painful when changes are being
made in different components. It's not possible to do simple
regressions.
4. Development across components is not convenient.

The downsides of a single repository for me are:

1. Makes it more difficult for 3rd parties to access different
components independently of others.
2. Encourages strong dependencies between projects. We're in a period
of consolidation of projects, but we may want to broaden things again
(e.g. multiple form compilers) in the future.
3. Will it be painful managing a very large repository with bzr?
4. If we use sub-project features in the version control system, will
it be more complicated for developers? I like it simple.

Do any of the hosting services have good support for sub-projects?

Garth

On 6 February 2013 18:36, Martin Sandve Alnæs <[email-masked]> wrote:
> and ufc into ffc. I dont know how well bzr will handle reverting and merging
> files that are moved around. Also, the dolfin utils and ufc design could get
> some overhaul for cleaner separation as Anders mentioned.
>
> Martin
>