Subject: [Fenics] Development model

From: Anders Logg <[email-masked]>
Date: Mon, 15 Apr 2013 14:37:16 +0200

I suggest we adopt the "gitworkflows" development model as described here:

  https://www.kernel.org/pub/software/scm/git/docs/gitworkflows.html

(Can also be read by the command 'man 7 gitworkflows'.)

In more detail, I suggest we

- create 'maint', 'master', 'next' branches in the official repository

- skip the 'pu' branch for now

- publish topic branches in personal repositories

- follow the "gitworkflows" model otherwise

Core developers should read up on the description of gitworkflows and
comment. Any objections to adopting this model?

The main motivation is that this is a standard model used by many
other projects, including our PETSc friends who can share their
experience and give us pointers when we stumble.

--
Anders

---

From: "Marie E. Rognes" <[email-masked]>
Date: Mon, 15 Apr 2013 14:43:14 +0200

> experience and give us pointers when we stumble.

So, you are referring to the PETSc model as described here:

   https://bitbucket.org/petsc/petsc/wiki/developer-instructions-git
   https://bitbucket.org/petsc/petsc/wiki/quick-dev-git

as suggested earlier by Garth? Sounds good to me.

--
Marie

> --
> Anders
>
> _______________________________________________
> Mailing list: https://launchpad.net/~fenics
> Post to     : [email-masked]
> Unsubscribe : https://launchpad.net/~fenics
> More help   : https://help.launchpad.net/ListHelp

---

From: Anders Logg <[email-masked]>
Date: Mon, 15 Apr 2013 14:47:51 +0200

> as suggested earlier by Garth? Sounds good to me.

No, I'm referring to

minus the 'pu' branch.

--
Anders

---

From: "Marie E. Rognes" <[email-masked]>
Date: Mon, 15 Apr 2013 15:01:23 +0200

>    https://www.kernel.org/pub/software/scm/git/docs/gitworkflows.html

Are there any crucial differences? As far as I can see, the PETSc wiki 
provides a bit more detail (very useful for those of us new to git) and 
specific naming suggestions.

--
Marie

---

From: Anders Logg <[email-masked]>
Date: Mon, 15 Apr 2013 15:08:23 +0200

On Mon, Apr 15, 2013 at 03:01:23PM +0200, Marie E. Rognes wrote:
> Are there any crucial differences? As far as I can see, the PETSc
> wiki provides a bit more detail (very useful for those of us new to
> git) and specific naming suggestions.

Yes it's useful so it is definitely worth reading. It's also very
close to gitworkflows. But if we should adopt a model, I prefer to say
that we adopt the "gitworkflows model", instead of "the gitworkflows
model as currently interpreted by the PETSc developers".

--
Anders

---

From: "Marie E. Rognes" <[email-masked]>
Date: Mon, 15 Apr 2013 15:10:03 +0200

> model as currently interpreted by the PETSc developers".

Ok, thanks for clarifying. Still sounds good to me.

--
Marie

---

From: Martin Sandve Alnæs <[email-masked]>
Date: Tue, 16 Apr 2013 12:30:46 +0200

Looks good!

It is challenging to be this diciplined in the development, but I hope
we can pull it off.

I'll try to set up ufl like this, with maint being a checkout of 1.2,
and the bugfix I did cherry-picked into maint.

Martin

---

From: Martin Sandve Alnæs <[email-masked]>
Date: Tue, 16 Apr 2013 14:58:42 +0200

I think the main challenge for us is to adapt the workflow to
cross-project changes.

How to deal with topic branches for cross-project features is not
discussed directly in gitworkflows, but I guess the natural extension
is simply to:
- Create topic branches with same name across relevant projects.
- Do each step such as merging into maint/master/next, etc.
simultaneously for all projects.

The core point being that maint/master/next/mytopic brances must be
kept compatible across projects at any time (when published).

There's a potential race condition if Alice and Bob try pushing
simultaneously to fenics/*/master, but that's not a new problem.

Martin

---

From: Anders Logg <[email-masked]>
Date: Tue, 16 Apr 2013 15:15:28 +0200

> simultaneously for all projects.

Sounds good for now.

> kept compatible across projects at any time (when published).

One of the reasons we move to git was that we might use subtrees.
I think we should let the dust settle (still issues to fix) and then
discuss how we might use subtrees of if we might merge some of the
repositories together.

--
Anders

---

From: "Garth N. Wells" <[email-masked]>
Date: Wed, 17 Apr 2013 12:48:30 +0100

So in summary, do we agree as a start to adopt the 'gitworkflows' work
flow minus 'pu', and to use the PETSc detailed git instructions for
beginners on how to implement gitworkflows (i.e. use the PETSc docs to
know what command to type . . . ).

Garth

---

From: Anders Logg <[email-masked]>
Date: Wed, 17 Apr 2013 14:07:22 +0200

Yes.

And when we know more about how it works, we should write up our own
list of instructions, in particular regarding branches that span
several projects.

--
Anders

---

From: Anders Logg <[email-masked]>
Date: Wed, 17 Apr 2013 14:08:04 +0200

One thing though is that it perhaps doesn't make sense to create the
'next' branch at this point, until it is being tested by the buildbot?

I think Johannes is working on getting it running.

--
Anders

---

From: Martin Sandve Alnæs <[email-masked]>
Date: Wed, 17 Apr 2013 14:15:32 +0200

Partly true, but from a personal workflow p.o.v. it still makes sense to me:

- work in local mytopic
- keep local master in sync by pulling
- pull master into local next and merge mytopic into local next and test
before merging mytopic into master and pushing

If you have several local topic branches you can merge them into next
separately or together for testing, then later merge them one at a time
into master for integration.

Martin
Den 17. apr. 2013 14:08 skrev "Anders Logg" <[email-masked]> følgende:

> > >>>>>> I suggest we adopt the "gitworkflows" development model as
> described
> > >>>>>> here:
> > >>>>>>
> > >>>>>>

---

From: Anders Logg <[email-masked]>
Date: Wed, 17 Apr 2013 14:18:33 +0200

So you mean it makes sense for each developer that has at least two
topic branches?

--
Anders

On Wed, Apr 17, 2013 at 02:15:32PM +0200, Martin Sandve Alnæs wrote:
>    - pull master into local next and merge mytopic into local next and
>    test before merging mytopic into master and pushing
>
>    Den 17. apr. 2013 14:08 skrev "Anders Logg" <[1][email-masked]>
>    følgende:
>
> Referenser
>    1. mailto:[email-masked]
>    2. mailto:[email-masked]
>    3. https://www.kernel.org/pub/software/scm/git/docs/gitworkflows.html
>    4. https://bitbucket.org/petsc/petsc/wiki/developer-instructions-git
>    5. https://bitbucket.org/petsc/petsc/wiki/quick-dev-git
>    6. https://www.kernel.org/pub/software/scm/git/docs/gitworkflows.html
>    7. https://launchpad.net/~fenics
>    8. mailto:[email-masked]
>    9. https://launchpad.net/~fenics
>   10. https://help.launchpad.net/ListHelp
>   11. https://launchpad.net/~fenics
>   12. mailto:[email-masked]
>   13. https://launchpad.net/~fenics
>   14. https://help.launchpad.net/ListHelp

---

From: Johannes Ring <[email-masked]>
Date: Wed, 17 Apr 2013 14:42:33 +0200

> I think Johannes is working on getting it running.

The buildbot is running again now, although I have currently only
enabled a few of the buildslaves since I have had some problems
getting polling of changes from Bitbucket to work. It looks like it is
working now, so I will enable the rest of the buildslaves soon.

Johannes

---

From: Martin Sandve Alnæs <[email-masked]>
Date: Wed, 17 Apr 2013 15:47:10 +0200

No need for two topic branches, it's also useful for for developers
who want to keep an eye on the latest integrated trunk (e.g. pulling
into local master daily) while working on a single local topic branch
for some time. Or maybe what I'm thinking about is more like a local
pu branch than next:

One of the points in the gitworkflows as I see it is that you try to
avoid a pattern like this:

branch master into topic
while not done:
    work on local topic
    merge official master into local topic
merge topic into master and push

because the merging of topic is then interleaved with other changes
and not easily revertable. If you instead do this:

branch master into topic
while not done:
    work on local topic
    pull official master into local master
    merge master and topic into pu for integration testing
merge topic into master and push

you get a lot of advantages like much cleaner history (the history of
topic is typically linear between the branch from master and the merge
back into master) which leads to easier debugging/blaming/reverting,
and can still test integration of others published work with your
ongoing work in the pu branch.

I actually used a similar pattern with bzr, with local branches
trunk=master, work=topicoftheday, scratch=pu, but I wasn't quite as
diciplined and probably followed the first workflow above from time to
time.

Martin

---

From: Martin Sandve Alnæs <[email-masked]>
Date: Wed, 17 Apr 2013 15:54:52 +0200

So, will we do something like this?

- checkout topic branch from master
- work on topic
- test locally

- merge topic into next
- test locally

- push to fenics/dolfin/next
- wait for green buildbot and allow time for others to (pro)test

- merge topic into master
- test locally

- push to fenics/dolfin/master
- wait for green buildbot

- breathe out relieved

Would it also be possible (in due time) to point the buildbot to e.g.
my personal martinal/<project>/next branches? I miss being able to
test changes to ufl/ufc/ffc/dolfin together on buildbot before
integrating.

Martin

---

From: Anders Logg <[email-masked]>
Date: Wed, 17 Apr 2013 15:58:23 +0200

Yes, this is the same as I imagine.

It's quite complex and even more so when we multiply by ~5 for the
number of different projects that may be involved.

--
Anders

---

From: "Garth N. Wells" <[email-masked]>
Date: Wed, 17 Apr 2013 15:21:10 +0100

> test changes to ufl/ufc/ffc/dolfin together on buildbot before

It was possible to ask the buildbot to run a particular branch. It
would be neat if we could just paste in the location of a git branch
and have the tests run. If it's made easy, we could insist that
contributors run their proposed changes through the buildbot before
making a merge request.

Garth

---

From: Martin Sandve Alnæs <[email-masked]>
Date: Wed, 17 Apr 2013 16:50:23 +0200

The send-a-patch-to-the-trybot approach is very hard to use if you
want to e.g. run ffc/dolfin tests with changes to ufl/ufc. The
approach with the old buildbot where a set of branches for all
projects can be used for running tests was much more useful. If I can
push my work to e.g. martinal/<project>/buildbot and run a 'buildbot
...' command to schedule tests that would be extremely helpful.

Martin

---

From: Anders Logg <[email-masked]>
Date: Wed, 17 Apr 2013 19:24:27 +0200

Another question is whether to include topic branches in the main
repository. PETSc does this and currently lists 59 branches:

https://bitbucket.org/petsc/petsc/overview

It gives a good overview of what's going on at the moment.

It might be possible to combine this with a larger group of people
having write access (but limit the integration/review/merge to master
to a smaller group).

--
Anders