Subject: [FFC-dev] [HG FFC] Added test suite to verify correctness of tabulate_tensor() compared to reference values.

From: [email-masked] (FFC)
Date: Wed, 10 Sep 2008 10:45:54 +0200 (CEST)

One or more new changesets pushed to the primary ffc repository.
A short summary of the last three changesets is included below.

changeset:   1201:46afb0cbcaf0b24300bdcc387ceed426d530fda3
tag:         tip
user:        "Kristian Oelgaard <[email-masked]>"
date:        Wed Sep 10 10:45:24 2008 +0200
files:       src/test/verify_tensors/references/README src/test/verify_tensors/test.py
description:
Added test suite to verify correctness of tabulate_tensor() compared to reference values.

changeset:   1200:e71fefc4b7a9e0551360d71a0554a4b2628e12a6
user:        "Kristian Oelgaard <[email-masked]>"
date:        Tue Sep 09 13:50:53 2008 +0200
files:       src/ffc/compiler/codegeneration/quadrature/reduce_operations.py
description:
Minor bug fix for quadrature representation.

changeset:   1199:2b9342b28e97b64522fcd18f10750e9d348c2347
user:        "Kristian Oelgaard <[email-masked]>"
date:        Mon Sep 08 19:59:31 2008 +0200
files:       ChangeLog TODO
description:
Update TODO and ChangeLog with display of operation count.

----------------------------------------------------------------------
For more details, visit http://www.fenics.org/hg/ffc

---

From: Kristian Oelgaard <[email-masked]>
Date: Wed, 10 Sep 2008 11:17:32 +0200

Hi,

This is my standard procedure for FFC development:

  1. Modify FFC
  2. Run regression tests
  3. Regression tests fails
  4. Look at code, to see if it makes sense
  5. Generate new references
  6. Push to repository

Instead of step 4 it would obviously be better to actually check if the new code
still computes the right thing. To this end I've created a module that verifies
if tabulate_tensor() is correct according to some reference. The module needs
ufc_benchmark to run.

have a look at ffc/src/test/verify_tensor/test.py

./test.py -h

Kristian

Quoting FFC <[email-masked]>:

> _______________________________________________
> FFC-dev mailing list
> [email-masked]
> http://www.fenics.org/mailman/listinfo/ffc-dev
>

---

From: Anders Logg <[email-masked]>
Date: Wed, 10 Sep 2008 11:22:00 +0200

> have a look at ffc/src/test/verify_tensor/test.py

I've looked at it and it looks very good. Will you add references for
all the forms?

Is the idea that we run this only when the regression tests fail
(since it may take some time to run)?

-- 
Anders

---

From: Kristian Oelgaard <[email-masked]>
Date: Wed, 10 Sep 2008 11:33:27 +0200

> I've looked at it and it looks very good. Will you add references for

Sure, I didn't want to flood the repository with a lot of references if we
decided we didn't need it. Currently, I'm assembling over the reference
elements. Would it be better to use arbitrary elements? I'm just wondering if
certain bugs will be picked up by an element defined with a lot of zeros and ones.
 
> (since it may take some time to run)?

Yes, if the regression test do not fail, the code will return the same values as
last time the verify_tensor/test.py was run. This is why I didn't include it in
the top test.py script.

Kristian

---

From: Anders Logg <[email-masked]>
Date: Wed, 10 Sep 2008 11:35:26 +0200

> certain bugs will be picked up by an element defined with a lot of zeros and ones.

Yes, it would definitely be better to use another element. I suggest
randomizing a triangle and a tet and then sticking those numbers into
the code.

> last time the verify_tensor/test.py was run. This is why I didn't include it in

ok.

-- 
Anders

---

From: "Martin Sandve Alnæs" <[email-masked]>
Date: Wed, 10 Sep 2008 11:48:14 +0200

>> last time the verify_tensor/test.py was run. This is why I didn't include it in

Great! I've been wanting something like this to ease the development
of the UFL based form compiler. I can probably modify your tests with
fairly small changes to use UFL, and make the form compiler an option,
so we can share the tests based on UFL. I'll wait until you've done
the improvements discussed above.

I do wonder though, whether it makes sense to keep two form compiler
projects if both will implement the full UFL language...

--
Martin

---

From: "Martin Sandve Alnæs" <[email-masked]>
Date: Wed, 10 Sep 2008 11:55:53 +0200

You're using instant-clean to clean the cache. I recommend using a
separate cache directory for the regression tests, as is done in the
instant tests. Simply pass "cache_dir = 'test_cache'" to
instant.build_module and instant.import_module. The cache contents are
easier to inspect that way as well.

-- 
Martin

---

From: Kristian Oelgaard <[email-masked]>
Date: Wed, 10 Sep 2008 15:18:29 +0200

Quoting Martin Sandve Alnæs <[email-masked]>:

> >> > > Instead of step 4 it would obviously be better to actually check if
> the new
> >> > code
> >> > > still computes the right thing. To this end I've created a module
> that
> >> > verifies
> >> > > if tabulate_tensor() is correct according to some reference. The
> module
> >> > needs
> >> > > ufc_benchmark to run.
> >> > >

Done. I've created a local folder for the cached forms in verify_tensors, I had
to modify jit.py a bit.

Kristian

---

From: Anders Logg <[email-masked]>
Date: Wed, 10 Sep 2008 21:49:37 +0200

> projects if both will implement the full UFL language...

It would make sense to have just one form compiler and make the code
generation strategy an option. But I suspect it will take a lot of
effort to merge. Do we want to do this now?

-- 
Anders

---

From: "Martin Sandve Alnæs" <[email-masked]>
Date: Wed, 10 Sep 2008 22:19:40 +0200

> effort to merge. Do we want to do this now?

Probably not.

--
Martin

---

From: Kristian Oelgaard <[email-masked]>
Date: Fri, 12 Sep 2008 12:11:13 +0200

> > effort to merge. Do we want to do this now?

I too think this would make sense and I also agree that we don't want to do this
now. However, two things to consider:

1) Having two different compilers make the philosophy behind UFL and UFC
   very obvious.

2) Specifying what strategy to use will add yet another command line option
   that one has to supply.

Kristian

---

From: Anders Logg <[email-masked]>
Date: Fri, 12 Sep 2008 12:18:01 +0200

> 1) Having two different compilers make the philosophy behind UFL and UFC

Yes, this is a good point.

> 2) Specifying what strategy to use will add yet another command line option

That is not such a big problem since we already have -r tensor,
quadrature. We could just add an option -r syfi.

But a bigger problem is dependencies. FFC depends on FIAT and SFC
depends on SyFi (which requires compilation of a C++ library and SWIG) 
so we would need to have optional dependencies.

-- 
Anders

---

From: "Martin Sandve Alnæs" <[email-masked]>
Date: Sat, 13 Sep 2008 17:39:59 +0200

> quadrature. We could just add an option -r syfi.

Nope. SFC also has multiple representations. And in real life,
quadrature is the most important.

--
Martin

---

From: Kristian Oelgaard <[email-masked]>
Date: Sat, 13 Sep 2008 17:53:39 +0200

Quoting Martin Sandve Alnæs <[email-masked]>:

> >> > >> >> > I've looked at it and it looks very good. Will you add
> references
> >> > for
> >> > >> >> > all the forms?
> >> > >> >>
> >> > >> >> Sure, I didn't want to flood the repository with a lot of
> references if
> >> > we
> >> > >> >> certain bugs will be picked up by an element defined with a lot
> of
> >> > zeros and ones.
> >> > >> >
> >> > >> >> Yes, if the regression test do not fail, the code will return the
> same
> >> > values as

Yes, I guess:

ffc -r quadrature -fno-evaluate_basis -fno-evaluate_basis_derivatives
-fsplit_implementation Poisson.form

and

compiler -r ffc -r quadrature -fno-evaluate_basis
-fno-evaluate_basis_derivatives -fsplit_implementation Poisson.form

are almost equally complex.

Kristian

---

From: "Martin Sandve Alnæs" <[email-masked]>
Date: Sat, 13 Sep 2008 17:58:27 +0200

> are almost equally complex.

But the reason I would want to combine the compilers into one isn't to
be able to write "ffc" instead of "sfc", it's so I wouldn't have to
rewrite and maintain the quadrature code generation :-)

--
Martin