Subject: [FFC-dev] BLAS mode verified

From: Anders Logg <[email-masked]>
Date: Tue, 4 Oct 2005 18:56:36 -0500

I have verified that the new FFC BLAS mode works for the stabilized
convection-diffusion module in DOLFIN (which is sufficiently complex
to catch any obvious bugs).

I still need to run some more extensive benchmarks, but the first
indications seem to be that the new BLAS mode is actually slower and
the generated code actually takes longer to compile. Possible
explanations:

1. For the convection-diffusion form, most of the work is computing
the geometry tensor and that is performed in the same way in both
cases. Maybe we could generate some loop-based code for computing the
geometry tensor also.

2. Assigning to array entries (G[i] = ... in BLAS mode) seems to take
longer to compile than assigning to variables (double Gi = ... in
default mode).

3. It actually matters that the default mode of FFC removes any
products with zeros in the tensor product. BLAS does not know about
zeros.

However, for high-order forms and possibly Navier-Stokes, the BLAS
mode could be faster.

To compile a form for use with BLAS mode, just add the flag -f blas
when compiling the form:

    ffc -f blas Poisson.form

Note that any generated XML data files (containing the values of the
reference tensor) must be present in the directory where you run the
executable.

Please try out the new BLAS mode and report any errors/performance
issues.

/Anders

---

From: Johan Jansson <[email-masked]>
Date: Fri, 7 Oct 2005 13:23:20 +0200

> to catch any obvious bugs).

Great job. I'll try it out with the elasticity forms as soon as I have
time.

> products with zeros in the tensor product. BLAS does not know about

This seems to be a severe limitation. For PDE systems with Lagrange
elements (the typical case I guess), there will be lots of zeros. From
previous discussions about FFC/Ferari, the conclusion was that
skipping the zero elements was the dominant optimization for computing
the element matrix.

  Johan

---

From: Robert C Kirby <[email-masked]>
Date: Fri, 7 Oct 2005 06:50:52 -0500

> skipping the zero elements was the dominant optimization for computing

This is exactly why I've been talking about inferring block structure 
inside the compiler and building a block 3x3 local stiffness matrix 
(each block might be different).  For the vector Laplacian, it's block 
diagonal, etc.  You can combine doing the set of all different blocks 
with level 3 BLAS (on each element) with getting to throw away all the 
coarse-grained zeros.

Rob

---

From: Anders Logg <[email-masked]>
Date: Fri, 7 Oct 2005 09:12:43 -0500

> skipping the zero elements was the dominant optimization for computing

Some initial benchmarks I've made indicate that level 2 BLAS
optimization does actually improve the run-time speed of assembly for
high-order elements. This improvement will be even better when we move
to level 3 BLAS, which is just a simple extension of what we already
have now in DOLFIN/FFC. However, the main advantage of the BLAS mode
is probably reduced compile-time (with gcc) for high-order elements
(reduced by a factor 100 for degree 8 Poisson).

I'll post some more details later from the benchmarks.

/Anders