Subject: [Ffc] Patch introducing memory cache

From: Johan Hake <[email-masked]>
Date: Mon, 16 Aug 2010 20:20:35 -0700

Hello!

Attached is a patch that introduce a small memory cache for preprocess forms. 
Fixing DOLFIN bug:

  <https://bugs.launchpad.net/dolfin/+bug/612579>

Johan
# Bazaar merge directive format 2 (Bazaar 0.90)
# revision_id: [email-masked]
# target_branch: bzr+ssh://bazaar.launchpad.net/~ffc-core/ffc/main/
# testament_sha1: b1b8c7ad15349aa80c014dbab5ecea3934c3eb2f
# timestamp: 2010-08-16 20:13:52 -0700
# base_revision_id: [email-masked]\
#   e1egwzz4kq5ksqae
# 
# Begin patch
=== modified file 'ChangeLog'
--- ChangeLog	2010-07-23 17:22:13 +0000
+++ ChangeLog	2010-08-17 03:10:45 +0000
@@ -1,10 +1,11 @@
+ - Added memory cache in jit(), for preprocessed forms
  - Added support for Conditional and added demo/Conditional.ufl.
  - Added support for new geometric quantity Circumradius in UFL.
  - Added support for new geometric quantity CellVolume in UFL.
 0.9.3 [2010-07-01]
  - Make global_dimension for Real return an int instead of double, bug # 592088
  - Add support for facet normal in 1D.
- - Expose -feliminate_zeros for quadrature optimisations to give user more 
+ - Expose -feliminate_zeros for quadrature optimisations to give user more
    control
  - Remove return of form in compile_form
  - Remove object_names argument to compile_element
@@ -14,7 +15,7 @@
  - Added support for Bubble elements
  - Added support for UFL.SpatialCoordinate.
 0.9.2 [2010-02-17]
- - Bug fix in removal of unused variables in Piola-mapped terms for tensor 
+ - Bug fix in removal of unused variables in Piola-mapped terms for tensor
    representation
 0.9.1 [2010-02-15]
  - Add back support for FErari optimizations

=== modified file 'ffc/jitcompiler.py'
--- ffc/jitcompiler.py	2010-02-14 22:05:02 +0000
+++ ffc/jitcompiler.py	2010-08-17 03:10:45 +0000
@@ -12,7 +12,7 @@
 # Last changed: 2010-02-14
 
 # Python modules
-import os
+import os, sys
 import instant
 import ufc_utils
 
@@ -42,6 +42,12 @@
 # Set debug level for Instant
 instant.set_logging_level("warning")
 
+# Memory cache for preprocessed forms
+_memory_cache = {}
+
+# Counter to prevent memory leak
+_memory_check = 1
+
 def jit(object, parameters=None, common_cell=None):
     """Just-in-time compile the given form or element
 
@@ -59,6 +65,7 @@
 
 def jit_form(form, parameters=None, common_cell=None):
     "Just-in-time compile the given form."
+    global _memory_check
 
     # Check that we get a Form
     if not isinstance(form, Form):
@@ -72,10 +79,28 @@
     set_prefix(parameters["log_prefix"])
 
     # Preprocess form
-    if form.form_data() is None:
+    
+    # First check if form is preprocessed
+    if form.form_data() is not None:
+        preprocessed_form = form
+
+    # Second check memory cache
+    elif _memory_cache.has_key(id(form)):
+        preprocessed_form = _memory_cache[id(form)]
+
+    # Else preprocess form and store in memory cache
+    else:
         preprocessed_form = preprocess(form, common_cell=common_cell)
-    else:
-        preprocessed_form = form
+        _memory_cache[id(form)] = preprocessed_form
+
+        # For each 10th time the refcount of the cached form are checked
+        # and superflous forms are poped
+        if (_memory_check % 10) == 0:
+            for key, cached_form in _memory_cache.items():
+                if sys.getrefcount(cached_form) < 6:
+                    _memory_cache.pop(key)
+        else:
+            _memory_check += 1
 
     # Wrap input
     jit_object = JITObject(form, preprocessed_form, parameters)

# Begin bundle

---

From: Anders Logg <[email-masked]>
Date: Tue, 17 Aug 2010 09:10:05 +0200

Great!

It will be interesting to see what the benchbot says tomorrow.

--
Anders

> IyBCYXphYXIgcmV2aXNpb24gYnVuZGxlIHY0CiMKQlpoOTFBWSZTWb3p5ZEAAr//gERUQABab//3
> e1fcCr////pQBg3bu91vedUW7w5m10xoOgkkgm1NNU8yGg0U9o0pm1I2TJNAaNDTTRkBKJpDEnk0
> 0pP0U9QHqeoAADIAAAAlBGpNoqfimiPKZlN6Q1GnqAwmCAAZNAYoFPVPaU8pppp5RptQaA9QAAGg
> aAaAkiE00ApmI0qeaU8UeTEmgD1ADQAaI0BR1fU/2fw2M3hed9vnZL3vrsGHQvwNIQdsDV1OlFhj
> UdbJqWrgy4VIAsXcUtdJK7Urm06o12oiBYqCDghLHFjS67YMYvJBEDJafQmhfEJ03a0/VrHDPvMv
> oVMc98WSSCSEd0uT7j35sNOs2QQ+tbhmKWxOdU7J+a9k00javNFPB86Sf2lPre4gUW+pPFkfR7NO
> B/NGtSRFomRdxMW27WnMcm6X3NKuErMMIeQPGNIlKKaeDVj4tvT9sWKV0RBkHf4nrd39XPbOD2K+
> H8vb1YfK7dbOTqjZDtZfwZnnMreC6NKmVWU9ivbMdNdtoF+V3MHMhWIJ8ji7SDqLzE5ZYVEN0gMG
> ScdO/cwLMXUyEzvLd8B5EZfC4Ck3LBQJ1kgeGAkioKNSYVwm0BU1FgEia0DKqEg7bTSDwspEABKs
> NwPEjGcvhZSYCWF/ThpMxy9KuyBhqy412EDkReBUcG0CbvRwTpWS5TM0VJmZt5EffPIVbVQ1VBM8
> 8YIUHZwSO/FLkq3PGUFdw4ccVhYYnM1ihliW74kGwHdkDkZQMiXHhbXvETUjziP5A13Twte11ojw
> kikFeZwvNCI4CNanwzflXjmRLw3IuCB7luA/0CkSlSDJllbJMh8wVobki/PE2JYGhWgrOG3qOmOL
> zSWlGmQUXKuCpJHcQLD476uoSoOA1k6IV6DsMZFhoGVwXLu3Ey0ZyxuRlHAWZLzaXQ4GaP2YUiak
> Zk33pmScmoFCOuzYa3FSh4MTe2mcSq1xuGc5SepDaCLWAK4nGJEIqnfCXAzsQGCmSS0TuBBj9TtF
> rwMUDjxqMlwqKL8Xor/qToRJiX5bl7mfJBoK/WIX3oYY2odwZGC8fRutuvtiucroPuO2mUDbGKtW
> Z+BrI7S3UHRy6Z3iXXRB58VIQ496PShXpDvxoMFOD0maK7eHKWZEBgSfFEanuvv6qTsCkkqOXJcS
> 4ckuDXD3YvbraFQRqHaCmhilrqMxvZMhjn1PnI9o12yKcCaqvjvhSL4Yg6wQzFTckLPthyLzQCAX
> skww9a3hrzjM2zJ0DslWG0ZI6wuRv581M5QY09pgn1QTJ8zuTci7Rfwk+lh9gyMkm15oQHSDKbm8
> mxoXAwONrc67C4q3t9Rsl5eMhPVKRYwHdZdm3ZySHJOEjPiuXfbdga7mI1j549LQ9zjSHyy5R3DO
> eLCU1RW5MmBUwVZtMOUnhCXZrYyTYUQWCojpUpo6DU3VjbDO2Yxsyr0V22uLy7issWHXxgFamGZt
> 6duYlYG30hn0BkJIhCL44shBrNli8CtgHkGkULtU/tcFCMyKREEUWaKypIv3CpYs3katCGj3VZ8/
> h6FoQtgqHqRp6rovVzIhfPVlh8nou+MpL0I8w0wbaHk8mvYdhrxDJUAzhGj9OtHR7vRUVZ5IXcGZ
> kjKAd/jUK4b0kKnTVrE36/hA+u/L1AsQTM1ELvdY4GQtBGEQ96n7FWAgrmpToiT1no82RUiseEGH
> Q4zq3z66vOcbK20GSRYgGMN+vEI384gTgTjK4ULLLWJKIX71NmtCmo3chZI4M6ETIIgwwwic42nR
> o58mVcWnswwOoV6PSOk3q5bVyF8Y2wFtti4gTA8Fou5UmqWuh/yTLsTez7ZIrhMIM1r04s6W5Bl3
> xVsm7qIQ8wK4vDqk7fcAwI4e1k7b8Kpm0qp3XUnXXPkpBYybJcGxPqsQ5aG+lvFXoOcHsRmNiOZP
> lUzTvcY8wIc0g0Td2yTuZDwdbEWWPaZpuWiUS5alC2NG188/bgrri0DojdLtArMEGYFUo4ymmR4+
> Si5ceuSLwN7phwUTwXDSi5V+xY/9/4u5IpwoSF708siA

> _______________________________________________
> Mailing list: https://launchpad.net/~ffc
> Post to     : [email-masked]
> Unsubscribe : https://launchpad.net/~ffc
> More help   : https://help.launchpad.net/ListHelp

---

From: "Garth N. Wells" <[email-masked]>
Date: Tue, 17 Aug 2010 12:06:23 +0100

> It will be interesting to see what the benchbot says tomorrow.

The change broke

    demo/adaptivity/adaptive-poisson/python

Garth

---

From: Johan Hake <[email-masked]>
Date: Tue, 17 Aug 2010 08:45:55 -0700

>     demo/adaptivity/adaptive-poisson/python

Should be fixed with the inclusion of the sent patch.

Johan

> > >   - Make global_dimension for Real return an int instead of double, bug
> > >   # 592088 - Add support for facet normal in 1D.
> > > 
> > > - - Expose -feliminate_zeros for quadrature optimisations to give user
> > > more + - Expose -feliminate_zeros for quadrature optimisations to give
> > > user more
> > > 
> > >     control
> > >   
> > > - - Bug fix in removal of unused variables in Piola-mapped terms for
> > > tensor + - Bug fix in removal of unused variables in Piola-mapped
> > > terms for tensor
> > > 
> > >     representation
> > >  
> > >  0.9.1 [2010-02-15]
> > >  
> > > +        # For each 10th time the refcount of the cached form are
> > > checked +        # and superflous forms are poped
> > > lUzTvcY8wIc0g0Td2yTuZDwdbEWWPaZpuWiUS5alC2NG188/bgrri0DojdLtArMEGYFUo4
> > > ymmR4+ Si5ceuSLwN7phwUTwXDSi5V+xY/9/4u5IpwoSF708siA
> > >