Subject: [FEniCS-dev] download statistics

From: "Robert Kirby" <[email-masked]>
Date: Wed, 29 Oct 2008 10:05:52 -0500

The download statistics on fenics.org are interesting and hopefully helpful
making the case that our stuff is widely used when it's time for proposal
writing.

It might also be interesting if we could tally the download requests by
country/continent, on the basis of incoming IP addresses.  We could
keep month-by-month and cumulative statistics on where the downloads
are going.

Rob

---

From: Anders Logg <[email-masked]>
Date: Wed, 29 Oct 2008 16:56:01 +0100

> keep month-by-month and cumulative statistics on where the downloads

I agree, and now that we also have Ubuntu packages and hg access, we
also need to track this. I imagine those downloads may be of the same
size as the tarball downloads.

I think this is already on the todo-list of one of our scientific
programmers, but I don't know where. Johannes? Ilmar?

-- 
Anders

---

From: "Johannes Ring" <[email-masked]>
Date: Thu, 30 Oct 2008 08:27:20 +0100 (CET)

> programmers, but I don't know where. Johannes? Ilmar?

It's not on my todo-list.

Johannes

---

From: Anders Logg <[email-masked]>
Date: Thu, 30 Oct 2008 08:31:10 +0100

> It's not on my todo-list.

ok, I'll gently ask your boss to put it on your todo-list... ;-)

-- 
Anders

---

From: "Johannes Ring" <[email-masked]>
Date: Thu, 30 Oct 2008 08:34:51 +0100 (CET)

> ok, I'll gently ask your boss to put it on your todo-list... ;-)

Okay :-)

Johannes

---

From: Ilmar Wilbers <[email-masked]>
Date: Thu, 30 Oct 2008 11:31:47 +0100

A few ideas:
I tried redefining some hooks to keep track of the number of initial 
clones, but that didn't work. So we have to use the apachelogs located 
in /var/log/apache2/www.fenics.org

A clone is indicated in the log file by 
cmd=changegroup&roots=0000000000000000000000000000000000000000

Any other value but a lot of zeros indicates a pull.

Using zgrep and the following command:
zgrep 'GET /hg/dolfin.*000000000000' fenics-access.log* | grep 2008 | wc -l

shows that the number of clones so far in 2008 is 8757. Grepping further 
for adresses within Simula/UiO (IP's starting with 129.240), shows that 
7960 of those 8757 clones came from within, leaving the number of fresh 
clones from other IP's at 797. The reason for the high number of 
internal requests is due to the buildbot and the cronjob running every 
hour to create snapshots.

Note that checkouts using ssh are not part of these statistics, only 
using http.

Should we create a cronjob running a script the first day of every month 
collecting the statistics from the previous month?

ilmar

Johannes Ring wrote:
> _______________________________________________
> FEniCS-dev mailing list
> [email-masked]
> http://www.fenics.org/mailman/listinfo/fenics-dev

---

From: Anders Logg <[email-masked]>
Date: Thu, 30 Oct 2008 19:22:15 +0100

> Note that checkouts using ssh are not part of these statistics, only 

That should be fine. The ssh clones are only by people with accounts
on fenics.org (developers) and we don't need to count ourselves... :-)

> collecting the statistics from the previous month?

Why not run it every night as we do now?

-- 
Anders

---

From: Ilmar Wilbers <[email-masked]>
Date: Fri, 31 Oct 2008 09:36:20 +0100

> Why not run it every night as we do now?
Sure, i can set up a cronjob. Just thought that there wasn't much point 
in grepping for dates and stuff every day, seeing as the log files are 
available anyway.

Alternatively, one could run it once a week. Every sunday a new log file 
is created (fenics-access.log) and the old one (last weeks one) is 
renamed to fenics-access.log.1. This approach is probably the easiest. I 
could write the scripts (one for the Debian packages as well) in an hour 
or so and add it to crontab, but maybe Johannes is already working on this?

ilmar

---

From: Anders Logg <[email-masked]>
Date: Fri, 31 Oct 2008 09:39:40 +0100

> or so and add it to crontab, but maybe Johannes is already working on this?

We already have a cronjob running that does this every night. The script
(dtrack.py) just needs to be updated.

-- 
Anders

---

From: Ilmar Wilbers <[email-masked]>
Date: Fri, 31 Oct 2008 09:43:59 +0100

> (dtrack.py) just needs to be updated.
Right... Yes, all we need is to add some code for HG and Debian. Like I 
said, I can easily do this, but would need access to the fenics user.

ilmar

---

From: "Johannes Ring" <[email-masked]>
Date: Mon, 10 Nov 2008 14:26:04 +0100 (CET)

Hi,

I have updated the download statistics on fenics.org. It now includes Ubuntu
packages and hg access and it also has a summary of downloads based on
country/continent.

Johannes