[stats-dev] new ogr-ver-incr method

[Chris Hodson]

I have started to run into some problems getting the incremental version
of ogr-verification (ogr-ver-incr or just -incr) to work properly.
Debugging this has been dragging and progress is slow, to say the least.
While I'm not giving up entirely on this, I think I can get a simpler
(code-wise) version running a bit quicker.  This probably won't be as
fast as what I'm trying to get done right now (see
http://cvs.distributed.net/viewcvs.cgi/stats-proc/ogr-verification/daily_update.sql?rev=1.17),
but it should be much much faster than what is currently in place and
much easier to debug.  I wrote up an outline of the new method and some
pseudo-code.  It assumes a working knowledge of the tables that are
there now.  If you're interested and don't know about the tables I'd be
happy to write something up.  Any comments would be appreciated.


1. Load day_results with today's work (normalized and de-duped)

2. Create affected_ids
	select L2.id from ogr_idlookup L1, ogr_idlookup L2
	where L1.retire_date = :RUNDATE
	AND L1.stats_id = L2.stats_id

3. Generate list of affected stubs
	select stub_id, nodecount from day_results
	UNION
	select stub_id, nodecount from ogr_results r, affected_ids aid
		where r.id = a.id

4. Delete stubs affected
	delete from ogr_summary
		where ogr_summary.stub_id = affected_stubs.stub_id
		and ogr_summary.nodecount = affected_stubs.nodecount

5. Generate day_summary
	SELECT stub_id, nodecount, count(DISTINCT l.stats_id) AS ids,
		max(p.version) AS max_version
		FROM day_results dr, OGR_idlookup l, platform p
		WHERE l.id = dr.id
			AND p.platform_id = dr.platform_id
			AND NOT dr.in_results
		GROUP BY stub_id, nodecount

6. Insert day_summary into ogr_summary (could really combine steps 5 and 6)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
Url : http://lists.distributed.net/pipermail/stats-dev/attachments/20031209/7c93a56e/attachment.bin