Replication monitor

Once per hour a segment database at each site is queried to collect a list of segments corresponding to LFNs published during the last two days. At the observatories segments corresponding to raw frames from /frames and /fb0_frames are also computed by directly scanning the disks (to check if one can trust segment database for a given time interval or not everything has been published yet: disk2disk script, LDAS, publishing scripts, replication could be down). The exact end time of the considered time interval is chosen as follows: (current time - 300 seconds)/60*60. The integer division is used to minimize the difference in current times of script execution at different sites (the scripts run as cron job at each site and depending on the load on gateways 'current time' might differ by a second or two between the sites). 300 second offset is used to allow for possible delays in replication between sites. Below this time segment is called query segment. To be able to compare the list of replicated and original segments, we would like in ideal to have the same query segment at all the sites. Once per hour at each site a script runs that collects the output from the above query from all 3 sites, compares the published segments between the sites and generates at each site a web page (which should be the same unless there are network problems): LHO, LLO, CIT. One might also find it useful to get the latest pickled objects (LHO.out, LLO.out, CIT.out) of a class containing all the information published on this web page. To be able to use those pickled objects in your python program, import a stripped down version of the Stats class before loading the objects. The are also summary pickled Stats objects that provide the same information for the whole run up to the time the script ran last time (summary_LHO.out, summary_LLO.out). The summary objects are obtained not by querying the database (that takes most of the day for the whole S5) but by adding a new SITE.out object (covering last two days) to the previous summary object summary_SITE.out (that covers last two days minus an hour). Since the segments in Stats objects are already coalesced, this is an instantaneous operation.

The information is organized in a table with a row corresponding to a replication source and a column corresponding to a replication target. For example, LLO row compares L-R-* LFNs published at LLO and other sites.

Diagonal elements in the table contain the following information:

Query segment: GPS start, end times, duration.
Same query segment in a more readable format.
Coalesced segment list of LFNs that got inside the query segment (LFN segments):
select start_time, end_time from LFN where start_time > query_startTime and end_time < query_endTime and name like 'site-R-%'
Each segment has index, start, end, duration.
Total duration of the above LFN segments.
Gap segments: difference between query segment and LFN segments. It is OK to have gaps of duration less than 32 seconds at the beginning and end of the query segment. Other gaps indicate either gaps in real data or gaps in what was published; such gaps are shown in red to indicate possible problems.
Total duration of the above gaps.
Intersection of a union of segments covered by raw frames from /frames and /fb0_frames with a query segment.

Offdiagonal elements contain the following information:

Difference between query segments of replication source and target sites; it should be 0,0 in ideal, if it is not, the corresponding line is shown in yellow.
Difference in total duration of LFN segments between the replication source and target sites; if the difference is not zero, the corresponding entry is shown in red.
Difference between the replication source and target LFN segments; if there is no difference, the line is shown in green, if there is a difference, it is shown in red. This is the most important entry in the table that indicates if replication is working or not.

-->LHO

-->LLO

-->CIT

LHO-->

897056700,897229500,172800;

Jun 09 2008 07:24:46 PDT, Jun 11 2008 07:24:46 PDT

1,897056704,897067616,10912;
2,897070720,897071744,1024;
3,897072032,897075616,3584;
4,897075840,897076832,992;
5,897111008,897112000,992;
6,897211008,897212000,992;

18496

1,897056700,897056704,4;

2,897067616,897070720,3104;

3,897071744,897072032,288;

4,897075616,897075840,224;

5,897076832,897111008,34176;

6,897112000,897211008,99008;

7,897212000,897229500,17500;

154304

1,897056700,897229500,172800;

19180800,19180800

-154112

1,897056704,897067616,10912;
2,897070720,897071744,1024;
3,897072032,897075616,3584;
4,897075840,897076832,992;
5,897111008,897112000,992;
6,897211008,897212000,992;

0,0

18496

1,897056704,897067616,10912;
2,897070720,897071744,1024;
3,897072032,897075616,3584;
4,897075840,897076832,992;
5,897111008,897112000,992;
6,897211008,897212000,992;

LLO-->

-19180800,-19180800

170528

1,877875904,878048416,172512;

877875900,878048700,172800;

Oct 31 2007 09:24:46 CDT, Nov 02 2007 09:24:46 CDT

1,877875904,878048416,172512;

172512

1,877875900,877875904,4;

2,878048416,878048700,284;

288

1,877875900,878048700,172800;

-19180800,-19180800

172512

1,877875904,878048416,172512;

CIT-->

0,0

19180800,19180800

897056700,897229500,172800;

Jun 09 2008 07:24:46 PDT, Jun 11 2008 07:24:46 PDT

1,897056700,897229500,172800;

172800

Unpublished raw frames for the whole S5 run

The table below shows the difference between segments corresponding to raw frames from /frames, /fb0_frames and /archive from the site of the origin and the segments corresponding to the published (in the segment database of the raw frame origin) LFNs.

Site	GPS start	GPS end	duration	Date start	Date end
H	878784800	879283840	499040	Nov 10 2007 18:53:06 PST	Nov 16 2007 13:30:26 PST
	879284320	879550048	265728	Nov 16 2007 13:38:26 PST	Nov 19 2007 15:27:14 PST
	879550144	879550208	64	Nov 19 2007 15:28:50 PST	Nov 19 2007 15:29:54 PST
	879550368	879550432	64	Nov 19 2007 15:32:34 PST	Nov 19 2007 15:33:38 PST
	879550560	880404064	853504	Nov 19 2007 15:35:46 PST	Nov 29 2007 12:40:50 PST
	880404608	880831680	427072	Nov 29 2007 12:49:54 PST	Dec 04 2007 11:27:46 PST
	880833248	881000032	166784	Dec 04 2007 11:53:54 PST	Dec 06 2007 10:13:38 PST
	881002176	882043616	1041440	Dec 06 2007 10:49:22 PST	Dec 18 2007 12:06:42 PST
	882044096	882046880	2784	Dec 18 2007 12:14:42 PST	Dec 18 2007 13:01:06 PST
	882047392	882047552	160	Dec 18 2007 13:09:38 PST	Dec 18 2007 13:12:18 PST
	882048064	883333312	1285248	Dec 18 2007 13:20:50 PST	Jan 02 2008 10:21:38 PST
	883333824	883441920	108096	Jan 02 2008 10:30:10 PST	Jan 03 2008 16:31:46 PST
	883442464	884311008	868544	Jan 03 2008 16:40:50 PST	Jan 13 2008 17:56:34 PST
	884312000	884382592	70592	Jan 13 2008 18:13:06 PST	Jan 14 2008 13:49:38 PST
	884383008	884386976	3968	Jan 14 2008 13:56:34 PST	Jan 14 2008 15:02:42 PST
	884387456	884411008	23552	Jan 14 2008 15:10:42 PST	Jan 14 2008 21:43:14 PST
	884412000	884511008	99008	Jan 14 2008 21:59:46 PST	Jan 16 2008 01:29:54 PST
	884512000	884611008	99008	Jan 16 2008 01:46:26 PST	Jan 17 2008 05:16:34 PST
	884612000	884711008	99008	Jan 17 2008 05:33:06 PST	Jan 18 2008 09:03:14 PST
	884712000	884811008	99008	Jan 18 2008 09:19:46 PST	Jan 19 2008 12:49:54 PST
	884812000	884911008	99008	Jan 19 2008 13:06:26 PST	Jan 20 2008 16:36:34 PST
	884912000	885011008	99008	Jan 20 2008 16:53:06 PST	Jan 21 2008 20:23:14 PST
	885012000	885111008	99008	Jan 21 2008 20:39:46 PST	Jan 23 2008 00:09:54 PST
	885112000	885211008	99008	Jan 23 2008 00:26:26 PST	Jan 24 2008 03:56:34 PST
	885212000	885236736	24736	Jan 24 2008 04:13:06 PST	Jan 24 2008 11:05:22 PST
	885237248	885311008	73760	Jan 24 2008 11:13:54 PST	Jan 25 2008 07:43:14 PST
	885312000	885411008	99008	Jan 25 2008 07:59:46 PST	Jan 26 2008 11:29:54 PST
	885412000	885511008	99008	Jan 26 2008 11:46:26 PST	Jan 27 2008 15:16:34 PST
	885512000	885611008	99008	Jan 27 2008 15:33:06 PST	Jan 28 2008 19:03:14 PST
	885612000	885711008	99008	Jan 28 2008 19:19:46 PST	Jan 29 2008 22:49:54 PST
	885712000	885811008	99008	Jan 29 2008 23:06:26 PST	Jan 31 2008 02:36:34 PST
	885812000	885911008	99008	Jan 31 2008 02:53:06 PST	Feb 01 2008 06:23:14 PST
	885912000	886011008	99008	Feb 01 2008 06:39:46 PST	Feb 02 2008 10:09:54 PST
	886012000	886111008	99008	Feb 02 2008 10:26:26 PST	Feb 03 2008 13:56:34 PST
	886112000	886211008	99008	Feb 03 2008 14:13:06 PST	Feb 04 2008 17:43:14 PST
	886212000	886272832	60832	Feb 04 2008 17:59:46 PST	Feb 05 2008 10:53:38 PST
	886276480	886311008	34528	Feb 05 2008 11:54:26 PST	Feb 05 2008 21:29:54 PST
	886312000	886411008	99008	Feb 05 2008 21:46:26 PST	Feb 07 2008 01:16:34 PST
	886412000	886460480	48480	Feb 07 2008 01:33:06 PST	Feb 07 2008 15:01:06 PST
	886461504	886462016	512	Feb 07 2008 15:18:10 PST	Feb 07 2008 15:26:42 PST
	886462208	886511008	48800	Feb 07 2008 15:29:54 PST	Feb 08 2008 05:03:14 PST
	886512000	886611008	99008	Feb 08 2008 05:19:46 PST	Feb 09 2008 08:49:54 PST
	886612000	886711008	99008	Feb 09 2008 09:06:26 PST	Feb 10 2008 12:36:34 PST
	886712000	886789696	77696	Feb 10 2008 12:53:06 PST	Feb 11 2008 10:28:02 PST
	886789824	886794048	4224	Feb 11 2008 10:30:10 PST	Feb 11 2008 11:40:34 PST
	886794112	886804928	10816	Feb 11 2008 11:41:38 PST	Feb 11 2008 14:41:54 PST
	886804992	886811008	6016	Feb 11 2008 14:42:58 PST	Feb 11 2008 16:23:14 PST
	886812000	886820800	8800	Feb 11 2008 16:39:46 PST	Feb 11 2008 19:06:26 PST
	886820928	886822464	1536	Feb 11 2008 19:08:34 PST	Feb 11 2008 19:34:10 PST
	886822496	886882304	59808	Feb 11 2008 19:34:42 PST	Feb 12 2008 12:11:30 PST
	886882400	886883264	864	Feb 12 2008 12:13:06 PST	Feb 12 2008 12:27:30 PST
	886883456	886911008	27552	Feb 12 2008 12:30:42 PST	Feb 12 2008 20:09:54 PST
	886912000	887011008	99008	Feb 12 2008 20:26:26 PST	Feb 13 2008 23:56:34 PST
	887012000	887073088	61088	Feb 14 2008 00:13:06 PST	Feb 14 2008 17:11:14 PST
	887078016	887111008	32992	Feb 14 2008 18:33:22 PST	Feb 15 2008 03:43:14 PST
	887112000	887211008	99008	Feb 15 2008 03:59:46 PST	Feb 16 2008 07:29:54 PST
	887212000	887311008	99008	Feb 16 2008 07:46:26 PST	Feb 17 2008 11:16:34 PST
	887312000	887351968	39968	Feb 17 2008 11:33:06 PST	Feb 17 2008 22:39:14 PST
	887374688	887376416	1728	Feb 18 2008 04:57:54 PST	Feb 18 2008 05:26:42 PST
	887383072	887397440	14368	Feb 18 2008 07:17:38 PST	Feb 18 2008 11:17:06 PST
	887398112	887410432	12320	Feb 18 2008 11:28:18 PST	Feb 18 2008 14:53:38 PST
	887412192	887429664	17472	Feb 18 2008 15:22:58 PST	Feb 18 2008 20:14:10 PST
	887475296	887483968	8672	Feb 19 2008 08:54:42 PST	Feb 19 2008 11:19:14 PST
	887484416	887494944	10528	Feb 19 2008 11:26:42 PST	Feb 19 2008 14:22:10 PST
	887496640	887499360	2720	Feb 19 2008 14:50:26 PST	Feb 19 2008 15:35:46 PST
	887501632	887506496	4864	Feb 19 2008 16:13:38 PST	Feb 19 2008 17:34:42 PST
	887531584	887556000	24416	Feb 20 2008 00:32:50 PST	Feb 20 2008 07:19:46 PST
	887562336	887605376	43040	Feb 20 2008 09:05:22 PST	Feb 20 2008 21:02:42 PST
	887631776	887631840	64	Feb 21 2008 04:22:42 PST	Feb 21 2008 04:23:46 PST
	887639008	887693024	54016	Feb 21 2008 06:23:14 PST	Feb 21 2008 21:23:30 PST
	887701344	887702144	800	Feb 21 2008 23:42:10 PST	Feb 21 2008 23:55:30 PST
	887729952	887730848	896	Feb 22 2008 07:38:58 PST	Feb 22 2008 07:53:54 PST
	887737248	887737888	640	Feb 22 2008 09:40:34 PST	Feb 22 2008 09:51:14 PST
	887738656	887776960	38304	Feb 22 2008 10:04:02 PST	Feb 22 2008 20:42:26 PST
	887779360	887780352	992	Feb 22 2008 21:22:26 PST	Feb 22 2008 21:38:58 PST
	887818944	887853056	34112	Feb 23 2008 08:22:10 PST	Feb 23 2008 17:50:42 PST
	887864800	887873856	9056	Feb 23 2008 21:06:26 PST	Feb 23 2008 23:37:22 PST
	887901088	887903072	1984	Feb 24 2008 07:11:14 PST	Feb 24 2008 07:44:18 PST
	887919328	887920064	736	Feb 24 2008 12:15:14 PST	Feb 24 2008 12:27:30 PST
	887942208	887944576	2368	Feb 24 2008 18:36:34 PST	Feb 24 2008 19:16:02 PST
	887965280	887970176	4896	Feb 25 2008 01:01:06 PST	Feb 25 2008 02:22:42 PST
	887997312	888003584	6272	Feb 25 2008 09:54:58 PST	Feb 25 2008 11:39:30 PST
	888010048	888010336	288	Feb 25 2008 13:27:14 PST	Feb 25 2008 13:32:02 PST
	888022784	888025184	2400	Feb 25 2008 16:59:30 PST	Feb 25 2008 17:39:30 PST
	888027040	888031232	4192	Feb 25 2008 18:10:26 PST	Feb 25 2008 19:20:18 PST
	888032768	888033440	672	Feb 25 2008 19:45:54 PST	Feb 25 2008 19:57:06 PST
	888038880	888039424	544	Feb 25 2008 21:27:46 PST	Feb 25 2008 21:36:50 PST
	888062528	888063712	1184	Feb 26 2008 04:01:54 PST	Feb 26 2008 04:21:38 PST
	888077728	888080864	3136	Feb 26 2008 08:15:14 PST	Feb 26 2008 09:07:30 PST
	888082624	888085344	2720	Feb 26 2008 09:36:50 PST	Feb 26 2008 10:22:10 PST
	888131520	888136128	4608	Feb 26 2008 23:11:46 PST	Feb 27 2008 00:28:34 PST
	888165696	888173408	7712	Feb 27 2008 08:41:22 PST	Feb 27 2008 10:49:54 PST
	888174400	888178912	4512	Feb 27 2008 11:06:26 PST	Feb 27 2008 12:21:38 PST
	888180160	888185632	5472	Feb 27 2008 12:42:26 PST	Feb 27 2008 14:13:38 PST
	888186464	888192096	5632	Feb 27 2008 14:27:30 PST	Feb 27 2008 16:01:22 PST
	888203328	888204576	1248	Feb 27 2008 19:08:34 PST	Feb 27 2008 19:29:22 PST
	888209024	888210048	1024	Feb 27 2008 20:43:30 PST	Feb 27 2008 21:00:34 PST
	888213984	888214976	992	Feb 27 2008 22:06:10 PST	Feb 27 2008 22:22:42 PST
	888226304	888229632	3328	Feb 28 2008 01:31:30 PST	Feb 28 2008 02:26:58 PST
	888247104	888247136	32	Feb 28 2008 07:18:10 PST	Feb 28 2008 07:18:42 PST
	888253984	888254752	768	Feb 28 2008 09:12:50 PST	Feb 28 2008 09:25:38 PST
	888256416	888260544	4128	Feb 28 2008 09:53:22 PST	Feb 28 2008 11:02:10 PST
	888263232	888264416	1184	Feb 28 2008 11:46:58 PST	Feb 28 2008 12:06:42 PST
	888265184	888266208	1024	Feb 28 2008 12:19:30 PST	Feb 28 2008 12:36:34 PST
	888269152	888272832	3680	Feb 28 2008 13:25:38 PST	Feb 28 2008 14:26:58 PST
	888275520	888277344	1824	Feb 28 2008 15:11:46 PST	Feb 28 2008 15:42:10 PST
	888281024	888281952	928	Feb 28 2008 16:43:30 PST	Feb 28 2008 16:58:58 PST
	888287008	888292160	5152	Feb 28 2008 18:23:14 PST	Feb 28 2008 19:49:06 PST
	888294080	888295520	1440	Feb 28 2008 20:21:06 PST	Feb 28 2008 20:45:06 PST
	888298272	888300512	2240	Feb 28 2008 21:30:58 PST	Feb 28 2008 22:08:18 PST
	888310720	888311008	288	Feb 29 2008 00:58:26 PST	Feb 29 2008 01:03:14 PST
	888312000	888314016	2016	Feb 29 2008 01:19:46 PST	Feb 29 2008 01:53:22 PST
	888335744	888339072	3328	Feb 29 2008 07:55:30 PST	Feb 29 2008 08:50:58 PST
	888346048	888348832	2784	Feb 29 2008 10:47:14 PST	Feb 29 2008 11:33:38 PST
	888356000	888357664	1664	Feb 29 2008 13:33:06 PST	Feb 29 2008 14:00:50 PST
	888362976	888363168	192	Feb 29 2008 15:29:22 PST	Feb 29 2008 15:32:34 PST
	888380736	888383360	2624	Feb 29 2008 20:25:22 PST	Feb 29 2008 21:09:06 PST
	888407008	888409952	2944	Mar 01 2008 03:43:14 PST	Mar 01 2008 04:32:18 PST
	888412512	888447968	35456	Mar 01 2008 05:14:58 PST	Mar 01 2008 15:05:54 PST
	888453248	888454016	768	Mar 01 2008 16:33:54 PST	Mar 01 2008 16:46:42 PST
	888470688	888472448	1760	Mar 01 2008 21:24:34 PST	Mar 01 2008 21:53:54 PST
	888473312	888474176	864	Mar 01 2008 22:08:18 PST	Mar 01 2008 22:22:42 PST
	888478528	888478944	416	Mar 01 2008 23:35:14 PST	Mar 01 2008 23:42:10 PST
	888501408	888511008	9600	Mar 02 2008 05:56:34 PST	Mar 02 2008 08:36:34 PST
	888512000	888512032	32	Mar 02 2008 08:53:06 PST	Mar 02 2008 08:53:38 PST
	888524448	888524896	448	Mar 02 2008 12:20:34 PST	Mar 02 2008 12:28:02 PST
	888529440	888534784	5344	Mar 02 2008 13:43:46 PST	Mar 02 2008 15:12:50 PST
	888572800	888574432	1632	Mar 03 2008 01:46:26 PST	Mar 03 2008 02:13:38 PST
	888588544	888588704	160	Mar 03 2008 06:08:50 PST	Mar 03 2008 06:11:30 PST
	888589824	888590016	192	Mar 03 2008 06:30:10 PST	Mar 03 2008 06:33:22 PST
	888594720	888595008	288	Mar 03 2008 07:51:46 PST	Mar 03 2008 07:56:34 PST
	888599296	888611008	11712	Mar 03 2008 09:08:02 PST	Mar 03 2008 12:23:14 PST
	888612000	888636256	24256	Mar 03 2008 12:39:46 PST	Mar 03 2008 19:24:02 PST
	888639520	888641664	2144	Mar 03 2008 20:18:26 PST	Mar 03 2008 20:54:10 PST
	888693888	888697088	3200	Mar 04 2008 11:24:34 PST	Mar 04 2008 12:17:54 PST
	888700096	888704448	4352	Mar 04 2008 13:08:02 PST	Mar 04 2008 14:20:34 PST
	888707776	888709280	1504	Mar 04 2008 15:16:02 PST	Mar 04 2008 15:41:06 PST
	888718880	888730720	11840	Mar 04 2008 18:21:06 PST	Mar 04 2008 21:38:26 PST
	888756768	888756832	64	Mar 05 2008 04:52:34 PST	Mar 05 2008 04:53:38 PST
	888762464	888767488	5024	Mar 05 2008 06:27:30 PST	Mar 05 2008 07:51:14 PST
	888778144	888781792	3648	Mar 05 2008 10:48:50 PST	Mar 05 2008 11:49:38 PST
	888790752	888811008	20256	Mar 05 2008 14:18:58 PST	Mar 05 2008 19:56:34 PST
	888812000	888819104	7104	Mar 05 2008 20:13:06 PST	Mar 05 2008 22:11:30 PST
	888832416	888833408	992	Mar 06 2008 01:53:22 PST	Mar 06 2008 02:09:54 PST
	888866976	888874304	7328	Mar 06 2008 11:29:22 PST	Mar 06 2008 13:31:30 PST
	888875776	888876544	768	Mar 06 2008 13:56:02 PST	Mar 06 2008 14:08:50 PST
	888879744	888882784	3040	Mar 06 2008 15:02:10 PST	Mar 06 2008 15:52:50 PST
	888885440	888886528	1088	Mar 06 2008 16:37:06 PST	Mar 06 2008 16:55:14 PST
	888899104	888902688	3584	Mar 06 2008 20:24:50 PST	Mar 06 2008 21:24:34 PST
	888949408	888951232	1824	Mar 07 2008 10:23:14 PST	Mar 07 2008 10:53:38 PST
	888954656	888956032	1376	Mar 07 2008 11:50:42 PST	Mar 07 2008 12:13:38 PST
	888958720	888960992	2272	Mar 07 2008 12:58:26 PST	Mar 07 2008 13:36:18 PST
	888963264	888963904	640	Mar 07 2008 14:14:10 PST	Mar 07 2008 14:24:50 PST
	888966048	888968576	2528	Mar 07 2008 15:00:34 PST	Mar 07 2008 15:42:42 PST
	888969344	888969696	352	Mar 07 2008 15:55:30 PST	Mar 07 2008 16:01:22 PST
	888970688	888997152	26464	Mar 07 2008 16:17:54 PST	Mar 07 2008 23:38:58 PST
	889010592	889011008	416	Mar 08 2008 03:22:58 PST	Mar 08 2008 03:29:54 PST
	889033440	889034272	832	Mar 08 2008 09:43:46 PST	Mar 08 2008 09:57:38 PST
	889039296	889039392	96	Mar 08 2008 11:21:22 PST	Mar 08 2008 11:22:58 PST
	889042496	889042752	256	Mar 08 2008 12:14:42 PST	Mar 08 2008 12:18:58 PST
	889060288	889060320	32	Mar 08 2008 17:11:14 PST	Mar 08 2008 17:11:46 PST
	889125248	889129728	4480	Mar 09 2008 12:13:54 PDT	Mar 09 2008 13:28:34 PDT
	889139680	889163200	23520	Mar 09 2008 16:14:26 PDT	Mar 09 2008 22:46:26 PDT
	889172128	889173600	1472	Mar 10 2008 01:15:14 PDT	Mar 10 2008 01:39:46 PDT
	889201664	889204704	3040	Mar 10 2008 09:27:30 PDT	Mar 10 2008 10:18:10 PDT
	889209920	889210368	448	Mar 10 2008 11:45:06 PDT	Mar 10 2008 11:52:34 PDT
	889214656	889215104	448	Mar 10 2008 13:04:02 PDT	Mar 10 2008 13:11:30 PDT
	889218208	889220256	2048	Mar 10 2008 14:03:14 PDT	Mar 10 2008 14:37:22 PDT
	889248896	889251904	3008	Mar 10 2008 22:34:42 PDT	Mar 10 2008 23:24:50 PDT
	889258400	889260544	2144	Mar 11 2008 01:13:06 PDT	Mar 11 2008 01:48:50 PDT
	889271040	889272224	1184	Mar 11 2008 04:43:46 PDT	Mar 11 2008 05:03:30 PDT
	889275040	889277632	2592	Mar 11 2008 05:50:26 PDT	Mar 11 2008 06:33:38 PDT
	889279872	889311008	31136	Mar 11 2008 07:10:58 PDT	Mar 11 2008 15:49:54 PDT
	889312000	889314080	2080	Mar 11 2008 16:06:26 PDT	Mar 11 2008 16:41:06 PDT
	889318176	889321376	3200	Mar 11 2008 17:49:22 PDT	Mar 11 2008 18:42:42 PDT
	889348544	889349248	704	Mar 12 2008 02:15:30 PDT	Mar 12 2008 02:27:14 PDT
	889365568	889370528	4960	Mar 12 2008 06:59:14 PDT	Mar 12 2008 08:21:54 PDT
	889383616	889386560	2944	Mar 12 2008 12:00:02 PDT	Mar 12 2008 12:49:06 PDT
	889387648	889388000	352	Mar 12 2008 13:07:14 PDT	Mar 12 2008 13:13:06 PDT
	889389984	889392576	2592	Mar 12 2008 13:46:10 PDT	Mar 12 2008 14:29:22 PDT
	889399520	889401952	2432	Mar 12 2008 16:25:06 PDT	Mar 12 2008 17:05:38 PDT
	889407072	889407648	576	Mar 12 2008 18:30:58 PDT	Mar 12 2008 18:40:34 PDT
	889410336	889411008	672	Mar 12 2008 19:25:22 PDT	Mar 12 2008 19:36:34 PDT
	889415808	889417856	2048	Mar 12 2008 20:56:34 PDT	Mar 12 2008 21:30:42 PDT
	889433824	889434784	960	Mar 13 2008 01:56:50 PDT	Mar 13 2008 02:12:50 PDT
	889459520	889511008	51488	Mar 13 2008 09:05:06 PDT	Mar 13 2008 23:23:14 PDT
	889512000	889547456	35456	Mar 13 2008 23:39:46 PDT	Mar 14 2008 09:30:42 PDT
L	874616576	874617152	576	Sep 23 2007 14:02:42 PDT	Sep 23 2007 14:12:18 PDT
	875969856	875985920	16064	Oct 09 2007 05:57:22 PDT	Oct 09 2007 10:25:06 PDT
	876082560	876082592	32	Oct 10 2007 13:15:46 PDT	Oct 10 2007 13:16:18 PDT
	876183776	876187904	4128	Oct 11 2007 17:22:42 PDT	Oct 11 2007 18:31:30 PDT
V	864147600	864151200	3600	May 25 2007 09:59:46 PDT	May 25 2007 10:59:46 PDT
	867798000	867815984	17984	Jul 06 2007 15:59:46 PDT	Jul 06 2007 20:59:30 PDT
	867827040	867862800	35760	Jul 07 2007 00:03:46 PDT	Jul 07 2007 09:59:46 PDT
	870591600	870593888	2288	Aug 07 2007 23:59:46 PDT	Aug 08 2007 00:37:54 PDT
	870613200	870628736	15536	Aug 08 2007 05:59:46 PDT	Aug 08 2007 10:18:42 PDT
	870635808	870636016	208	Aug 08 2007 12:16:34 PDT	Aug 08 2007 12:20:02 PDT
	870640128	870657120	16992	Aug 08 2007 13:28:34 PDT	Aug 08 2007 18:11:46 PDT
	870661488	870661984	496	Aug 08 2007 19:24:34 PDT	Aug 08 2007 19:32:50 PDT
	870663760	870709184	45424	Aug 08 2007 20:02:26 PDT	Aug 09 2007 08:39:30 PDT
	870715856	870764848	48992	Aug 09 2007 10:30:42 PDT	Aug 10 2007 00:07:14 PDT
	870781280	871011216	229936	Aug 10 2007 04:41:06 PDT	Aug 12 2007 20:33:22 PDT
	871022304	871172336	150032	Aug 12 2007 23:38:10 PDT	Aug 14 2007 17:18:42 PDT
	871177984	871247520	69536	Aug 14 2007 18:52:50 PDT	Aug 15 2007 14:11:46 PDT
	871251808	871257360	5552	Aug 15 2007 15:23:14 PDT	Aug 15 2007 16:55:46 PDT
	871273088	871280880	7792	Aug 15 2007 21:17:54 PDT	Aug 15 2007 23:27:46 PDT
	871981200	871981328	128	Aug 24 2007 01:59:46 PDT	Aug 24 2007 02:01:54 PDT
	872003120	872114400	111280	Aug 24 2007 08:05:06 PDT	Aug 25 2007 14:59:46 PDT
	872636400	872655024	18624	Aug 31 2007 15:59:46 PDT	Aug 31 2007 21:10:10 PDT
	872659280	872661488	2208	Aug 31 2007 22:21:06 PDT	Aug 31 2007 22:57:54 PDT
	872665136	872693216	28080	Aug 31 2007 23:58:42 PDT	Sep 01 2007 07:46:42 PDT
	872698320	872731776	33456	Sep 01 2007 09:11:46 PDT	Sep 01 2007 18:29:22 PDT
	872741840	872787264	45424	Sep 01 2007 21:17:06 PDT	Sep 02 2007 09:54:10 PDT
	872788880	872794800	5920	Sep 02 2007 10:21:06 PDT	Sep 02 2007 11:59:46 PDT
	873212384	873269376	56992	Sep 07 2007 07:59:30 PDT	Sep 07 2007 23:49:22 PDT
	873271376	873278064	6688	Sep 08 2007 00:22:42 PDT	Sep 08 2007 02:14:10 PDT
	873284704	873425584	140880	Sep 08 2007 04:04:50 PDT	Sep 09 2007 19:12:50 PDT
	873431856	873617856	186000	Sep 09 2007 20:57:22 PDT	Sep 12 2007 00:37:22 PDT
	873649040	873668464	19424	Sep 12 2007 09:17:06 PDT	Sep 12 2007 14:40:50 PDT
	873671504	873676960	5456	Sep 12 2007 15:31:30 PDT	Sep 12 2007 17:02:26 PDT
	873700304	873715728	15424	Sep 12 2007 23:31:30 PDT	Sep 13 2007 03:48:34 PDT
	873718576	873830032	111456	Sep 13 2007 04:36:02 PDT	Sep 14 2007 11:33:38 PDT
	873831600	874030176	198576	Sep 14 2007 11:59:46 PDT	Sep 16 2007 19:09:22 PDT
	874034256	874094400	60144	Sep 16 2007 20:17:22 PDT	Sep 17 2007 12:59:46 PDT
	875260800	875263152	2352	Oct 01 2007 00:59:46 PDT	Oct 01 2007 01:38:58 PDT
	875264672	875279712	15040	Oct 01 2007 02:04:18 PDT	Oct 01 2007 06:14:58 PDT
	875282608	875283920	1312	Oct 01 2007 07:03:14 PDT	Oct 01 2007 07:25:06 PDT
	875286400	875286704	304	Oct 01 2007 08:06:26 PDT	Oct 01 2007 08:11:30 PDT
	875310352	875321088	10736	Oct 01 2007 14:45:38 PDT	Oct 01 2007 17:44:34 PDT
	875343488	875364768	21280	Oct 01 2007 23:57:54 PDT	Oct 02 2007 05:52:34 PDT
	875368464	875374176	5712	Oct 02 2007 06:54:10 PDT	Oct 02 2007 08:29:22 PDT
	875376032	875430880	54848	Oct 02 2007 09:00:18 PDT	Oct 03 2007 00:14:26 PDT
	875477232	875481312	4080	Oct 03 2007 13:06:58 PDT	Oct 03 2007 14:14:58 PDT
	875518592	875518720	128	Oct 04 2007 00:36:18 PDT	Oct 04 2007 00:38:26 PDT
	875522048	875533440	11392	Oct 04 2007 01:33:54 PDT	Oct 04 2007 04:43:46 PDT
	875535696	875539184	3488	Oct 04 2007 05:21:22 PDT	Oct 04 2007 06:19:30 PDT
	875558832	875562176	3344	Oct 04 2007 11:46:58 PDT	Oct 04 2007 12:42:42 PDT
	876126368	876145840	19472	Oct 11 2007 01:25:54 PDT	Oct 11 2007 06:50:26 PDT
	876163600	876171504	7904	Oct 11 2007 11:46:26 PDT	Oct 11 2007 13:58:10 PDT
	876176048	876176096	48	Oct 11 2007 15:13:54 PDT	Oct 11 2007 15:14:42 PDT
	876179648	876204112	24464	Oct 11 2007 16:13:54 PDT	Oct 11 2007 23:01:38 PDT
	876218256	876219920	1664	Oct 12 2007 02:57:22 PDT	Oct 12 2007 03:25:06 PDT
	876230416	876238384	7968	Oct 12 2007 06:20:02 PDT	Oct 12 2007 08:32:50 PDT
	876251104	876258016	6912	Oct 12 2007 12:04:50 PDT	Oct 12 2007 14:00:02 PDT
	876508656	876513216	4560	Oct 15 2007 11:37:22 PDT	Oct 15 2007 12:53:22 PDT
	876644432	876665376	20944	Oct 17 2007 01:20:18 PDT	Oct 17 2007 07:09:22 PDT
	876691056	876725552	34496	Oct 17 2007 14:17:22 PDT	Oct 17 2007 23:52:18 PDT
	876775760	876775808	48	Oct 18 2007 13:49:06 PDT	Oct 18 2007 13:49:54 PDT
	876778176	876778528	352	Oct 18 2007 14:29:22 PDT	Oct 18 2007 14:35:14 PDT
	876811680	876812880	1200	Oct 18 2007 23:47:46 PDT	Oct 19 2007 00:07:46 PDT
	876829952	876830288	336	Oct 19 2007 04:52:18 PDT	Oct 19 2007 04:57:54 PDT
	876848256	876852048	3792	Oct 19 2007 09:57:22 PDT	Oct 19 2007 11:00:34 PDT
	876907136	876908432	1296	Oct 20 2007 02:18:42 PDT	Oct 20 2007 02:40:18 PDT
	876911248	876912752	1504	Oct 20 2007 03:27:14 PDT	Oct 20 2007 03:52:18 PDT
	877094496	877095488	992	Oct 22 2007 06:21:22 PDT	Oct 22 2007 06:37:54 PDT
	877098416	877107712	9296	Oct 22 2007 07:26:42 PDT	Oct 22 2007 10:01:38 PDT
	877112112	877150816	38704	Oct 22 2007 11:14:58 PDT	Oct 22 2007 22:00:02 PDT
	877157088	877172864	15776	Oct 22 2007 23:44:34 PDT	Oct 23 2007 04:07:30 PDT
	877179216	877187360	8144	Oct 23 2007 05:53:22 PDT	Oct 23 2007 08:09:06 PDT
	877190640	877204800	14160	Oct 23 2007 09:03:46 PDT	Oct 23 2007 12:59:46 PDT