pywws.sqlite3data

Store weather data in an SQLite3 database file.

Introduction

This module is an alternative to the pywws file based storage system. It stores data using the mature SQLite3 database system, but with underlying queries wrapped so the user need not know about them.

It should also be possible for this module to form the basis of a full client-server based SQL module using, for example, MySQL etc.

The Python builtin sqlite3 module is used which has a threadsafety of 1, therefore this module creates a connection with every Store (sub)class instance. This however brings concurrancy issues and so this module makes use of the underlying sqlite3’s Write-Ahead-Loging and Shared Cache modes to relieve this. These rely on up to date sqlite3 libraries and may not work on older networked drives which do not support the right locking semantics required by sqlite3.

The external API is as per the original pywws file store, but with some enhancements so as to behave more like a mapping container (dict).

From a “user” point of view, the data is accessed as a cross between a list and a dictionary. Each data record is indexed by a datetime.datetime object (dictionary behaviour), but records are stored in order and can be accessed as slices (list behaviour).

For example, to access the hourly data for Christmas day 2009, one might do the following:

from datetime import datetime
import pywws.filedata
hourly = pywws.filedata.HourlyStore("weather_data")
for data in hourly[datetime(2009, 12, 25):datetime(2009, 12, 26)]:
  print(data["idx"], data["temp_out"])

Some more examples of data access:

# get value nearest 9:30 on Christmas day 2008
data[data.nearest(datetime(2008, 12, 25, 9, 30))]
# get entire array, equivalent to data[:]
data[datetime.min:datetime.max]
# get last 12 hours worth of data
data[datetime.utcnow() - timedelta(hours=12):]

Note that the datetime.datetime index is in UTC. You may need to apply an offset to convert to local time.

The module provides five classes to store different data. RawStore takes “raw” data from the weather station; CalibStore, HourlyStore, DailyStore and MonthlyStore store processed data (see pywws.process). All are derived from the same CoreStore class, they only differ in the keys and types of data stored in each record.

Detailed API

Classes

`CalibStore`(dir_name)	Stores "calibrated" weather station data.
`CoreStore`(dir_name)	Provides a dictionary/list like interface to an underlying SQLite3 database
`DailyStore`(dir_name)	Stores daily summary weather station data.
`HourlyStore`(dir_name)	Stores hourly summary weather station data.
`MonthlyStore`(dir_name)	Stores monthly summary weather station data.
`RawStore`(dir_name)	Stores raw weather station data.

Comments or questions? Please subscribe to the pywws mailing list http://groups.google.com/group/pywws and let us know.