Welcome to cachings’s documentation!¶

caching (Caching Module)¶

Author:

Dirk Alders <sudo-dirk@mount-mockery.de>

Description:

This Module supports functions and classes for caching e.g. properties of other instances.

Submodules:

caching.property_cache_json
caching.property_cache_pickle

Unittest:

See also the unittest documentation.

class caching.property_cache_json(source_instance, cache_filename, load_all_on_init=False, callback_on_data_storage=None)¶

Class to cache properties, which take longer on initialising than reading a file in json format. See also parent property_cache_pickle

Parameters:	source_instance (instance) – The source instance holding the data cache_filename (str) – File name, where the properties are stored as cache load_all_on_init – Optionally init behaviour control parameter. True will load all available properties from source on init, False not.

Warning

This class uses json. You should only use keys of type string!
Unicode types are transfered to strings

Note

source_instance needs to have at least the following methods: uid(), keys(), data_version(), get()

uid(): returns the unique id of the source.
keys(): returns a list of all available keys.
data_version(): returns a version number of the current data (it should be increased, if the get method of the source instance returns improved values or the data structure had been changed).
get(key, default): returns the property for a key. If key does not exists, default will be returned.

Reasons for updating the complete data set:

UID of source_instance has changed (in comparison to the cached value).
data_version is increased

Example:

#!/usr/bin/env python
# -*- coding: UTF-8 -*-

import sys
import time
sys.path.append('../..')

import caching
import report


report.stdoutLoggingConfigure(log_name_lvl=[('root', 'DEBUG'), ])


class test_slow_data(object):
    _ONE = '1'
    _TWO = '2'
    _THREE = '_property_cache_data_version_'
    _FOUR = '_property_cache_uid_'
    _FIVE = '__property_cache_uid_'
    KEYS = [_ONE, _TWO, _THREE, _FOUR, _FIVE]
    VERS = 0.1

    def data_version(self):
        return self.VERS

    def one(self):
        return self.get(self._ONE)

    def two(self):
        return self.get(self._TWO)

    def three(self):
        return self.get(self._THREE)

    def four(self):
        return self.get(self._FOUR)

    def five(self):
        return self.get(self._FIVE)

    def get(self, key, default=None):
        def print_n_sleep(k):
            sys.stdout.write('slow get executed for %s\n' % k)
            time.sleep(3)
        if key == self._ONE:
            print_n_sleep(key)
            return 'one'
        if key == self._TWO:
            print_n_sleep(key)
            return 'two'
        if key == self._THREE:
            print_n_sleep(key)
            return 'three'
        if key == self._FOUR:
            print_n_sleep(key)
            return 'four'
        if key == self._FIVE:
            print_n_sleep(key)
            return 'five'
        return default

    def keys(self):
        return self.KEYS

    def uid(self):
        return None


class tsd_cache_json(test_slow_data):
    def __init__(self, *args, **kwargs):
        test_slow_data.__init__(self, *args, **kwargs)
        self._cached_data = caching.property_cache_json(test_slow_data(*args, **kwargs), 'cache.json', load_all_on_init=False)

    def two(self):
        return test_slow_data.get(self, self._TWO)

    def get(self, key, default=None):
        return self._cached_data.get(key, default)


data = tsd_cache_json()
print('Testing property_cache (json):\\n--------------------------------')
print(data.one())
print(data.two())
print(data.three())
print(data.four())
print(data.five())

Will result on the first execution to the following output (with a long execution time):

Testing property_cache (json):\n--------------------------------
2021-01-14 00:50:45,282: root.caching - DEBUG - JsonCache: Cache file does not exists (yet).
2021-01-14 00:50:45,283: root.caching - INFO - JsonCache: cache-file stored (cache.json)
slow get executed for 1
2021-01-14 00:50:48,285: root.caching - DEBUG - JsonCache: Loading property for '1' from source instance ('one')
2021-01-14 00:50:48,286: root.caching - INFO - JsonCache: cache-file stored (cache.json)
one
slow get executed for 2
two
slow get executed for _property_cache_data_version_
2021-01-14 00:50:54,292: root.caching - DEBUG - JsonCache: Loading property for '_property_cache_data_version_' from source instance ('three')
2021-01-14 00:50:54,293: root.caching - INFO - JsonCache: cache-file stored (cache.json)
three
slow get executed for _property_cache_uid_
2021-01-14 00:50:57,297: root.caching - DEBUG - JsonCache: Loading property for '_property_cache_uid_' from source instance ('four')
2021-01-14 00:50:57,297: root.caching - INFO - JsonCache: cache-file stored (cache.json)
four
slow get executed for __property_cache_uid_
2021-01-14 00:51:00,301: root.caching - DEBUG - JsonCache: Loading property for '__property_cache_uid_' from source instance ('five')
2021-01-14 00:51:00,301: root.caching - INFO - JsonCache: cache-file stored (cache.json)
five

With every following execution (slow for getting “two” which is not cached - see implementation):

Testing property_cache (json):\n--------------------------------
2021-01-14 00:51:00,380: root.caching - INFO - JsonCache: Loading properties from cache (cache.json)
2021-01-14 00:51:00,380: root.caching - DEBUG - JsonCache: Providing property for '1' from cache ('one')
one
slow get executed for 2
two
2021-01-14 00:51:03,381: root.caching - DEBUG - JsonCache: Providing property for '_property_cache_data_version_' from cache ('three')
three
2021-01-14 00:51:03,381: root.caching - DEBUG - JsonCache: Providing property for '_property_cache_uid_' from cache ('four')
four
2021-01-14 00:51:03,381: root.caching - DEBUG - JsonCache: Providing property for '__property_cache_uid_' from cache ('five')
five

class caching.property_cache_pickle(source_instance, cache_filename, load_all_on_init=False, callback_on_data_storage=None)¶

Class to cache properties, which take longer on initialising than reading a file in pickle format.

Parameters:	source_instance (instance) – The source instance holding the data cache_filename (str) – File name, where the properties are stored as cache load_all_on_init – Optionally init behaviour control parameter. True will load all available properties from source on init, False not.

Note

source_instance needs to have at least the following methods: uid(), keys(), data_version(), get()

uid(): returns the unique id of the source.
keys(): returns a list of all available keys.
data_version(): returns a version number of the current data (it should be increased, if the get method of the source instance returns improved values or the data structure had been changed).
get(key, default): returns the property for a key. If key does not exists, default will be returned.

Reasons for updating the complete data set:

UID of source_instance has changed (in comparison to the cached value).
data_version is increased

Example:

#!/usr/bin/env python
# -*- coding: UTF-8 -*-

import sys
import time
sys.path.append('../..')

import caching
import report


report.stdoutLoggingConfigure(log_name_lvl=[('root', 'DEBUG'), ])


class test_slow_data(object):
    _ONE = '1'
    _TWO = '2'
    _THREE = '_property_cache_data_version_'
    _FOUR = '_property_cache_uid_'
    _FIVE = '__property_cache_uid_'
    KEYS = [_ONE, _TWO, _THREE, _FOUR, _FIVE]
    VERS = 0.1

    def data_version(self):
        return self.VERS

    def one(self):
        return self.get(self._ONE)

    def two(self):
        return self.get(self._TWO)

    def three(self):
        return self.get(self._THREE)

    def four(self):
        return self.get(self._FOUR)

    def five(self):
        return self.get(self._FIVE)

    def get(self, key, default=None):
        def print_n_sleep(k):
            sys.stdout.write('slow get executed for %s\n' % k)
            time.sleep(3)
        if key == self._ONE:
            print_n_sleep(key)
            return 'one'
        if key == self._TWO:
            print_n_sleep(key)
            return 'two'
        if key == self._THREE:
            print_n_sleep(key)
            return 'three'
        if key == self._FOUR:
            print_n_sleep(key)
            return 'four'
        if key == self._FIVE:
            print_n_sleep(key)
            return 'five'
        return default

    def keys(self):
        return self.KEYS

    def uid(self):
        return None


class tsd_cache_pickle(test_slow_data):
    def __init__(self, *args, **kwargs):
        test_slow_data.__init__(self, *args, **kwargs)
        self._cached_data = caching.property_cache_pickle(test_slow_data(*args, **kwargs), 'cache.pickle', load_all_on_init=False)

    def two(self):
        return test_slow_data.get(self, self._TWO)

    def get(self, key, default=None):
        return self._cached_data.get(key, default)


data = tsd_cache_pickle()
print('Testing property_cache (pickle):\\n--------------------------------')
print(data.one())
print(data.two())
print(data.three())
print(data.four())
print(data.five())

Will result on the first execution to the following output (with a long execution time):

Testing property_cache (pickle):\n--------------------------------
2021-01-14 00:50:27,103: root.caching - DEBUG - PickCache: Cache file does not exists (yet).
2021-01-14 00:50:27,103: root.caching - INFO - PickCache: cache-file stored (cache.pickle)
slow get executed for 1
2021-01-14 00:50:30,107: root.caching - DEBUG - PickCache: Loading property for '1' from source instance ('one')
2021-01-14 00:50:30,107: root.caching - INFO - PickCache: cache-file stored (cache.pickle)
one
slow get executed for 2
two
slow get executed for _property_cache_data_version_
2021-01-14 00:50:36,112: root.caching - DEBUG - PickCache: Loading property for '_property_cache_data_version_' from source instance ('three')
2021-01-14 00:50:36,113: root.caching - INFO - PickCache: cache-file stored (cache.pickle)
three
slow get executed for _property_cache_uid_
2021-01-14 00:50:39,116: root.caching - DEBUG - PickCache: Loading property for '_property_cache_uid_' from source instance ('four')
2021-01-14 00:50:39,117: root.caching - INFO - PickCache: cache-file stored (cache.pickle)
four
slow get executed for __property_cache_uid_
2021-01-14 00:50:42,120: root.caching - DEBUG - PickCache: Loading property for '__property_cache_uid_' from source instance ('five')
2021-01-14 00:50:42,121: root.caching - INFO - PickCache: cache-file stored (cache.pickle)
five

With every following execution (slow for getting “two” which is not cached - see implementation):

Testing property_cache (pickle):\n--------------------------------
2021-01-14 00:50:42,200: root.caching - INFO - PickCache: Loading properties from cache (cache.pickle)
2021-01-14 00:50:42,200: root.caching - DEBUG - PickCache: Providing property for '1' from cache ('one')
one
slow get executed for 2
two
2021-01-14 00:50:45,201: root.caching - DEBUG - PickCache: Providing property for '_property_cache_data_version_' from cache ('three')
three
2021-01-14 00:50:45,201: root.caching - DEBUG - PickCache: Providing property for '_property_cache_uid_' from cache ('four')
four
2021-01-14 00:50:45,201: root.caching - DEBUG - PickCache: Providing property for '__property_cache_uid_' from cache ('five')
five

get(key, default=None)¶

Method to get the cached property. If key does not exists in cache, the property will be loaded from source_instance and stored in cache (file).

Parameters:	key – key for value to get. default – value to be returned, if key does not exists.
Returns:	value for a given key or default value.

keys()¶: Method to get the available keys (from source_instance).

Welcome to cachings’s documentation!¶

caching (Caching Module)¶

Indices and tables¶