ahvn.klstore package

class ahvn.klstore.BaseKLStore(name=None, condition=None, *args, **kwargs)

Bases: ToolRegistry, ABC

An abstract base class for KLStore implementations.

This class provides a protocol for storing and managing Knowledge items, each identified by a unique id attribute, typically using a BaseUKF instance as the storage unit. The class defines the required interface for storage, retrieval, insertion, batch operations, and removal of KL items, as well as iteration and clearing of the store.

Subclasses must implement the abstract methods for storage and retrieval by id, and may override batch and existence-checking methods for performance optimization.

Abstract Methods:

_get(key, default): Retrieve a KL (BaseUKF) by its key (id). Should return default (Ellipsis by default) if not found. _upsert(kl): Insert or update a KL in the store. _remove(key): Remove a KL from the store by its key (id). _itervalues(): Return an iterator over all KLs in the store. _clear(): Clear all KLs from the store.

Optional Override Methods:

_has(key): Determines if a KL exists for the given key. __len__(): Returns the number of entries in the store.

Notes

• The default implementations of __len__, _has, and batch operations are not optimized and may be slow for large stores. Subclasses are encouraged to override these for efficiency.

Parameters:

• name (str)

• condition (Callable | None)

__init__(name=None, condition=None, *args, **kwargs)

Initialization.

Parameters:

• name (Optional[str]) – Name of the KLStore instance. If None, defaults to “default”.

• condition (Optional[Callable]) – Optional upsert/insert condition to apply to the KLStore. KLs that do not satisfy the condition will be ignored. If None, all KLs are accepted.

• *args – Additional positional arguments.

• **kwargs – Additional keyword arguments.

name: str

__contains__(key)

Checks if a KL exists for the given key.

Parameters:

key (Union[int, str, BaseUKF]) – The key or BaseUKF instance to check.

Returns:

True if the KL exists, False otherwise.

Return type:

bool

exists(key)

Checks if a KL exists for the given key.

Parameters:

key (Union[int, str, BaseUKF]) – The key or BaseUKF instance to check.

Returns:

True if the KL exists, False otherwise.

Return type:

bool

__getitem__(key)

Retrieve a KL by its key.

Parameters:

key (Union[int, str, BaseUKF]) – The key or BaseUKF instance to retrieve.

Returns:

The retrieved KL instance if found. Otherwise Ellipsis.

Return type:

BaseUKF

get(key, default=...)

Retrieves a KL by its key.

Parameters:

• key (Union[int, str, BaseUKF]) – The key or BaseUKF instance to retrieve.

• default (Any) – The default value to return if the KL is not found.

Returns:

The retrieved KL instance if found. Otherwise default.

Return type:

BaseUKF

batch_get(keys, default=..., progress=None)

Retrieves multiple KLs by their keys. The default batch get is not optimized nor parallelized. It is recommended to override _batch_get for better performance.

Parameters:

• keys (Iterable[Union[int, str, BaseUKF]]) – The keys or BaseUKF instances to retrieve.

• default (Any) – The default value to return if a KL is not found.

• progress (Type[Progress])

Returns:

A list of retrieved KL instances. Missing KLs are replaced with default.

Return type:

list

upsert(kl, **kwargs)

Upsert a KL.

Parameters:

• kl (BaseUKF) – The KL to upsert.

• **kwargs – Additional keyword arguments.

insert(kl, **kwargs)

Insert a KL.

Parameters:

• kl (BaseUKF) – The KL to insert.

• kwargs – Additional keyword arguments.

batch_upsert(kls, progress=None, **kwargs)

Upsert multiple KLs. The default batch upsert is not optimized nor parallelized. It is recommended to override this method for better performance.

Parameters:

• kls (Iterable[BaseUKF]) – The KLs to upsert.

• progress (Type[Progress]) – Progress class for reporting. None for silent, TqdmProgress for terminal.

• **kwargs – Additional keyword arguments.

batch_insert(kls, progress=None, **kwargs)

Insert multiple KLs. The default batch insert first checks for existing keys and then batch upserts. When overriding batch_upsert, batch insert is automatically optimized. Nevertheless, the existence check uses _has, which is by default not optimized. It is recommended to override batch_insert or _has for better performance.

Parameters:

• kls (Iterable[BaseUKF]) – The KLs to insert.

• progress (Type[Progress]) – Progress class for reporting. None for silent, TqdmProgress for terminal.

• **kwargs – Additional keyword arguments.

__delitem__(key, **kwargs)

Removes a KL from the store.

Parameters:

• key (Union[int, str, BaseUKF]) – The key or BaseUKF instance to remove.

• **kwargs – Additional keyword arguments.

remove(key, conditioned=True, **kwargs)

Removes a KL from the store.

Parameters:

• key (Union[int, str, BaseUKF]) – The key or BaseUKF instance to remove.

• conditioned (bool) – Remove only if the KL satisfies the store’s condition. Default is True. Notice that conditioned removal only applies when passing a BaseUKF instance as key.

• **kwargs – Additional keyword arguments.

batch_remove(kls, conditioned=True, progress=None, **kwargs)

Removes multiple KLs from the store. The default batch remove is not optimized nor parallelized. It is recommended to override this method for better performance.

Parameters:

• kls (Iterable[Union[int, str, BaseUKF]]) – The keys or BaseUKF instances to remove.

• conditioned (bool) – Remove only if the KLs satisfy the store’s condition. Default is True. Notice that conditioned removal only applies when passing BaseUKF instances in kls.

• progress (Type[Progress]) – Progress class for reporting. None for silent, TqdmProgress for terminal.

• **kwargs – Additional keyword arguments.

__iter__()

Iterates over the stored KLs.

Yields:

BaseUKF – The stored KLs in the KLStore.

Return type:

Generator[BaseUKF, None, None]

batch_iter(batch_size=None, **kwargs)

Iterates over the stored KLs in batches.

Parameters:

• batch_size (Optional[int]) – The size of each batch. If None, use the default batch size from configuration (512). If <= 0, yields all KLs in a single batch.

• **kwargs – Additional keyword arguments.

clear()

Clears the store.

close()

Closes the store, if applicable.

flush()

Flushes the store, if applicable.

Submodules

• ahvn.klstore.base module
  • BaseKLStore
    • BaseKLStore.__init__()
    • BaseKLStore.name
    • BaseKLStore.__contains__()
    • BaseKLStore.exists()
    • BaseKLStore.__getitem__()
    • BaseKLStore.get()
    • BaseKLStore.batch_get()
    • BaseKLStore.upsert()
    • BaseKLStore.insert()
    • BaseKLStore.batch_upsert()
    • BaseKLStore.batch_insert()
    • BaseKLStore.__delitem__()
    • BaseKLStore.remove()
    • BaseKLStore.batch_remove()
    • BaseKLStore.__iter__()
    • BaseKLStore.batch_iter()
    • BaseKLStore.clear()
    • BaseKLStore.close()
    • BaseKLStore.flush()
• ahvn.klstore.cache_store module
  • CacheKLStore
    • CacheKLStore.__init__()
    • CacheKLStore.close()
    • CacheKLStore.flush()
• ahvn.klstore.cascade_store module
  • CascadeKLStore
    • CascadeKLStore.__init__()
    • CascadeKLStore.__len__()
    • CascadeKLStore.close()
    • CascadeKLStore.flush()
• ahvn.klstore.db_store module
  • DatabaseKLStore
    • DatabaseKLStore.__init__()
    • DatabaseKLStore.close()
• ahvn.klstore.mdb_store module
  • MongoKLStore
    • MongoKLStore.__init__()
    • MongoKLStore.__len__()
    • MongoKLStore.close()
    • MongoKLStore.flush()
• ahvn.klstore.vdb_store module
  • VectorKLStore
    • VectorKLStore.__init__()
    • VectorKLStore.close()