API Reference¶
This documents the current version of the Pypeline DB API.
pypeline.DB¶
- class pypeline.DB(database_path, **kwargs)¶
The pypeline LevelDB database. This class contains collections and provides some system-level organization.
All arguments beyond the database path are passed directly to the underlying plyvel DB constructor. More details can be found at https://plyvel.readthedocs.org/en/latest/api.html#DB
Arguments: database_path – The path to the folder for database storage
- collection(collection_name, reset_collection=False, create_if_missing=True, error_if_exists=False)¶
Returns the collection stored at collection_name, or creates it if it doesn’t exist.
Arguments:collection_name – the name of the collection to returnKeyword arguments:reset_collection – when True any existant data in the collection is deleted before it is returnedcreate_if_missing – when False a ValueError is raised if the collection doesn’t existerror_if_exists – When True a ValueError is raised if the collection already exists
- collections()¶
Returns a list of keys of collections contained in the database.
- copy_collection(old_collection, new_collection, start=None, end=None, **kwargs)¶
Copies all instances in the old_collection into the new_collection
Arguments:old_collection – The string name of the old collectionnew_collection – The string name of the new collectionKeyword arguments:start – (Optional) The index to begin copying from old_collectionend – (Optional) The index to end copying from old_collectioncreate_if_missing – when False a ValueError is raised if the new collection doesn’t exist (default: True)error_if_exists – When True a ValueError is raised if the collection already exists (default: False)
- delete(collection_name)¶
Deletes a collection.
Arguments:collection_name – the name of the collection to delete
- close()¶
Closes the database.
- open()¶
Opens the database.
pypeline.Collection¶
- class pypeline.Collection(database, items_set, name)¶
A collection of records stored in a database
This class should never be instantiated directly. Use the DB.collection() method instead
- append(record)¶
Appends a single record.
Arguments:record – Any JSON-serializable python object (dicts, lists, ints, strings, etc.)
- refresh()¶
Reloads the collection from the database.
- delete(index)¶
Deletes an item from the collection.
Arguments:index – Index of the item to be deleted.
- delete_all()¶
Deletes all items in the collection
- append_all(iterable)¶
Appends every item in the iterable to the collection
- map(function, new_collection, **kwargs)¶
Maps a collection to a new collection with a provided function.
Arguments:function – The function used for mapping.new_collection – The name of the collection to insert the new values into. Any existing values will be deleted. If None, values are mapped to the same collection.Keyword arguments:create_if_missing – when False a ValueError is raised if the new collection doesn’t existerror_if_exists – When True a ValueError is raised if the new collection already exists
- filter(function, new_collection, **kwargs)¶
Filters a collection into a new collection with a given function.
Arguments:function – The function used for filtering.new_collection – The name of the collection to insert the new values into. Any existing values will be deleted. If None, values are filtered in the same collection.Keyword arguments:create_if_missing – when False a ValueError is raised if the new collection doesn’t existerror_if_exists – When True a ValueError is raised if the new collection already exists
- reduce(function, new_collection, initializer=None, **kwargs)¶
Reduces a collection into a new collection with a given function.
Arguments:function – The function used for reducing.new_collection – The name of the collection to insert the new value into. Any existing values will be deleted. If None, the current collection is replaced with the reduction output.Keyword arguments:create_if_missing – when False a ValueError is raised if the new collection doesn’t existerror_if_exists – When True a ValueError is raised if the new collection already exists
- random_subset(number, new_collection, **kwargs)¶
Produces a random subset of a given collection and inserts it into a new collection.
Arguments:new_collection – The name of the collection to insert the new values into. Any existing values will be deleted. If None, the subset is stored to the current collection.Keyword arguments:create_if_missing – when False a ValueError is raised if the new collection doesn’t existerror_if_exists – When True a ValueError is raised if the new collection already exists
- iterator(start=None, end=None)¶
Returns a collection iterator