Core Classes

All protocol implementations are subclasses (directly or indirectly) of Duct. This base class manages the basic life-cycle, connection management and protocol registration. When a subclass of Duct is loaded into memory, and has at least one protocol name in the PROTOCOLS attribute, then Duct registers that class into its subclass registry. This class can then be conveniently accessed by: Duct.for_protocol(‘<protocol_name>’). This empowers the accompanying registry tooling bundled with omniduct, as documented in Registry Management.

Protocol implementations may also (directly or indirectly) be subclasses of MagicsProvider, which provides a common API to registry IPython magics into the user’s session. If implemented, the accompanying registry tooling can automatically register these magics, as documented in Registry Management.

Duct

class omniduct.duct.Duct(protocol=None, name=None, registry=None, remote=None, host=None, port=None, username=None, password=None, cache=None, cache_namespace=None)

Bases: object

The abstract base class for all protocol implementations.

This class defines the basic lifecycle of service connections, along with some magic that provides automatic registration of Duct protocol implementations. All connections made by Duct instances are lazy, meaning that instantiation is “free”, and no protocol connections are made until required by subsequent interactions (i.e. when the value of any attribute in the list of connection_fields is accessed). All Ducts will automatically connnect and disconnect as required, and so manual intervention is not typically required to maintain connections.

Attributes:

• protocol (str) – The name of the protocol for which this instance was created (especially useful if a Duct subclass supports multiple protocols).

• name (str) – The name given to this Duct instance (defaults to class name).

• host (str) – The host name providing the service (will be ‘127.0.0.1’, if service is port forwarded from remote; use ._host to see remote host).

• port (int) – The port number of the service (will be the port-forwarded local port, if relevant; for remote port use ._port).

• username (str, bool) – The username to use for the service.

• password (str, bool) – The password to use for the service.

• registry (None, omniduct.registry.DuctRegistry) – A reference to a DuctRegistry instance for runtime lookup of other services.

• remote (None, omniduct.remotes.base.RemoteClient) – A reference to a RemoteClient instance to manage connections to remote services.

• cache (None, omniduct.caches.base.Cache) – A reference to a Cache instance to add support for caching, if applicable.

• connection_fields (tuple<str>, list<str>) – A list of instance attributes to monitor for changes, whereupon the Duct instance should automatically disconnect. By default, the following attributes are monitored: ‘host’, ‘port’, ‘remote’, ‘username’, and ‘password’.

• prepared_fields (tuple<str>, list<str>) – A list of instance attributes to be populated (if their values are callable) when the instance first connects to a service. Refer to Duct.prepare and Duct._prepare for more details. By default, the following attributes are prepared: ‘_host’, ‘_port’, ‘_username’, and ‘_password’.

• Additional attributes including `host`, `port`, `username` and `password` are

• documented inline.

• Class Attributes –

  AUTO_LOGGING_SCOPE (bool): Whether this class should be used by omniduct

  logging code as a “scope”. Should be overridden by subclasses as appropriate.

  DUCT_TYPE (Duct.Type): The type of Duct service that is provided by

  this Duct instance. Should be overridden by subclasses as appropriate.

  PROTOCOLS (list<str>): The name(s) of any protocols that should be

  associated with this class. Should be overridden by subclasses as appropriate.

class Type

Bases: enum.Enum

The Duct.Type enum specifies all of the permissible values of Duct.DUCT_TYPE. Also determines the order in which ducts are loaded by DuctRegistry.

__init__(protocol=None, name=None, registry=None, remote=None, host=None, port=None, username=None, password=None, cache=None, cache_namespace=None)

protocol (str, None): Name of protocol (used by Duct registries to inform

Duct instances of how they were instantiated).

name (str, None): The name to used by the Duct instance (defaults to

class name if not specified).

registry (DuctRegistry, None): The registry to use to lookup remote

and/or cache instance specified by name.

remote (str, RemoteClient): The remote by which the ducted service

should be contacted.

host (str): The hostname of the service to be used by this client. port (int): The port of the service to be used by this client. username (str, bool, None): The username to authenticate with if necessary.

If True, then users will be prompted at runtime for credentials.

password (str, bool, None): The password to authenticate with if necessary.

If True, then users will be prompted at runtime for credentials.

cache(Cache, None): The cache client to be attached to this instance.

Cache will only used by specific methods as configured by the client.

cache_namespace(str, None): The namespace to use by default when writing

to the cache.

classmethod for_protocol(protocol)

Retrieve a Duct subclass for a given protocol.

Parameters:protocol (str) – The protocol of interest. Returns:

The appropriate class for the provided,

partially constructed with the protocol keyword argument set appropriately.

Return type:functools.partial object Raises:DuctProtocolUnknown – If no class has been defined that offers the named protocol.

prepare()

Prepare a Duct subclass for use (if not already prepared).

This method is called before the value of any of the fields referenced in self.connection_fields are retrieved. The fields include, by default: ‘host’, ‘port’, ‘remote’, ‘cache’, ‘username’, and ‘password’. Subclasses may add or subtract from these special fields.

When called, it first checks whether the instance has already been prepared, and if not calls _prepare and then records that the instance has been successfully prepared.

This method may be overridden by subclasses, but provides the following default behaviour:

• Ensures self.registry, self.remote and self.cache values are instances of the right types.
• It replaces string values of self.remote and self.cache with remotes and caches looked up using self.registry.lookup.
• It looks through each of the fields nominated in self.prepared_fields and, if the corresponding value is callable, sets the value of that field to result of calling that value with a reference to self. By default, prepared_fields contains ‘_host’, ‘_port’, ‘_username’, and ‘_password’.
• Ensures value of self.port is an integer (or None).

_prepare()

This method may be overridden by subclasses, but provides the following default behaviour:

• Ensures self.registry, self.remote and self.cache values are instances of the right types.
• It replaces string values of self.remote and self.cache with remotes and caches looked up using self.registry.lookup.
• It looks through each of the fields nominated in self.prepared_fields and, if the corresponding value is callable, sets the value of that field to result of calling that value with a reference to self. By default, prepared_fields contains ‘_host’, ‘_port’, ‘_username’, and ‘_password’.
• Ensures value of self.port is an integer (or None).

reset()

Reset this Duct instance to its pre-preparation state.

This method disconnects from the service, resets any temporary authentication and restores the values of the attributes listed in prepared_fields to their values as of when Duct.prepare was called.

Returns:A reference to this object. Return type:Duct instance

host

The host name providing the service, or ‘127.0.0.1’ if self.remote is not None, whereupon the service will be port-forwarded locally. You can view the remote hostname using duct._host, and change the remote host at runtime using: duct.host = ‘<host>’.

Type:str

port

The local port for the service. If self.remote is not None, the port will be port-forwarded from the remote host. To see the port used on the remote host refer to duct._port. You can change the remote port at runtime using: duct.port = <port>.

Type:int

username

Some services require authentication in order to connect to the service, in which case the appropriate username can be specified. If not specified at instantiation, your local login name will be used. If True was provided, you will be prompted to type your username at runtime as necessary. If False was provided, then None will be returned. You can specify a different username at runtime using: duct.username = ‘<username>’.

Type:str

password

Some services require authentication in order to connect to the service, in which case the appropriate password can be specified. If True was provided at instantiation, you will be prompted to type your password at runtime when necessary. If False was provided, then None will be returned. You can specify a different password at runtime using: duct.password = ‘<password>’.

Type:str

connect()

Connect to the service backing this client.

It is not normally necessary for a user to manually call this function, since when a connection is required, it is automatically created.

Returns:A reference to the current object. Return type:Duct instance

is_connected()

Check whether this Duct instances is currently connected.

This method checks to see whether a Duct instance is currently connected. This is performed by verifying that the remote host and port are still accessible, and then by calling Duct._is_connected, which should be implemented by subclasses.

Returns:Whether this Duct instance is currently connected. Return type:bool

disconnect()

Disconnect this client from backing service.

This method is automatically called during reconnections and/or at Python interpreter shutdown. It first calls Duct._disconnect (which should be implemented by subclasses) and then notifies the RemoteClient subclass, if present, to stop port-forwarding the remote service.

Returns:A reference to this object. Return type:Duct instance

reconnect()

Disconnects, and then reconnects, this client.

Note: This is equivalent to duct.disconnect().connect().

Returns:A reference to this object. Return type:Duct instance

MagicsProvider

class omniduct.utils.magics.MagicsProvider

Bases: object