Remotes

All remote clients are expected to be subclasses of RemoteClient, and so will share a common API. Protocol implementations are also free to add extra methods, which are documented in the “Subclass Reference” section below.

Common API

class omniduct.remotes.base.RemoteClient(smartcards=None, **kwargs)[source]

Bases: omniduct.filesystems.base.FileSystemClient

An abstract class providing the common API for all remote clients.

Attributes:smartcard (dict) – Mapping of smartcard names to system libraries compatible with ssh-add -s ‘<system library>’ ….
Attributes:smartcard (dict) – Mapping of smartcard names to system libraries compatible with ssh-add -s ‘<system library>’ ….
Attributes inherited from Duct:
protocol (str): The name of the protocol for which this instance was
created (especially useful if a Duct subclass supports multiple protocols).
name (str): The name given to this Duct instance (defaults to class
name).
host (str): The host name providing the service (will be ‘127.0.0.1’, if
service is port forwarded from remote; use ._host to see remote host).
port (int): The port number of the service (will be the port-forwarded
local port, if relevant; for remote port use ._port).

username (str, bool): The username to use for the service. password (str, bool): The password to use for the service. registry (None, omniduct.registry.DuctRegistry): A reference to a

DuctRegistry instance for runtime lookup of other services.
remote (None, omniduct.remotes.base.RemoteClient): A reference to a
RemoteClient instance to manage connections to remote services.
cache (None, omniduct.caches.base.Cache): A reference to a Cache
instance to add support for caching, if applicable.
connection_fields (tuple<str>, list<str>): A list of instance attributes
to monitor for changes, whereupon the Duct instance should automatically disconnect. By default, the following attributes are monitored: ‘host’, ‘port’, ‘remote’, ‘username’, and ‘password’.
prepared_fields (tuple<str>, list<str>): A list of instance attributes to
be populated (if their values are callable) when the instance first connects to a service. Refer to Duct.prepare and Duct._prepare for more details. By default, the following attributes are prepared: ‘_host’, ‘_port’, ‘_username’, and ‘_password’.

Additional attributes including host, port, username and password are documented inline.

Class Attributes:
AUTO_LOGGING_SCOPE (bool): Whether this class should be used by omniduct
logging code as a “scope”. Should be overridden by subclasses as appropriate.
DUCT_TYPE (Duct.Type): The type of Duct service that is provided by
this Duct instance. Should be overridden by subclasses as appropriate.
PROTOCOLS (list<str>): The name(s) of any protocols that should be
associated with this class. Should be overridden by subclasses as appropriate.
__init__(smartcards=None, **kwargs)[source]
Parameters:smartcards (dict) – Mapping of smartcard names to system libraries compatible with ssh-add -s ‘<system library>’ ….
connect()[source]

Connect to the service backing this client.

It is not normally necessary for a user to manually call this function, since when a connection is required, it is automatically created.

Returns:A reference to the current object.
Return type:Duct instance
RemoteClient Quirks:

Connect to the remote server.

It is not normally necessary for a user to manually call this function, since when a connection is required, it is automatically created.

Compared to base Duct.connect, this method will automatically catch the first DuctAuthenticationError error triggered by Duct.connect, and (if smartcards have been configured) attempt to re-initialise the smartcards before trying once more.

Returns:
Duct instance: A reference to the current object.
prepare_smartcards()[source]

Prepare smartcards for use in authentication.

This method checks attempts to ensure that the each of the nominated smartcards is available and prepared for use. This may result in interactive requests for pin confirmation, depending on the card.

Returns:
Returns True if at least one smartcard was activated, and
False otherwise.
Return type:bool
execute(cmd, **kwargs)[source]

Execute a command on the remote server.

Parameters:
  • cmd (str) – The command to run on the remote associated with this instance.
  • **kwargs (dict) – Additional keyword arguments to be passed on to ._execute.
Returns:

The result of the execution.
Return type:

SubprocessResults
port_forward(remote_host, remote_port=None, local_port=None)[source]

Initiate a port forward connection.

This method establishes a local port forwarding from a local port local to remote port remote. If local is None, an available local port is automatically chosen. If the remote port is already forwarded, a new connection is not established.

Parameters:
  • remote_host (str) – The hostname of the remote host in form: ‘hostname(:port)’.
  • remote_port (int, None) – The remote port of the service.
  • local_port (int, None) – The port to use locally (automatically determined if not specified).
Returns:

The local port which is port forwarded to the remote service.
Return type:

int
has_port_forward(remote_host=None, remote_port=None, local_port=None)[source]

Check whether a port forward connection exists.

Parameters:
  • remote_host (str) – The hostname of the remote host in form: ‘hostname(:port)’.
  • remote_port (int, None) – The remote port of the service.
  • local_port (int, None) – The port used locally.
Returns:

Whether a port-forward for this remote service exists, or if

local port is specified, whether that port is locally used for port forwarding.
Return type:

bool
port_forward_stop(local_port=None, remote_host=None, remote_port=None)[source]

Disconnect an existing port forward connection.

If a local port is provided, then the forwarding (if any) associated with that port is found and stopped; otherwise any established port forwarding associated with the nominated remote service is stopped.

Parameters:
  • remote_host (str) – The hostname of the remote host in form: ‘hostname(:port)’.
  • remote_port (int, None) – The remote port of the service.
  • local_port (int, None) – The port used locally.
port_forward_stopall()[source]

Disconnect all existing port forwarding connections.

get_local_uri(uri)[source]

Convert a remote uri to a local one.

This method takes a remote service uri accessible to the remote host and returns a local uri accessible directly on the local host, establishing any necessary port forwarding in the process.

Parameters:uri (str) – The remote uri to be made local.
Returns:A local uri that tunnels all traffic to the remote host.
Return type:str
show_port_forwards()[source]

Print to stdout the active port forwards associated with this client.

is_port_bound(host, port)[source]

Check whether a port on a remote host is accessible.

This method checks to see whether a particular port is active on a given host by attempting to establish a connection with it.

Parameters:
  • host (str) – The hostname of the target service.
  • port (int) – The port of the target service.
Returns:

Whether the port is active and accepting connections.
Return type:

bool
dir(path=None)

Retrieve information about the children of a nominated directory.

This method returns a generator over FileSystemFileDesc objects that represent the files/directories that a present as children of the nominated path. If path is not a directory, an exception is raised. The path is interpreted as being relative to the current working directory (on remote filesytems, this will typically be the home folder).

Parameters:path (str) – The path to examine for children.
Returns:The children of path represented as FileSystemFileDesc objects.
Return type:generator<FileSystemFileDesc>
RemoteClient Quirks:
This method should return a generator over FileSystemFileDesc objects.
disconnect()

Disconnect this client from backing service.

This method is automatically called during reconnections and/or at Python interpreter shutdown. It first calls Duct._disconnect (which should be implemented by subclasses) and then notifies the RemoteClient subclass, if present, to stop port-forwarding the remote service.

Returns:A reference to this object.
Return type:Duct instance
download(source, dest=None, overwrite=False, fs=None)

Download files to another filesystem.

This method (recursively) downloads a file/folder from path source on this filesystem to the path dest on filesytem fs, overwriting any existing file if overwrite is True.

Parameters:
  • source (str) – The path on this filesystem of the file to download to the nominated filesystem (fs). If source ends with ‘/’ then contents of the the source directory will be copied into destination folder, and will throw an error if path does not resolve to a directory.
  • dest (str) – The destination path on filesystem (fs). If not specified, the file/folder is downloaded into the default path, usually one’s home folder. If dest ends with ‘/’, and corresponds to a directory, the contents of source will be copied instead of copying the entire folder. If dest is otherwise a directory, an exception will be raised.
  • overwrite (bool) – True if the contents of any existing file by the same name should be overwritten, False otherwise.
  • fs (FileSystemClient) – The FileSystemClient into which the nominated file/folder source should be downloaded. If not specified, defaults to the local filesystem.
exists(path)

Check whether nominated path exists on this filesytem.

Parameters:path (str) – The path for which to check existence.
Returns:
True if file/folder exists at nominated path, and False
otherwise.
Return type:bool
find(path_prefix=None, **attrs)

Find a file or directory based on certain attributes.

This method searches for files or folders which satisfy certain constraints on the attributes of the file (as encoded into FileSystemFileDesc). Note that without attribute constraints, this method will function identically to self.dir.

Parameters:
  • path_prefix (str) – The path under which files/directories should be found.
  • **attrs (dict) – Constraints on the fields of the FileSystemFileDesc objects associated with this filesystem, as constant values or callable objects (in which case the object will be called and should return True if attribute value is match, and False otherwise).
Returns:

A generator over FileSystemFileDesc

objects that are descendents of path_prefix and which statisfy provided constraints.
Return type:

generator<FileSystemFileDesc>
is_connected()

Check whether this Duct instances is currently connected.

This method checks to see whether a Duct instance is currently connected. This is performed by verifying that the remote host and port are still accessible, and then by calling Duct._is_connected, which should be implemented by subclasses.

Returns:Whether this Duct instance is currently connected.
Return type:bool
isdir(path)

Check whether a nominated path is directory.

Parameters:path (str) – The path for which to check directory nature.
Returns:True if folder exists at nominated path, and False otherwise.
Return type:bool
isfile(path)

Check whether a nominated path is a file.

Parameters:path (str) – The path for which to check file nature.
Returns:True if a file exists at nominated path, and False otherwise.
Return type:bool
mkdir(path, recursive=True, exist_ok=False)

Create a directory at the given path.

Parameters:
  • path (str) – The path of the directory to create.
  • recursive (bool) – Whether to recursively create any parents of this path if they do not already exist.

Note: exist_ok is passed onto subclass implementations of _mkdir rather that implementing the existence check using .exists so that they can avoid the overhead associated with multiple operations, which can be costly in some cases.

open(path, mode='rt')

Open a file for reading and/or writing.

This method opens the file at the given path for reading and/or writing operations. The object returned is programmatically interchangeable with any other Python file-like object, including specification of file modes. If the file is opened in write mode, changes will only be flushed to the source filesystem when the file is closed.

Parameters:
  • path (str) – The path of the file to open.
  • mode (str) – All standard Python file modes.
Returns:

An opened file-like object.
Return type:

FileSystemFile or file-like
path_home

The path prefix to use as the current users’ home directory. Unless cwd is set, this will be the prefix to use for all non-absolute path references on this filesystem. This is assumed not to change between connections, and so will not be updated on client reconnections. Unless global_writes is set to True, this will be the only folder into which this client is permitted to write.

Type:str
path_separator

The character(s) to use in separating path components. Typically this will be ‘/’.

Type:str
prepare()

Prepare a Duct subclass for use (if not already prepared).

This method is called before the value of any of the fields referenced in self.connection_fields are retrieved. The fields include, by default: ‘host’, ‘port’, ‘remote’, ‘cache’, ‘username’, and ‘password’. Subclasses may add or subtract from these special fields.

When called, it first checks whether the instance has already been prepared, and if not calls _prepare and then records that the instance has been successfully prepared.

RemoteClient Quirks:

This method may be overridden by subclasses, but provides the following default behaviour:

  • Ensures self.registry, self.remote and self.cache values are instances of the right types.
  • It replaces string values of self.remote and self.cache with remotes and caches looked up using self.registry.lookup.
  • It looks through each of the fields nominated in self.prepared_fields and, if the corresponding value is callable, sets the value of that field to result of calling that value with a reference to self. By default, prepared_fields contains ‘_host’, ‘_port’, ‘_username’, and ‘_password’.
  • Ensures value of self.port is an integer (or None).
remove(path, recursive=False)

Remove file(s) at a nominated path.

Directories (and their contents) will not be removed unless recursive is set to True.

Parameters:
  • path (str) – The path of the file/directory to be removed.
  • recursive (bool) – Whether to remove directories and all of their contents.
walk(path=None)

Explore the filesystem tree starting at a nominated path.

This method returns a generator which recursively walks over all paths that are children of path, one result for each directory, of form: (<path name>, [<directory 1>, …], [<file 1>, …])

Parameters:path (str) – The path of the directory from which to enumerate contents.
Returns:A generator of tuples, each tuple being associated with one directory that is either path or one of its descendants.
Return type:generator<tuple>

Subclass Reference

For comprehensive documentation on any particular subclass, please refer to one of the below documents.