The University of California, San Diego
UCSD

Site Contents

Page Contents

  1. Usher Development
  2. Code Overview
    1. Directory Structure
    2. Usher Components
      1. Controller (usher/ctrl/)
        1. Unraveling Twisted
      2. Local Node Manager (usher/lnm/)
      3. Client API (usher/cli/)
      4. Utilities (usher/utils/)
    3. Usher Events
      1. Client Events
      2. Cluster Events
      3. Controller Events
      4. LNM Events
      5. VM Events
  3. Writing Client Applications
  4. Writing Plugins
    1. Caveats
  5. Writing Credential Checkers
  6. Code Repositories

Usher Development

This page gives an overview of the Usher code including information on writing Usher clients and plugins. For additional help or information, check out the Usher mailing lists referred to on the mailing lists page.

Another excellent source of information about Usher and design goals of the Usher system can be found in the paper:

Usher: An Extensible Framework for Managing Clusters of Virtual Machines, Marvin McNett, Diwaker Gupta, Amin Vahdat, and Geoffrey M. Voelker, 21st Large Installation System Administration Conference, (LISA 2007).

Finally, Let me know if you have questions about this documentation or if I'm lacking any important details. IOW, don't bang your head against the wall, just send me an email if you get stuck or confused. I'll also try to clarify any confusing points anyone brings to my attention.

Code Overview

Before reading too much, if you're only interested in writing an Usher client or plugin, you can skim or completely skip this section and go directly to either the Writing Client Applications or Writing Plugins section.

Code for the Usher system is written in Python and makes heavy use of Twisted, an event-driven networking framework also written in Python. In particular, Twisted's Perspective Broker module. See the documentation on the Twisted site for the nitty gritty details of using this module. As shown in Figure 1 in the Software Overview section of the Documentation page, the Usher system consists of three main components: a centralized controller, local node managers (LNMs), and clients.

Let’s go straight to the code to see where these are implemented.

Directory Structure

Below is a tree listing of the code directory (usher/) in the top-level project directory. 

usher/
|-- __init__.py
|-- cli
|   |-- __init__.py
|   |-- api.py
|   |-- callbacks.py
|   |-- config.py
|   `-- vm.py
|-- ctrl
|   |-- __init__.py
|   |-- app.tac
|   |-- client.py
|   |-- cluster.py
|   |-- config.py
|   |-- lnm.py
|   |-- request.py
|   |-- server.py
|   `-- vm.py
|-- lnm
|   |-- __init__.py
|   |-- app.tac
|   |-- config.py
|   |-- server.py
|   |-- vm.py
|   `-- xen_vmm.py
|-- plugins
|   |-- sample.py
|   `-- sample2
|       |-- __init__.py
|       `-- plugin.py
`-- utils
    |-- __init__.py
    |-- config.py
    |-- credcheck.py
    |-- events.py
    |-- misc.py
    |-- notify.py
    |-- plugin.py
    |-- result.py
    |-- upb.py
    `-- usherr.py

6 directories, 34 files

Also in the top-level directory are configs/ and initscripts/. The configs directory contains default config files for each of the controller, lnms, and client. Each file contains a line indicating where Usher will check to find the file or an environment variable to specify where to look. The /initscripts directory contains simple startup scripts for both the local node managers and controller. After editing the files for your system, they should be placed where your system will find and run them at boot time, or run manually when you want to start the services.

Usher Components

From the listing above, notice that each of the main components has a directory in the usher/ directory wherein its source code resides. I'll now discuss these in turn.

Controller (usher/ctrl/)

Beginning with the controller, we see eight files. Below, I'll give a brief explanation about the purpose of each and any important details. For further information, see the code itself.

app.tac: This is a Twisted application file. In a nutshell, Twisted provides facilities for creating a forking, daemonized server using their twistd application. This saves us from having to write that part of the code ourselves. Regular Python code is used in the app.tac file which essentially contains code for starting various Twisted servers under a single controlling process. If you need an additional listening server (or client connection) at process start, it should be added there.

client.py: This file provides the implementation of the Twisted IRealm interface for use by clients as well as the Client API (the pb.Avatar). This is the API exposed to clients connecting to the controller.

cluster.py: This defines the Cluster object which is a pb.Cacheable. Not much is really done with this at this point. It is updated and made available to plugins for their use.

config.py: This file subclasses the Config object (found in usher/utils/config.py). It contains a ConfigObj specification file which specifies what configuration parameters will be accepted and default values for each. Each new configuration parameter should be added to this specification. Configuration files are read at service startup and values set at that time. The config.py file contains a function named get_cfg for retrieving a reference to the configuration object. An optional section parameter can be passed to get_cfg to get only values from a particular section of the configuration file.

lnm.py: This is the LNM counterpart to client.py. It provides the implementation of the Twisted IRealm interface for use by LNMs as well as the LNM API (the pb.Avatar). This is the API exposed to LNMs connecting to the controller.

request.py: This file defines the Request object and all Request subclasses (StartRequest. MigrateRequest, etc). Every request to the controller from a client generates a Request object which is first passed to plugins registered for the request type. After traversing its plugin chains, this object is used to initiate the request via its execute method.

server.py: This file contains the core of the server code. Looking at the code for the controller, we first see the UsherCtrl class. The UsherCtrl class is defined here which is basically a container class for maintaining global state variables and provides accessor methods to those needed elsewhere. In addition, the constructor sets up the services we start in the app.tac file and retains references to those.

vm.py: This file provides the implementation of the VM object. This is a pb.Cacheable which gets passed to clients and LNMs. Clients which own a VM always receive a cached copy of the VM object on the controller. Likewise, LNMs get a cached copy of every VM running locally. This object provides most methods for manipulating VMs (e.g. start, shutdown, migrate, etc). In addition, all operations on a particular VM are serialized here.

Unraveling Twisted

Before moving on, just a few notes on what all the Twisted lingo used above really means, and how Twisted's Perspective Broker works. If you're just skimming, you can probably skip this section.

First, in Twisted’s Perspective Broker module, a Realm generates capabilities which are returned to authenticated clients. In Twisted speak, the Realm returns an ‘Avatar’, which is a remote reference to an API. What do we do with a Realm in Twisted? We pass it, along with a list of Twisted credential checkers (something that implements the twisted.cred.checkers.ICredentialsChecker interface), to the constructor of the twisted.cred.portal.Portal class to generate our portal. From the Twisted API documentation for twisted.cred.portal.Portal

A portal is associated with one Realm and zero or more credentials checkers. When a login is attempted, the portal finds the appropriate credentials checker for the credentials given, invokes it, and if the credentials are valid, retrieves the appropriate avatar from the Realm.

That about says it all as far as portals are concerned. The portal is then given to the server factory for perspective broker (twisted.spread.pb.PBServerFactory), which is passed to the twisted.application.internet.SSLServer method (in app.tac) to connect the factory to the network and start the Twisted event loop.

Getting down to the gory details, when a client connects to the controller (I'll restrict the discussion to CliRealm since it's basically the same for LNMRealm), the requestAvatar method of the CliRealm instance gets fired. This method returns a reference to a pb.Avatar (a capability) to the client. This method does a few things, like add the actual avatar to the client's dictionary in the UsherCtrl instance if it doesn’t already exist, and calls the avatar’s attached method.

More interesting is the mind parameter to the requestAvatar method. This is a remote reference back to the client which is passed by the client when running login. Through this, the controller is able to call back methods which the client exposes to the controller (by prepending the method names with remote_). This is quite handy for sending notifications back to the client. Finally, notice that the requestAvatar returns a 3-tuple. The first item of the tuple is simply the interface which the avatar implements. The second is the remote reference to the avatar. The third is a no-argument function which the protocol should call when the client connection has been lost.

Now, the fun begins. Let’s continue with the client. When a client connects (via login), the CliRealm instance returns a reference to an instance of the Client Avatar. This Avatar is essentially a remote reference to an API exposed by the controller to the client. With this remote reference (capability), the remote client can call any methods beginning with perspective_ in the Client class (i.e. the pb.Avatar class). These methods, in turn, generate Request objects which are passed to plugin chains before being carried out.

In Twisted, all remote method calls are asynchronous, immediately returning a deferred (i.e. an instance of the twisted.internet.defer.Deferred class). This deferred is then called back with the result of the remote procedure call when it becomes available. Handling this result is done via a callback chain which is created on the caller’s side using the deferred’s addCallback method. A callback chain can be made arbitrarily long by simply adding more and more callbacks with this method. The first callback in the chain is passed the result of the RPC as it’s first parameter. Subsequent callbacks are called with the result of the previous as their first parameter. Errors can be handled in a similar manner using the deferred’s addErrback method. The fine details of Twisted deferred can be found in the main Twisted manual.

So, nearly all remote API calls are followed by the creation of a callback chain to handle the results. Hence, all the real work is done by functions added to the callback chain in the API method.

Local Node Manager (usher/lnm/)

Next, I'll give a brief explanation about the purpose of each of the LNM files and any important details. For further information, see the code itself.

app.tac: See the explanation for the controller's app.tac file above.

config.py: See the explanation for the controller's config.py file above.

server.py: This file contains the main LNM server code. It also provides the API used by the controller to request VM administrative operations. These methods are found in the LNM class beginning with 'remote_'.

vm.py: This file implements the LNMVM class which is a pb.RemoteCache object for the VM objects created at the controller. This file handles setting up the cached copy and methods for keeping it synchronized with the real VM object at the controller.

xen_vmm.py: This file provides a wrapper around the Xen virtual machine manager administrative API for use by the LNM. This wrapper is used by the LNM to manage locally running virtual machines. Perhaps this should be pulled out of the main distribution, but I'll worry about that if and when additional wrappers are written for other VMMs.

LNMs are simply clients of the controller which connect to a different service than Usher clients. Being a client is slightly different from a server, and the main class, LNM in the LNM server.py file, subclasses pb.Referenceable. This allows our main class to define methods beginning with remote_ to be made available to anything holding a reference to the LNM instance. Hence, when our LNM server calls the login function, it passes a reference to itself (the client argument), to the controller. Now, any methods beginning with remote_ can be called by the controller.

So, why isn’t the controller a client of the LNMs? In a way, it is, since the PB protocol is symmetric once a connection is established -- each end has a reference to an API on the other. So we make the LNM the client in this connection since it’s much easier for the LNMs to find the centralized controller than the other way around. In addition, we may want the LNMs to have to provide credentials to the controller before becoming a trusted member of the system. Users may not want their VMs running on unauthenticated nodes.

Finally, notice that all Virtual Machine Manager specific code has been pulled out into a separate module. The module should be named <VMM type>_VMM.py. The LNM class of server.py subclasses the LNMBase class of the VMM specific module. Currently, only Xen_VMM.py exists.

Client API (usher/cli/)

Next, I'll give a brief explanation about the purpose of each of the client files and any important details. For further information, see the code itself.

api.py: This is a library which actual clients can use to access the Usher controller. Developers wishing to tie their application to the Usher system can use the API presented in api.py to do so. Clients create an instance of the API by simply importing api. From there, the instance can be grabbed using the api.get_api method.

callbacks.py: A callback interface which applications can subclass to receive event notifications. Applications must override the methods there to receive callbacks for the respective event. This isn't well thought out and will probably change.

config.py: See the explanation for the controller's config.py file above.

vm.py: See the explanation for the LNM's vm.py file above.

Utilities (usher/utils/)

Utility modules are located in the usher/utils/ directory. These include:

config.py: Other modules get their config object by importing this module, and using the 'get_cfg' function to receive a ConfigObj instance containing configuration parameters. This module also adds a few parameter types.

credcheck.py: Implements Twisted's checkers.ICredentialsChecker interface to allow plugins registered for the client_authenticate and lnm_authenticate events to authenticate users. See the Writing Credential Checkers section below for information on writing credential checkers.

events.py: Contains a dictionary with all valid events for which plugins can be registered in the Usher system. These are listed below. Also defines the important EventListDispatcher class which is the object with which plugins are registered to be called for the respective event.

misc.py: Just what it sounds like. Some miscellaneous methods which don't really belong anywhere else.

notify.py: Logging and error notification methods

plugin.py: Provides the UsherPlugin class which all plugins subclass. Also contains utility methods for finding available plugins at controller startup and handling plugin specifications.

result.py: Defines the very important UsherResult class. An UsherResult instance is returned by the controller for most Client API calls. Also contains the UsherResPkgr class which packages results from a list of deferreds from an API call into an UsherResult.

upb.py: Necessary because of Twisted's inadequate credential checking system

usherr.py: Defines all Usher Exceptions.

Usher Events

Here are all valid events for which plugins can be registered. The input passed to the plugin chain for each event is also included.

Client Events

  • client_authenticate: A Client is attempting to authenticate. Plugin input: ((username:str, password:str), valid:bool). If valid=True after completion of plugin chain, user is authenticated
  • client_connect: A client has connected. Plugin input: (usher.ctrl.client.Client)
  • client_disconnect: A client has disconnected'. Plugin input: (usher.ctrl.client.Client)

Cluster Events

  • cluster_register: A new cluster has been registered with the controller. Plugin input: (usher.ctrl.cluster.Cluster)

Controller Events

  • ctrl_start: The Controller has started. Plugin input: (usher.ctrl.server.UsherCtrl)
  • list_request: A request to list VMs has been received. Plugin input: (usher.ctrl.request.ListRequest)
  • periodic: A periodic timer has fired. For scheduling periodic tasks. Plugin input: (). Note configuration must have a 'period' field
  • request: A request has been received. Plugin input: (usher.ctrl.request.Request)
  • timer: A timer has fired. For scheduling 1-shot future tasks. Plugin input: (). Note configuration must have a 'when' field.
  • lnm_list_request: A request to list LNMs has been received. Plugin input: (usher.ctrl.request.LNMListRequest)

LNM Events

  • lnm_authenticate: An LNM is attempting to authenticate. Plugin input: ((username:str, password:str), valid:bool). If valid=True after completion of plugin chain, user is authenticated
  • lnm_connect: A Local Node Manager has connected. Plugin input: (usher.ctrl.lnm.LNM)
  • lnm_disconnect: A Local Node Manager has disconnected. Plugin input: (usher.ctrl.lnm.LNM)

VM Events

  • cycle: A VM has been cycled. Plugin input: (usher.ctrl.vm.VM)
  • cycle_failure: A VM cycle has failed. Plugin input: (usher.ctrl.vm.VM, twisted.python.failure.Failure)
  • cycle_request: A request to cycle a list of VMs has been received. Plugin input: (usher.ctrl.request.CycleRequest)
  • migrate: A VM has been migrated. Plugin input: (usher.ctrl.vm.VM, from_lnm:usher.ctrl.lnm.LNM)
  • migrate_failure: A VM migrate has failed. Plugin input: (usher.ctrl.vm.VM, twisted.python.failure.Failure)
  • migrate_request: A request to migrate a list of VMs has been received. Plugin input: (usher.ctrl.request.MigrateRequest)
  • pause: A VM has been paused. Plugin input: (usher.ctrl.vm.VM)
  • pause_failure: A VM pause has failed. Plugin input: (usher.ctrl.vm.VM, twisted.python.failure.Failure)
  • pause_request: A request to pause a list of VMs has been received. Plugin input: (usher.ctrl.request.PauseRequest)
  • poweroff: A VM has been powered off. Plugin input: (usher.ctrl.vm.VM)
  • poweroff_failure: A VM poweroff has failed. Plugin input: (usher.ctrl.vm.VM, twisted.python.failure.Failure)
  • poweroff_request: A request to poweroff a list of VMs has been received. Plugin input: (usher.ctrl.request.PoweroffRequest)
  • reboot: A VM has been rebooted. Plugin input: (usher.ctrl.vm.VM)
  • reboot_failure: A VM reboot has failed. Plugin input: (usher.ctrl.vm.VM, twisted.python.failure.Failure)
  • reboot_request: A request to reboot a list of VMs has been received. Plugin input: (usher.ctrl.request.RebootRequest)
  • register: A VM has been registered with the controller. Plugin input: (usher.ctrl.vm.VM). This happens before starting and when a newly discovered VM is reported by and lnm (E.g. when the LNM connects and already has running VMs)
  • resume: A VM has been resumed. Plugin input: (usher.ctrl.vm.VM)
  • resume_failure: A VM resume has failed. Plugin input: (usher.ctrl.vm.VM, twisted.python.failure.Failure)
  • resume_request: A request to resume a list of VMs has been received. Plugin input: (usher.ctrl.request.ResumeRequest)
  • shutdown: A VM has been shutdown. Plugin input: (usher.ctrl.vm.VM)
  • shutdown_failure: A VM shutdown has failed. Plugin input: (usher.ctrl.vm.VM, twisted.python.failure.Failure)
  • shutdown_request: A request to shutdown a list of VMs has been received. Plugin input: (usher.ctrl.request.ShutdownRequest)
  • unregister: A VM has unregistered with the controller. Plugin input: (usher.ctrl.vm.VM)
  • start: A VM has been started. Plugin input: (usher.ctrl.vm.VM)
  • start_failure: A VM start has failed. Plugin input: (usher.ctrl.vm.VM, twisted.python.failure.Failure)
  • start_request: A request to start a list of VMs has been received. Plugin input: (usher.ctrl.request.StartRequest)
  • state_change: The state of a VM has changed. Plugin input: (usher.ctrl.vm.VM)

Writing Client Applications

First, you'll need to have the client requirements given on the Documentation page installed to develop your client.

Usher client applications utilize the Usher client API (usher/cli/api.py) to interact with the Usher controller. Clients import usher.cli.api, then create an usher.cli.api.API instance for interaction with the controller.

After that, it's a simple matter of having your application use the methods provided by your API instance. Not really much more to it than that. See the docstrings in the usher/cli/api.py file for information on using these methods (signatures, parameter descriptions, etc).

The only additional comment I'll add is about configuration files for your clients. First, decide upon an appropriate name for a section title for your application's configuration file. For example, Ush uses 'ush' for its configuration file section. Then, using the following, create a configuration file handler for your application by specifying the section_title variable and your configuration specification and saving it as config.py in your application's source directory. 

import os
from usher.cli import config

# Title of section for this application in its configuration file
section_title=<set this for your application>



# Environment variable used to specify config_file
config_file_env_var = '%s_CONFIG'%section_title.upper()

# Default path to the config and specification files.
default_config = '/etc/usher/%s.config'%section_title

# Define your specification below.  See:
#     http://www.voidspace.org.uk/python/configobj.html#validation
# for more information on writing your specification
specification = """
[%s]

#var1 = boolean
#var2 = integer
#var3 = float
# string is default type if not specified
#var4 = string
#var2 = boolean(default=False)
#var1 = integer(0,5000, default=300)
#var5 = float(default=3.14159)
#var3 = string(max=20, default=0)
#var4 = option('onefish', 'twofish', 'redfish', 'bluefish')

"""%section_title

config_files = [default_config]

# user config
config_files.append(os.path.expanduser('~/.usher/%s.config'%section_title))

env_config = os.getenv(config_file_env_var)
if env_config:
    config_files.append(env_config)

def get_cfg(section=None, spec=specification):
    """Get the Config instance.
    """
    return config.get_cfg(config_files, spec, section=section)

See the the ConfigObj Validation section of the ConfigObj documentation for details on writing a specification. A specification isn't required, but is probably a good idea. It's also fairly straightforward how to write one using the ConfigObj docs and examples.

With the above, configuration files will be read from:

  • /etc/usher/<section_title>.config where 'section_title' is whatever you named your section.
  • ~/.usher/<section_title>.config
  • anything pointed to by the environment variable <SECTION_TITLE>_CONFIG (where 'SECTION_TITLE' is whatever you named your section capitalized)

To get both your application's and the client API's configuration sections as a dictionary of dictionaries, just call the get_cfg method without specifying a section. Then, to get options from a particular section, it's just a matter of indexing with that section first. For example, to get the value of the client API's ctrl_host option, use 

cfg = config.get_cfg()
ctrl_host = cfg['cli']['ctrl_host']

If your section (named 'mysection') has an option named 'myopt': 

myopt = cfg['mysection']['myopt']

If you only need options from a particular section, you can specify that section in the get_cfg call. Then, you only need the option as an index. For example, if your section (named 'mysection') has an option named 'myopt': 

cfg = config.get_cfg('mysection')
myopt = cfg['myopt']

Not much more to say here. Let me know if you are or are planning on writing a client and I'll be happy to provide support and feedback.

Writing Plugins

Here, I'll assume you're familiar with the contents of the Plugins page.

The usher/utils/plugin.py file provides the UsherPlugin class which all plugins must subclass. Check out this class to see what it does. The most important thing to see here is that subclasses of UsherPlugin must define the entry_point method.

When a plugin is registered for an event, an instance of its UsherPlugin subclass is placed on the callback list for that event. When the corresponding event fires, the entry_point method is called for each plugin on the callback list in order of priority (set by the order instance variable or order of registration). The arguments passed to the plugin are given above in the Usher Events section (and also in the usher/utils/events.py file).

When the plugin completes execution, its entry_point method must return a tuple of the same type as it received. That is, the return type for the entry_point method must be the same as its arguments. This is necessary since this is passed as arguments to subsequent plugins on the same callback list. This is not to say that your plugin cannot modify these arguments (for mutable objects), or replace them (for immutable objects). On the contrary, being able to modify these arguments or return something different (but with the same type) is a very powerful feature of the callback list.

If your plugin will be registered for multiple events, the entry_point method will essentially be a dispatcher to the appropriate class method designed to handle the event. As a simple example, let's look at the entry_point method for the Usher DNS plugin: 

def entry_point(self, *args):
    if self.event == 'register':
        vm = args[0]
        self.zone = vm.usherctrl.cfg.get('name_suffix')
        self.add_vm(vm)
    elif self.event == 'unregister':
        vm = args[0]
        self.zone = vm.usherctrl.cfg.get('name_suffix')
        self.remove_vm(vm)
    return args

For the register and unregister events, a VM object is passed as the only item in the args tuple. The entry_point method calls the appropriate method based on the event for which its instance was registered. Also notice that this entry_point method simply returns the args tuple it received unmodified.

For a more complicated entry_point method example, see the Usher IP Management plugin.

Getting back to the UsherPlugin class, plugin writers also to set the following class variables in their UsherPlugin subclass:

  • name - plugin name (often just __name__)
  • author - plugin author
  • description - brief description of the plugin (often just __doc__ if you've added a docstring to your module)
  • version - plugin version

Also, plugin writers should define their plugin's configuration specification in the specification variable at the top of their plugin module, or in the plugin's __init__.py file if the plugin is a package rather than a single file. There's a simple specification example in the Writing Client Applications section above. See the the ConfigObj Validation section of the ConfigObj documentation for additional details on writing a specification. A specification isn't required, but is probably a good idea. It's also fairly straightforward how to write one using the ConfigObj docs and examples.

Plugins should also be shipped with a sample configuration file (at least in a README) which shows an example of registering the plugin and what events the plugin was written to handle. See the README that ships with the udns plugin for an example.

Optionally, plugins can create new events for which plugins can be registered. This is another powerful feature of plugins. To do this, simply create a dictionary named events in your plugin module or in your plugin packages __init__.py file with the name of the events as dictionary keys, and a brief description of each event as values. Then, at the appropriate point in your plugin, call the controller's fire_event method, passing the new event name as the argument.

Two sample plugins have been included in the main distribution in the usher/plugins directory. The first sample.py is a single file plugin which writes a few messages to the controller's log file. In addition, this file creates a new event called 'sample_event' which it fires after sleeping for 5 seconds. The second sample, sample2, is a plugin package illustrating multi-file plugin setup. This sample also merely prints a few messages to the controller's log file.

This should be all you need to get started writing your plugin. If your plugin needs to access or modify controller state, you'll need to familiarize yourself with the controller code. Start by reading the Code Overview section above, then digging into the code and docstrings therein. Also, be sure to check out any existing plugins for additional examples. Plugins are typically short, single files, so going to the code is often the best course of action when you have questions.

Caveats

When a plugin is registered, it is placed on a callback list for the event for which it was registered. Each of these callback lists runs in a separate thread so that slow plugins only slow down their callback chain (and not the entire controller). This also makes it easier for an administrator to identify slow, or broken plugins. This design does come at a cost.

Plugins are called in order on any given event callback list. However, since each list itself is run as a separate thread, there may be multiple event callback lists in execution at any one time. For this reason, plugins should be used with caution since they are free to perform arbitrary action, even actions which manipulate the internal state of the controller. In particular proper locking should be considered when writing plugins which modify controller state.

Since most plugins do not directly modify controller state, the above is typically not a problem. Just be aware of it if you do write or use a plugin which does.

Writing Credential Checkers

Writing a credential checker is surprisingly easy. Plugins registered for the client_authenticate and lnm_authenticate events receive a tuple of the form: ((username:str, password:str), valid:bool) as input. Note that the return type of a plugin must be the same as its input type (see the Writing Plugins section above for information about writing plugins)

So, credential checkers merely need to return a tuple of type: ((str, str), bool) with the boolean field set to True for authentication successful, or False for authentication failed.

Code Repositories

The Usher code is maintained in Mercurial repositories and are made publicly available. See the Mercurial section of the downloads page for links to these repositories.

Last Updated: 2008-09-18 by mmcnett
Send comments and problem reports to: Marvin McNett