e2etest
— Framework for writing integration tests for aioxmpp
¶
This subpackage provides utilities for writing end-to-end or intgeration tests
for aioxmpp
components.
Warning
For now, the API of this subpackage is classified as internal. Please do not test your external components using this API, as it is experimental and subject to change.
Overview¶
The basic concept is that tests are written like normal unittests. However,
tests are written by inheriting classes from aioxmpp.e2etest.TestCase
instead of unittest.TestCase
. e2etest.TestCase
has the
provisioner
attribute which provides access to a
provision.Provisioner
instance.
Provisioners are objects which provide a way to obtain a connected XMPP client.
The JID to which the client is bound is unspecified; however, each client gets
a unique bare JID and the clients are able to communicate with each other. In
addition, provisioners provide information about the environment in which the
clients act. This includes providing JIDs of entities implementing specific
protocols or features. The details are explained in the documentation of the
Provisioner
base class.
By default, tests which are written with e2etest.TestCase
are skipped
when using the normal test runners. This is because the provisioners need to be
configured; this is handled using a custom nosetests plugin which is not loaded
by default (for good reasons). To run the tests, use (instead of the normal
nosetests3
binary):
$ python3 -m aioxmpp.e2etest
The command line interface is identical to the one of nosetests3
, except
that additional options are provided to configure the plugin. In fact,
aioxmpp.e2etest
is simply a nose test runner with an additional plugin.
By default, the configuration is read from ./.local/e2etest.ini
. For
details on configuring the provisioners, see the developer guide.
Main API¶
Decorators for test methods¶
The following decorators can be used on test methods (including setUp
and
tearDown
):
- @aioxmpp.e2etest.require_feature(feature_var, argname=None, *, multiple=False)[source]¶
- Parameters:
feature_var (
str
) – XEP-0030 featurevar
of the required featureargname (
str
orNone
) – Optional argument name to pass theFeatureInfo
tomultiple (
bool
) – If true, all peers are returned instead of a random one.
Before running the function, it is tested that the feature specified by feature_var is provided in the environment of the current provisioner. If it is not,
unittest.SkipTest
is raised to skip the test.If the feature is available, the
FeatureInfo
instance is passed to the decorated function. If argname isNone
, the feature info is passed as additional positional argument. otherwise, it is passed as keyword argument using the argname.If multiple is true, all peers supporting the given feature are passed in a set. Otherwise, only a random peer is returned.
This decorator can be used on test methods, but not on test classes. If you want to skip all tests in a class, apply the decorator to the
setUp
method.
- @aioxmpp.e2etest.skip_with_quirk[source]¶
- Parameters:
quirk (
Quirks
) – The quirk to skip on
If the provisioner indicates that the environment has the given quirk, the test is skipped.
This decorator can be used on test methods, but not on test classes. If you want to skip all tests in a class, apply the decorator to the
setUp
method.
General decorators¶
- @aioxmpp.e2etest.blocking[source]¶
The decorated coroutine function is run using the
run_until_complete()
method of the current (at the time of call) event loop.The decorated function behaves like a normal function and is not a coroutine function.
This decorator must be applied to a coroutine function (or method).
- @aioxmpp.e2etest.blocking_timed[source]¶
Like
blocking_with_timeout()
, the decorated coroutine function is executed usingasyncio.AbstractEventLoop.run_until_complete()
with a timeout, but the timeout is configured in the end-to-end test configuration (see End-to-end tests (or integration tests)).This is the recommended decorator for any test function or method, to prevent the tests from hanging when anythin goes wrong. The timeout is under control of the provisioner configuration, which means that it can be adapted to different setups (for example, running against an XMPP server in the internet will be slower than if it runs on localhost).
The decorated function behaves like a normal function and is not a coroutine function.
This decorator must be applied to a coroutine function (or method).
- @aioxmpp.e2etest.blocking_with_timeout[source]¶
The decorated coroutine function is run using the
run_until_complete()
method of the current (at the time of call) event loop.If the execution takes longer than timeout seconds,
asyncio.TimeoutError
is raised.The decorated function behaves like a normal function and is not a coroutine function.
This decorator must be applied to a coroutine function (or method).
Class for test cases¶
- class aioxmpp.e2etest.TestCase(methodName='runTest')[source]¶
A subclass of
unittest.TestCase
for end-to-end test cases.This subclass provides a single additional attribute:
- provisioner¶
This is the configured
provision.Provisioner
instance.If no provisioner is configured (for example because the e2etest nose plugin is not loaded), this reads as
None
.Note
Under nosetests and the vanilla unittest runner, tests inheriting from
TestCase
are automatically skipped ifprovisioner
isNone
.
Provisioners¶
- class aioxmpp.e2etest.provision.Provisioner(logger=<Logger aioxmpp.e2etest.provision (WARNING)>)[source]¶
Base class for provisioners.
Provisioners are responsible for providing test cases with XMPP accounts and client objects connected to these accounts, as well as information about the environment the accounts live in.
A provisioner must implement the following methods:
- abstract async _make_client(logger)[source]¶
- Parameters:
logger – The logger to pass to the client.
- Returns:
Client with a fresh account.
Construct a new
aioxmpp.PresenceManagedClient
connected to a new account. This method must be re-implemented by subclasses.
- abstract configure(section)[source]¶
Read the configuration and set up the provisioner.
- Parameters:
section – mapping of config keys to values
Subclasses will implement this to configure their account setup and servers to use.
See also
configure_tls_config()
for a function which extracts TLS-related arguments for
aioxmpp.security_layer.make()
configure_quirks()
for a function which extracts a set of
Quirk
enumeration members from the configurationconfigure_blockmap()
for a function which extracts a mapping which allows to block features from specific hosts
The following methods are the API used by test cases:
- async get_connected_client(presence=<PresenceState available>, *, services=[], prepare=None)[source]¶
Return a connected client to a unique XMPP account.
- Parameters:
presence (
aioxmpp.PresenceState
) – initial presence to emitprepare (coroutine receiving the client as argument) – a coroutine run after the services are summoned but before the client connects.
- Raises:
OSError – if the connection failed
RuntimeError – if a client could not be provisioned due to resource constraints
- Returns:
Connected presence managed client
- Return type:
Each account used by the clients returned from this method is unique; all clients are guaranteed to have different bare JIDs.
The clients and accounts are cleaned up after the tear down of the test runs. Some provisioners may have a limit on the number of accounts which can be used in the same test.
Clients obtained from this function are cleaned up automatically on tear down of the test. The clients are stopped and the accounts deleted or cleared, so that each test starts with a fully fresh state.
A coroutine may be passed as prepare argument. It is called with the client as the single argument after all services in services have been summoned but before the client connects, this is for example useful to connect signals that fire early in the connection process.
- get_feature_provider(feature_nses)[source]¶
- Parameters:
feature_ns (iterable of
str
) – Namespace URIs to find a provider for- Returns:
JID of the entity providing all features
- Return type:
If there is no entity supporting all requested features,
None
is returned.
- has_quirk(quirk)[source]¶
- Parameters:
quirk (
Quirk
) – Quirk to check for- Returns:
true if the environment has the given quirk
These methods can be used by provisioners to perform plumbing tasks, such as shutting down clients or deleting accounts:
- async initialise()[source]¶
Called once on test framework startup.
Subclasses may run service discovery code here to detect features of the environment they are connected to.
See also
discover_server_features()
for a function which uses XEP-0030 service discovery to find features.
- async teardown()[source]¶
Called after each test run.
The default implementation cleans up the clients obtained from
get_connected_client()
.
- class aioxmpp.e2etest.provision.AnonymousProvisioner[source]¶
This provisioner uses SASL ANONYMOUS to obtain accounts.
It is dead-simple to configure: it needs a host to connect to, and optionally some TLS and quirks configuration. The host is specified as configuration key
host
, TLS can be configured as documented inconfigure_tls_config()
and quirks are set as described inconfigure_quirks()
. A configuration for a locally running Prosody instance might look like this:[aioxmpp.e2etest.provision.AnonymousProvisioner] host=localhost no_verify=true quirks=[]
The server configured in
host
must support SASL ANONYMOUS and must allow communication between the clients connected that way. It may provide PubSub and/or MUC services, which will be auto-discovered if they are provided in the XEP-0030 items of the server.
- class aioxmpp.e2etest.provision.AnyProvisioner[source]¶
This provisioner randomly generates usernames and uses a hardcoded password to authenticate with the XMPP server.
This is for use with
mod_auth_any
of prosody.It is dead-simple to configure: it needs a host to connect to, and optionally some TLS and quirks configuration. The host is specified as configuration key
host
, TLS can be configured as documented inconfigure_tls_config()
and quirks are set as described inconfigure_quirks()
. A configuration for a locally running Prosody instance might look like this:[aioxmpp.e2etest.provision.AnyProvisioner] host=localhost no_verify=true quirks=[]
The server configured in
host
must allow authentication with any username/password pair and allow communication between the clients connected that way. It may provide PubSub and/or MUC services, which will be auto-discovered if they are provided in the XEP-0030 items of the server.
- class aioxmpp.e2etest.provision.StaticPasswordProvisioner[source]¶
This provisioner expects a list of username/password pairs to authenticate against the tested server.
This is for use with servers which support neither SASL ANONYMOUS nor a
mod_auth_any
equivalent.The configuration of this provisioner is slightly unwieldy since we do not want to add a dependency to a more sane configuration file format. Here is an example on how to configure a provisioner with two accounts:
[aioxmpp.e2etest.provision.StaticPasswordProvisioner] host=localhost accounts=[("user1", "password1"), ("user2", "password2")] skip_on_too_few_accounts=false
All accounts need to have exactly the same privileges on the server. The first account will be used to auto-discover any features offered by the test environment.
If skip_on_too_few_accounts is set to true (the default is false), tests will be skipped if the provisioner runs out of accounts instead of failing.
- class aioxmpp.e2etest.Quirk(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)[source]¶
Enumeration of implementation quirks.
Each enumeration member represents a quirk of an implementation. A quirk is a behaviour of an implementation which does not directly violate standards, but which is unfortunate in a way that it disables some features of
aioxmpp
.One example of such a quirk is the rewriting of message stanza IDs which some MUC implementations do when reflecting the messages. This breaks the stanza tracking of
aioxmpp.muc.Room.send_tracked_message()
.The following quirks are defined:
- MUC_REWRITES_MESSAGE_ID https://zombofant.net/xmlns/aioxmpp/e2etest/quirks#muc-id-rewrite¶
This quirk must be configured when the environment the provisioner provides rewrites the message IDs when they are reflected by the MUC implementation.
The quirk does not need to be set if the environment does not provide a MUC implementation at all.
- PUBSUB_GET_ITEMS_BY_ID_BROKEN https://zombofant.net/xmlns/aioxmpp/e2etest/quirks#broken-pubsub-get-multiple-by-id¶
Indicates that the “Get Items by Id” operation in the PubSub service used is broken when more than one item is requested.
Helper functions¶
- async aioxmpp.e2etest.provision.discover_server_features(disco, peer, recurse_into_items=True, blockmap={})[source]¶
Use XEP-0030 service discovery to discover features supported by the server.
- Parameters:
disco (
aioxmpp.DiscoClient
) – Service discovery client which can query the peer server.peer (
JID
) – The JID of the server to queryrecurse_into_items – If set to true, the XEP-0030 items exposed by the server will also be queried for their features. Only one level of recursion is performed.
- Returns:
A mapping which maps XEP-0030 feature vars to the JIDs at which the service is provided.
This uses XEP-0030 service discovery to obtain a set of features supported at peer. The set of features is returned as a mapping which maps the
var
values of the features to the JID at which they were discovered.If recurse_into_items is true, a XEP-0030 items query is run against peer. For each JID discovered that way,
discover_server_features()
is re-invoked (with recurse_into_items set to false). The resulting mappings are merged with the mapping obtained from querying the features of peer (existing entries are not overridden – so peer takes precedence).
- aioxmpp.e2etest.provision.configure_tls_config(section)[source]¶
Generate keyword arguments for use with
security_layer.make()
from the configuration which control the TLS behaviour of the security layer.- Parameters:
section – Configuration section to work on.
- Returns:
Keyword arguments for
security_layer.make()
- Return type:
dict
The generated keyword arguments are
pin_type
,pin_store
andno_verify
. The options in the config file have the same names and the semantics are the following:pin_store
andpin_type
can be used to configure certificate pinning, in case the server you want to test against does not have a certificate which passes the default OpenSSL PKIX tests.If set,
pin_store
must point to a JSON file, which consists of a single object mapping host names to arrays of strings containing the base64 representation of what is being pinned. This is determined bypin_type
, which can be0
for Public Key pinning and1
for Certificate pinning.There is also the
no_verify
option, which, if set to true, will disable certificate verification altogether. This does not much harm if you are testing against localhost anyways and saves the configuration nuisance for certificate pinning.no_verfiy
takes precedence overpin_store
andpin_type
.
- aioxmpp.e2etest.provision.configure_quirks(section)[source]¶
Generate a set of
Quirk
enum members from the given configuration section.- Parameters:
section – Configuration section to work on.
- Returns:
Set of
Quirk
members
This parses the configuration key
quirks
as a python literal (seeast.literal_eval()
). It expects a list of strings as a result.The strings are interpreted as
Quirk
enum values. If a string starts with#
, it is prefixed withhttps://zombofant.net/xmlns/aioxmpp/e2etest/quirks
for easier manual writing of the configuration. SeeQuirk
for the currently defined quirks.