pyARS - from python to ARS

details of the pyARS implementation

Unless you want to get involved with the development and bug fixing for pyARS, I guess, there is no need to read behind this point...

the layers

There are five layers (from bottom to top):

  • ARS C API, accessed via Python and ctypes (a python module)
  • _carsxx module (wrapper for all C struct declarations, depending on the Remedy version)
  • cars and ccmdb module (loading the appropriate version of _carsxx, version independent)
  • ars and cmdb module (exposing the low level API functions of ARSystem and Atrium, repectively, using the C structures)
  • erars and ercmdb module (exposing a more pythonic API, using Python data structures and offering conversion methods)

ARSystem C API

The ARSystem C API is the library as provided by Remedy (now, BMC) - the .dll or .so library accessed via the Python module “ctypes”. The ctypes module allows the Python interpreter to load the DLLs and call the functions in the DLLS.

structures and definitions

_carsxx.py (“Constants for ARS” for different versions) and _ccmdbapixx.py/ _carosapi63.py (“Constants for CMDB”) contains all #defines and data structures defined for ARSystem. These files are automatically generated from .h files as supplied by BMC and depend on the version – therefore you will find e.g. _cars63.py as well as _cars71.py.

cars.py tries to load the ARSystem DLL; depending on the version it finds, it will load the appropriate _carsxx.py module (the following explanation also holds true for the CMDB code, therefore I will not always say: ars and cmdb): It has a built in list of DLLs corresponding to the different versions of the ARS API (e.g. 7.1 vs. 6.3 vs. 5.1), that it will try to load (beginning with the highest version). Depending on which ARS DLL it could load, it will load the appropriate _cars51.py, _cars60.py, _cars63.py,... file. In other words, with this mechanism, the client API version determines, which protocol version the pyars package talks to the server.

cars.py uses the system path to locate the ARSystem dlls. Unfortunately, Remedy does not update the system path to include the directory where it stores its dlls. Therefore, on windows, cars looks up the windows registry for the installation path of aruser and will append this directory to the systempath. So the version of the installed user tool will determine the API version cars.py will offer you.

If you want to use another dll, simply copy the according dll files into a separate directory, put this directory at the beginning of your system path and start python with those new settings. cars should then find the dlls in this directory before any other directory and use this version.

You should never need to import any file with a name beginning with the underscore “_” (e.g. _cars70.py). If you need to access the structures defined in those files, you do:

from pyars import cars

which will fetch the right version of _carsxx.py based on the library that it finds first in your system path.

structure conversion and usage

The Remedy data structures have been completely ported to python. Originally, this used to be a highly manual process resulting in nicely formatted and highly readable python code, e.g.:

class ARFieldInfoStruct(Structure):
_fields_ = \[("fieldId", ARInternalId),
("fieldName", ARNameType),
("timestamp", ARTimestamp),
("fieldMap", ARFieldMappingStruct),
("dataType", c_uint),
("option", c_uint),
("createMode", c_uint),
("defaultVal", ARValueStruct),
("permList", ARPermissionList),
("limit", ARFieldLimitStruct),
("dInstanceList", ARDisplayInstanceList),
("owner", ARAccessNameType),
("lastChanged", ARAccessNameType),
("helpText", c_char_p),
("changeDiary", c_char_p)]

The list defines all member variables with their names and types. So, if you have a variable that contains this ARFieldInfoStruct, you can easily access all members of that struct in the usual way:

variable.fieldId

Or, in more complex scenarios, it allows you to follow the deeply nested Remedy structures.

Very often, you will find the following combination of structs and according lists:

class ARPermissionStruct(Structure):
_fields_ = \[("groupId", ARInternalId),
("permissions", c_uint)]
class ARPermissionList(Structure):
_fields_ = \[("numItems", c_uint),
("permissionList", POINTER(ARPermissionStruct))]
class ARPermissionListList(Structure):
_fields_ = \[("numItems", c_uint),
("permissionList", POINTER(ARPermissionList))]

It is very easy to iterate over such a list (see also example above):

for i in range(arpermissionList.numItems):
print 'line #%d: groupId: %d has permissions: %d' % (
i,
arpermissionList.permissionList[i].groupId,
arpermissionList.permissionList[i].permissions)

file structure

Here is a list of files and what they do:

  • directory docs: contains documentation about the pyars module.
  • _carosapi63.py: contains the structures and definitions for accessing the Atrium CMDB in version 1.0 and 1.1 patch 001
  • _cars51.py, _cars60.py, etc.: contains the structures and definitions for accessing any Remedy server version 5.1 and higher
  • _ccmdbapi20.py: contains the structures and definitions for accessing the Atrium CMDB in 2.0 and higher
  • _ccmdbapi63.py: contains the structures and definitions for accessing the Atrium CMDB in version 1.1 patch 002
  • ars_test.py: contains unit tests of the pyars.ars module
  • cars.py: This file is imported by ars and loads the ARSystem library. Depending on the library version that it finds, it then loads the according _carsxx file that contains the coresponding structure definitions.
  • ars.py: the main working horse for the pyars module: it contains low level wrappers for all functions (AR...) that the ARSystem API provides.
  • erars.py: building on top of the low level API functions, this module exposes the higher, more pythonic versions of the functions. Each low level function (e.g. ARGetSchema) has a higher level equivalent without the prefix AR (e.g. GetSchema).
  • ccmdb.py: This file is imported by cmdb and loads the Atrium CMDB library. Depending on the library version that it finds, it then loads the according _carosapixx or _ccmdbapixx file that contains the coresponding structure definitions.
  • cmdb_test.py: contains unit tests of the pyars.cmdb module definitions.
  • cmdb.py: it contains wrappers for all functions that the Atrium CMDB API provides.

ars.py and cmdb.py

The pyars module is mainly split into two parts: ars.py for all ARSystem API calls and cmdb.py for all Atrium CMDB calls (the other files are support files).

ars.py defines a base class that holds all function definitions for ARS V5.1.2. It then defines a subclass ARS60 (and ARS63 and ...) that inherits everything from the class before and implements the difference between 5.1 and 6.0 (between 6.0 and 6.3, respectively). A third class definition finally (ARS) maps to one of the two classes (pyars relies on the version information in cars, see above).

The same holds true for cmdb.py, although it’s a bit more complicated: Let’s start with CMDB Version 1.1 patch 002 and later: cmdb.py defines a base class that holds all function definitions for V1.1, CMDB11. It then defines a subclass CMDB20 that inherits everything from the class before and implements the difference between 1.1 and 2.0. A third class definition finally (CMDB) maps to one of the two classes (cmdb relies on the version information in ccmdb, see above). The reason, why it’s a bit more complicated is, because BMC changed the naming convention mid stream. In Version 1.0 and 1.1 before patch 002, all Atrium API function names started with AROS instead of CMDB. In cmdb.py you will find a class AROS10. To make your life easier, we provide another wrapper around that which provides a CMDB10 class with a coresponding CMDB naming convention for all functions (in other words, your scripts don’t have to worry if they work against a CMDB 1.0 or CMDB 2.0, because the CMDB wrapper allows you to use CMDB functions regardless of the CMDB version).

ars.py is the main building block which exposes the interface to ARS via a class called ARS that will handle all the session handling for you (see tutorial). In addition, it defines a couple of comfort structures, that the API does not define (e.g. ARMenuStruct or ARFilterStruct), that do nothing but collect all relevant information in one structure (especially useful as return values in the GetMultiple... calls).

convenience structures

In addition to the original Remedy data structures, I have defined some useful helper structures:

  • ARActiveLinkStruct
  • ARActiveLinkList
  • ARContainerStruct
  • AREscalationStruct
  • ARFilterStruct
  • ARFilterList
  • ARMenuStruct
  • ARSchema
  • ARSchemaList

As those helper structures proofed rather helpful, I also introduced similar classes for CMDB:

  • CMDBAttributeStruct
  • CMDBAttributeList
  • CMDBClassStruct
  • CMDBClassList

unit tests

We supply unit tests for pyars.ars, pyars.erars and pyars.cmdb. You can run those test suites against your ARSystem or CMDB, by issuing:

ars_test.py -x <server> -u <user> -p <password>

or:

cmdb_test.py -x <server> -u <user> -p <password>

where server can be given as servername:portname. All you should see is for ars_test.py:

D:\\privat\\Ergo\\code\\pyars\\pyars>python ars_test.py -x <server>[:port] -u <user> -p <password>
..................................................................................
----------------------------------------------------------------------
Ran 82 tests in 4.292s

For cmdb_test.py it should look similar. And with the option -v you get verbose (debugging) output.

If you mix the versions of client libraries and server, you might crash the server. Therefore, we implemented a test to make sure that both versions match at the beginning of those tests. However, if you know what you are doing you can override this behaviour with the -f (force) option.

The User tool typically does not install the xml libraries alongside the arapi.dll. Therefore we moved the xml tests into its own test file: arxml_test.py. If you run into problems with the XML api calls, please check that you have the necessary libraries in your PATH. The arxml_test.py gives you according hints.