=================================
Tutorial for usage of pyars.erars
=================================

This tutorial covers the more convenient function wrappers for the
ARSystem API that you can access using pythonic data structures (as opposed
to the low level C API functions, which are covered in the `original tutorial <tutorial-ar.html>`_).

The ARSystem API offers many different function calls to talk to
the ARSystem server. The pyars.ars module makes them available to Python scripts
(e.g. ARGetListSchema) -- you have to use the Remedy data structures. When
migration from an existing C API program, this may be the most appropriate
way.

The pyars.erars module offers its own version of
each API function (without the AR at the beginning, e.g. GetListSchema) that
you can use with pythonic arguments (lists and tuples instead of
C based data structures).

So where is the main difference, you ask? You can either

- import pyars.ars and use the C level functions (beginning with AR, like
  ARGetEntry) using low level, C data structures

- import pyars.erars and use the more pythonic versions of the wrappers
  (which all have the exact same name as the C level functions, without the 
  AR prefix, like GetEntry)

Having said this, so far mainly the \*Entry functions have seen a much nicer
interface due to the pythonic conversion layer, where you can now use
dictionaries for retrieving entries, setting entries or retrieving many
entries at once etc. Also, handing over arguments is now easier, as many
functions will convert python data structures into the appropriate C
data structures. Looking at `the generated documentation of 
erARS <pyars.erars.html>`_ 
should give you an indication of what you need to handover.

There is also a module called ercmdb which follows the same
guidelines and tries to make the CMDB functions better accesible. However,
as my project currently has no need for this and the user community has not
cried out for this, it's currently a sleeping beauty.

Retrieving a single entry
#########################

The most often used use case is probably working with entries. Let's see,
how  we can retrieve, modify and create entries... We start with retrieving
the entry with id '000000000000001' from Form User. If this does not
exist on your server, please modify this example accordingly to match your
environment.

::

    from pyars import erars, cars
    ars = erars.erARS()
    ars.Login('server', 'user', 'password')
    # error handling is left as an exercise to the reader :-)
    entry = ars.GetEntry('User', '000000000000001')

'entry' now contains a pythonic dictionary with the fieldids as keys, and
the values as value: {field1: 'string', field2: 236, ...}. In our example,
we would like to print out all fields::

    for fieldid in entry.keys():
        print '%d: %s' % (fieldid, entry[fieldid])

Or if you would like to limit the output to the character fields:
::

    for fieldid in entry.keys():
        if isinstance(entry[fieldid], str):
            print '%d: %s' % (fieldid, entry[fieldid])

Modifying an entry
##################

Now let's modify this entry (reusing the entryId from the example above)
by changing the contents of the field
Full Name (that's the name of field 8 on the form User, typically)::

    newFieldValue = { 8 : 'new user name', 103 : 'new_email@domain.com'}
    res = ars.SetEntry('User', entryId, newFieldValue)
    if ars.errnr > 1:
        print 'ERROR modifying entry!'

Of course, the dictionary newFieldValue can take on more than just one
field with a new value!

If you compare those two examples with the `C level examples <tutorial-ar.html>`_, you will see that using the more pythonic
functions reduced the amount of glue code dramatically and makes for
a much better readable and maintainable code.

Creating entries
################

Before you can retrieve or modify entries, you will have to create them.
::

    fieldValue = { 101 : 'testuser',
        8 : 'New User',
        103: 'my@email.com',
        7 : 0, # status: Active
        109 : 2, # floating license
        110 : 0 # no fts license
    }
    entryId = ars.CreateEntry('User', fieldValue)
    if ars.errnr > 1:
        print 'ERROR modifying entry!'

If this did not work out, see below the section for error handling -- maybe
your user form is customized and requires some additional fields?

Delete an entry
###############

Then, finally, if you want to delete an entry::

    result = ars.DeleteEntry('User', entryId)

Working with field names instead of field ids
#############################################

As you can see in the example above, the code is not utterly readable.
As python is very much about readability, this needs to be changed. There is
this useful function called GetFieldTable (the idea is taken over from ARSPerl),
that gets you another dictionary of fieldnames : fieldids::

    userFields = ars.GetFieldTable('User')

If you look at the resulting dictionary userFields, you will see something
like::

    >>> userFields
    {'Application License': 122L, 'Force Password Change On Login': 124L,
    'Status': 7L, 'Computed Grp List': 119L...}

With this information, we can rewrite the last example to::

    userFields = ars.GetFieldTable('User')
    fieldValue = {
        userFields['Login Name']       : 'testuser',
        userFields['Full Name']        : 'New User',
        userFields['Email Address']    : 'my@email.com',
        userFields['Status']           : 0,          # status: Active
        userFields['License Type']     : 2,          # floating license
        userFields['Full Text License Type'] : 0     # no fts license
    }
    entryId = ars.CreateEntry('User', fieldValue)
    if ars.errnr > 1:
        print 'ERROR modifying entry!'

Retrieve a list of entries
##########################

If you don't have the entry ids, you can search for entries (GetListEntry) or
even get the data for the entries returned immediately (GetListEntryWithFields).
Let's concentrate on the second and retrieve the values for the fieldids 1, 5
and 8 for all entries in the form 'User'::

    # the following is the list of fields we will retrieve for every result entry
    fields = (1,5,8)
    (entries, numMatches) = ars.GetListEntryWithFields(form,
                                                       None,
                                                       fields)

entries now contains a tuple, consisting of a dictionary and
and integer showing the number of found entries that match the query on the form.
On my server, entering "entries" and hitting return prints out the complete
dictionary with 500 entries, therefore let's first::

    >>> numMatches
    500L
    >>> entries['000000000000002']
    {8L: 'Axel Seibert', 1L: '000000000000002', 5L: 'Demo'}
    >>> entries.keys()
    ['000000000020462', '000000000023613', '000000000020393', '000000000020099', '000000000020392', ...]
    >>> entries['000000000000003'].keys()
    [8L, 1L, 5L]

This was interactive, but of course, it's very easy to go through all
the elements::

    for entry in entries:
        print '''User: %s with entryId: %s''' % (entries[entry][8],
                                                 entries[entry][1])

Now, let's change this to use the fieldnames instead of the fieldids::

    for entry in entries:
        print '''User: %s with entryId: %s''' % (entries[entry][userFields['Full Name']],
                                                 entries[entry][userFields['Request ID']])

And again, we don't need to remember to free the memory, as erARS has already
done so.

Please note: This example leaves the error handling as an exercise to the reader;
typically,
you should test ars.errnr after each API call for values greater than 1 (errors)
and possibly 1 (warning). Only the error number 0 signals a successful call.

Retrieving a certain subset of entries
######################################

When we used GetListEntry in the last example, the second parameter was
set to None. This is a query that you can pass on to limit the results::

    entries = ars.GetListEntryWithFields(form,
                        "'Email Address' LIKE \"%@google.com\"",
                        fields)

The syntax is the same as in the extended search bar (in other words, field names
need to be encapsulated with single quotation marks), just make sure to escape
the quotation marks within the string correctly.

Retrieving an attachment into a buffer
######################################

This is also quite easy, but might be an exercise for the
future to also offer a more pythonic version for the ARLocStruct...
::

    # prepare a structure for the attachment, specify option BUFFER
    loc = cars.ARLocStruct(cars.AR_LOC_BUFFER)
    att = ars.GetEntryBLOB (form, 'yourentryid', 600010600, loc)
    if att.u.buf.bufSize > 0:
        # assuming a plain text file
        print string_at(att.u.buf.buffer)

Retrieving an attachment as a file
##################################

::

    # prepare a structure for the attachment, specify option BUFFER
    loc = cars.ARLocStruct(cars.AR_LOC_FILENAME)
    loc.u.filename = r'c:\temp\attachment.txt'
    att = ars.GetEntryBLOB (form, 'yourentryid', fieldid, loc)

And the file is created in the filesystem automatically for you. You wonder,
what the 'r' is in front of the file name? It's the python convention for
raw strings, in other words, python will ignore the '\', as it normally
indicates the beginning of a special character.

Uploading an attachment as a file
#################################

Starting with release 1.2.5 of pyars, uploading an attachment is
especially easy, when you have it as
a file (other usecases are currently not supported by erars).
::

    # prepare a structure for the attachment, specify option BUFFER
    result = ars.SetEntry(form, 'yourentryid',
    {fieldId: (fileName, 0, 0, fileName)})

erars will recognize when you hand over a 4-tuple as a special use case
and assume that you define an attachment with (attachmentName, origSize,
compressedSize, fileName). Then, the file is uploaded from the filesystem
automatically for you.

Get a list of schemas from an ARSystem
######################################

In this part we will retrieve the list of schemas from the ARSystem:
::

    listSchema = ars.GetListSchema()   # fetch a list of all schema names
    if ars.errnr < 2:
        for i in range(len(listSchema)):
            print listSchema[i]
            schema = ars.GetSchema(listSchema[i]) # now fetch the detail information for this schema
            # schema is one of the convenience structures...
            # print out the help text for this schema:
            print schema.helpText
            ars.ARFree(schema) # free the memory
    else:
        print "error retrieving list of schemas!"

As there is no nice structure defined by Remedy/BMC that
would hold all relevant schema information (as retrieved by ARGetSchema) we
came up with our own structure::

    class ARSchema(Structure):
    _fields_ =  [("name", cars.ARNameType),
                 ("schema", cars.ARCompoundSchema),
                 ("schemaInheritanceList", cars.ARSchemaInheritanceList),
                 ("groupList", cars.ARPermissionList),
                 ("admingrpList", cars.ARInternalIdList),
                 ("getListFields",  cars.AREntryListFieldList),
                 ("sortList", cars.ARSortList),
                 ("indexList", cars.ARIndexList),
                 ("archiveInfo", cars.ARArchiveInfoStruct),
                 ("defaultVui", cars.ARNameType),
                 ("helpText", c_char_p),
                 ("timestamp", cars.ARTimestamp),
                 ("owner", cars.ARAccessNameType),
                 ("lastChanged", cars.ARAccessNameType),
                 ("changeDiary", c_char_p),
                 ("objPropList", cars.ARPropList)]

So, in a sense, those additional structures allow us to be a litte bit more
pythonic than with the usual data structures.

Please note: looking back at this, with erARS in the wild, this
now seems like a bad decision to implement those helper data structures
already in pyars.ars. They really should have gone into pyars.erars. Maybe
I will correct this in a later release, as it will break backwards
compatibility

To demonstrate how you can use the lookup dictionary
(assuming, you set up
a connection with your ARS)::

    from pyars import cars
    field = ars.GetField('User', 7)
    print 'field %s(%d) of type %s' % (field.fieldName,
                      field.fieldId,
                      cars.ars_const['AR_DATA_TYPE'][field.dataType])
    ars.ARFree(field)

In the file cars.py you will find the definition of this dictionary. So if
you want to print out something, go in there and search for the constant.

At the end of your session, it's always a good idea to logoff::

    ars.Logoff()

Python makes it easy to log off from the system in any case::

    try:
        ... # some code
    finally:
        ars.Logoff()

Error handling
##############

If something goes wrong in one of the examples above, you typically
see the following output in your console::

    2009-08-18 18:35:06,187 pyars.ars DEBUG enter ARCreateEntry...
    2009-08-18 18:35:06,375 pyars.ars ERROR ARCreateEntry for schema User failed

To make sure you catch errors in the API calls, you can check the ars.errnr
after each api call::

    if ars.errnr > 1:
        ars.logger.error('something went wrong with this api call!')

Maybe you did not fill in all required fields? The Remedy server offers
the status message, that most often explains why an API call did
not work. In order to give you easier access to the status messages, I created a
function called statusText() that will return the strings of the message::

    ars.statusText()

For example, this will print out the following error::

    'Status:
    ERROR (307): Required field (without a default) not specified
    600010025
    ERROR (307): Required field (....'

Maybe you do not see the line break but just a concatenated single liner...

A more detailed look at retrieving an entry
###########################################

::

    from pyars import erars

If you run into problems at this line, pyars cannot find the Remedy libraries to
connect to your Remedy server. On windows, that means, that you don't have a
user tool or admin tool installed or strange entries in the registry.
Make sure that the path to your libraries is contained in the system path.
::

    ars = erars.erARS() # instantiate a session object
    connection = ars.Login('server', 'user', 'password') # store the connection

There are different API calls to retrieve
form entries from an ARsystem, let's start with ARGetEntry, and let's fetch
the 'User' entry with entryid 1::

    form = 'User' # this is the form
    entryid = 1 # this is the entryid that we would like to retrieve
    entry = ars.GetEntry(form, entryid)

Here you can see the difference to the low level C data structures (compare
that with the `C level function call <tutorial-ar.html>`_!).
And you do not need to worry about the request-id: erARS will try to convert
it for you, so in this case, even the integer 1 is enough. This of course
is not always the case, but you could as well call with a string::

    entry = ars.GetEntry(form, '1')

or with the correct request-id::

    entry = ars.GetEntry(form, '000000000000001')

Now, that we have received something from the server, let's have a look
at that piece of data: When you enter::

    entry

you will see something along the following lines::

    >>> {128L: None, 1L: '000000000000001', 2L: 'Demo1', 3L: 1172650162,
    4L: None, 5L: '....}

So, erARS has converted the C level data structures into a pythonic
dictionary where you can use the keys() functions to get all
fieldids and then use those to retrieve the values for each fieldid
extremely fast. Or you can use the len() function to find out
how many values you have been returned::

    len(entry)

Now the old tutorial guides you through some helper functions that
erARS already brings to the table, so we can skip all this and
directly use the Python dictionary::

    print entry[1]

And finally, we don't need to free the memory any more, as erARS has
taken care of this.

migrating from ARSPerl
######################

The following functions are defined in ARSPerl, and have been
implemented in pyARS as well to ease migration:

- GetFieldByName: is available as erars.GetFieldByName()

- GetFieldTable: is available as erars.GetFieldTable()

- APIVersion(): This routine is available as erars.APIVersion() and
  returns the "major" version of the API that
  pyARS has found on your system and that it will use. As pyARS is not compiled
  against a specific API version, it's impossible to implement exactly
  the same behaviour.

- decodeStatusHistory: is available as erars.DecodeStatusHistory()

- EncodeDiary: is available as erars.EncodeDiary()

- GetControlStructFields: is available as ars.ARGetControlStructFields() or
  erars.GetControlStructFields() (without any differences)

- GetCurrentServer: is available as ars.ARGetCurrentServer() or
  erars.GetCurrentServer() (without any differences)

- GetProfileInfo: seems to be similar to erars.GetServerStatistics().
  If you know of any differences, please let me know.

- padEntryid: is available as ars.padEntryid()

- errstr: is available as erars.errstr()

- perl_qualifier: currently not supported; if you need this, let
  me know your use case, and either we find a workaround or we will try to
  implement this

- simpleMenu: currently not supported

How to proceed
##############

Generally speaking, you need to follow those steps:

- Identify which API function you want to call

- Identify which data structures this API call expects

- Prepare those data structures

- Call the API function

- Identify which data structures this API call returns; have a look at the
  examples in ars_test.py and cmdb_test.py, respectively. You will
  find many helpful code snippets in there.

- Have a look at the unit tests (ars_test.py and cmdb_test.py) which
  provide many useful examples.

- Give us feedback, possibly submit code, patches.