Using a Database Driven Design with IPA

The Database Driven Design is a consideration for every process you automate with IPA. That doesn't mean every process will require a Database Driven Design, but I suspect most will if they're of any size, complexity and/or duration (start to finish elapsed time for the process).

As with many of the Best Practices I'll post here, the concept of a Database Driven Design for your IPA Processes most definitely is not mine. I'm not the creator, merely the conveyor. The concept originated with a couple of guys deploying, maintaining and ever-enhancing a large, highly complex internal process that also has customer facing components at a software vendor with which I am vaguely familiar. I'll simply refer to them as the PRs, short for Punk Rockers. Anything else I call'em would probably too easily identify them and where they're employed, and they're plenty busy as it is.

Much thanks to these gents and their 3rd partner in crime for their willingness to share their lessons learned, which are the foundation of the evolving Best Practices for IPA. Without their insight, the IPA community would collectively have to stumble across applying these concepts the hard way. Thankfully, they've already applied considerable thought to some common problems they already encountered, so we won't have to. We'll find our own challenges along the way, of course...!

The PRs have been working with IPA coming on a couple of years now, and I'll go out on a limb to say they're probably the most experienced with deploying IPA. As a result, mostly over time and through trial and error, they've developed many of the Best Practices that I'll explain in detail as I'm applying them to my own IPA deployments.

So one of the first things I was told by these Jedi Masters of IPA was "Use a database driven design, you will." I nodded slowly as I wrote this down in my trusty old school black cover composition notebook. "Yeeeesss... a database driven design," I mumbled as I double-underlined the words before finally looking up and saying "um, what the heck does that mean exactly?"

"Well," one of them began, "there are several reasons to use a Database Driven Design in IPA..." and a fury of note-taking and questions ensued for the remainder of our session. The concept is as simple as it is powerful, and I'll use my Finance Purchase Order (PO) Request to illustrate.

The first point is having a custom unique identifier for your entire process instance, including as it calls other processes and external systems. The CIC system and IPA will assign a unique process identifier, which we'll call processid or pid for short. For the CIC-initiated, processid is like the callid for the process, a unique, system-generated and assigned number for an IPA process. This is very helpful but it's not really what will benefit us the most as a unique identifier for our overall process because we might call additional processes from our main process.

Here's an overview of the Database Driven Design concept:

1. Each time when a new process instance is launched, it will first go out to a custom database for that process to pull a new unique, Custom Process Identifier (or cpid for short). The cpid will be used throughout any activities associated with this process instance.
2. The process will then immediately update that record for the cpid in the custom database with initial pertinent information, such as the requesting user and initial State. 
3. As the process moves through it's various States, Task, Work Items and any associated Actions, there will be frequent updates to the database for the custom processid for the process.
4. This means, at any time, we or our process can look in the custom database and see where all of our processes are State-wise and also any data associated with it.

Clear as mud, right? Let's move to the example and dive right into the custom database.

The PO Request will have a custom database, or custom db, for storing all of the required process information associated with each individual process run, such as the management approval chain that will be passed Work Items to Approve or Reject the request until the proper approval level has been reached. The custom db will also store dynamic business rules that are created along the way, which will be covered in another post.

Concerning the PO Request, my first Work Item when someone launches the PO Request will give the Requester the opportunity to create a new PO Request, or simply click the link to the Sharepoint page we're setting up for status and reporting purposes. Notice I didn't say anything about hitting the database to assign the cpid yet? The purpose of this first Work Item is to give them a chance to see the status link if they haven't bookmarked it, so we don't burn through PO Request IDs needlessly.

If the Requester has launched the PO Request process to actually request a PO, and if they click Next instead of Cancel, that's the first time we'll hit the database to assign the cpid, which in this case would be referred to as the PO Request ID. The name of the cpid will be determined by the type of process you're creating and language of the business that will use it.

Benefits of a Database Driven Design in IPA:

• Use fewer Global Variables in IPA - leaving more system resources available to CIC. It's always good software development technique to use Global Variables sparingly, regardless of the system or application. With a custom database and a unique custom process id for each instance of your processes, basically all you need to pass around is the cpid, which also means fewer changes to functional calls since the parameters passed will tend to be fewer with adds going into the database instead.
• Re-starter Processes after Re-Publishing - so that active processes can pick up the changes. I'll cover the PRs concept of a Re-starter Process in depth in a later post, the whole goal of which is to allow you to re-publish your process and re-start all of them from scratch. Much like a handler, a process instance lives until it's complete, which is fine for phone calls that usually last no more than an hour or so, but a process instance might last up to weeks or months. Make a change to the process and you can Re-start all of your processes to pick up the changes.
• Custom, Detailed Real-Time and Historic Reporting - that is, if you make data available via any kind of pulling mechanism, such as Sharepoint page or a custom report. We'll cover custom reporting in an upcoming post. One point to consider for now is the out-of-the-box reports for IPA will contain State Labels, outcomes and other info along those lines, but not many deeper details of the processes and their custom data.

Considering Reporting, think of CIC and calls. We can use call detail and Distribution Queue Reports reports to get volume information, IVR reports (assuming we've enabled them and used meaningful labels in the Attendant Profiles...), Recorder and iFeedback. The IPA Standard reports will cover basically the equivalents to call detail and Distribution Queue Reports. Knowing more about what's happening inside of your process requires custom logging and therefore a custom database to get the equivalent data in IPA that roughly correlates to IVR Reporting, a recorded call and iFeedback for calls.

Now one important question to ask might be when wouldn't I use a Database Driven Design? I'm pressed to think of any to be honest. The process would have to be short lived and infrequently used to avoid spending a little time up front to use a custom database. There might be a customer application that does not want a detailed record of in-process data, even though they risk losing a process instance if the system is taken offline before a process completes. I'll update this post if any concrete examples arise.

The IPA Design and Implementation Certification course that I completed at ININ did not use a Database Driven Design for any of the labs, but that's largely because the concept didn't exist when the material was written. The course is going to be re-written and hopefully we can get some of these Best Practices worked into it.

Storage is cheap these days and from a system resource perspective on the CIC server, it's more cost-efficient to have a few more database calls than to store all the data for a process as global variables, especially if there's a lot of data and a lot of concurrent process instances running. I'd expect all of the above to be typical.

For any business making the effort to use IPA to automate a process, they should strongly consider the Database Driven Design! Once again many thanks to the PRs for their valuable input an insight into this IPA Best Practice.

The concept of a Database Driven Design with IPA is a breeze to illustrate or talk through pretty easily but as you can see to write it out is more involved. Comments are especially welcome if there are areas in the article that can be enhanced for clarity.

Happy Processing!