Logging

Each stage within an Object can be set to one of the following levels:

• Enabled - the stage name and actions will be logged
• Disabled - no information about the stage's activity will be logged
• Errors Only - no information will be logged unless the actions produce an error condition

To quickly set all stages within an Object to be Disabled you can open the Object and choose the option Edit -> All Stages -> Disabled. This will set logging for all stages within all pages within the Object to be disabled.

Each stage within a Process can be set to one of the following levels:

• Enabled - the stage name and actions will be logged
• Disabled - no information about the stage's activity will be logged
• Errors Only - no information will be logged unless the actions produce an error condition
• Don't log parameters on this stage - toggles logging of data parameters used by the stage on or off (e.g. business data)

Stage logging can be easily switched on, off or to errors only in the same way as with an Object, i.e. using Edit -> All Stages.

Note: It is not possible to disable parameter logging for all stages using this method. This setting must be set on each stage individually. It is therefore very important that each stage's parameter data implications should be assessed before any Process goes into Production where there is the potential for recording business data in the Blue Prism logging tables.

It is possible to override the logging in a process or object and set the logging at machine level, where each Runtime Resource can have its own logging level defined. This is achieved using the Resources -> Management screen, where a context menu option is provided to set Runtime Resource Logging Level options.

The basic logging level options are:

• Default - Logs any stages have Stage Logging set to 'Enabled' in the Process or Object which the Runtime Resource is running. 
• All Stages - Logs all stages regardless of which stages specifically have Stage Logging set to 'Enabled'. Note this may slow down operation of any Processes run by the Runtime Resource and will result in fuller logs. However this may be useful in diagnosing problems.
• Key Stages - Overrides only the following stages:
  • Action, Code, Navigate, Process, Read, Write, WaitStart - regardless of which stages specifically have Stage Logging set to 'Enabled'.

Using the same context menu, it is possible to record the memory levels of a machine in the session logs.

• Log Memory Usage - if enabled, all process log entries are accompanied by a record of how much memory (Working Set) was in use by the main Blue Prism executable at the time. Additionally, if the logged action refers to a target application, the identity and memory usage of that target application is also recorded.
• Include Memory Cleanup - enabling this may produce more useful figures when diagnosing suspected memory leak issues.

Web Service logging

Web service details can also be added to the logs via the context menu.

Log Web Service Communication - enabling this causes low-level web service communication to be logged - specifically, SOAP messages are logged in full detail. For consumed web services, the information is logged as part of the corresponding session log. For exposed web services, it is logged in the event log for the relevant Resource PC.

Note: If a Runtime Resource is online when its diagnostics settings are changed, it does not need to be restarted for the new settings to take effect. It will pick up and begin using the modified settings within two minutes at the most.

Unicode Logging

In Blue Prism, errors are logged in Non-Unicode format by default. Unicode logging can be enabled using the System Settings with the following option:

Note: It is generally not recommended to use Unicode, however, there are some instances where it will be required (i.e. logging text containing non-standard characters like logographic or symbols).

Whenever a Process runs, it makes a record of each step it takes to create a Session Log. You can access this log from from System > Audit section or Control Room:

• Select one of the Session rows in the Environment list in Control Room.
• Right-click to open the mouse menu and select View Log.

The Log Viewer is used to inspect the log of a session, either as it is running or after it has finished. This is especially handy for reviewing the workings of a progress and tracking down problems. The Log Viewer has a search function enabling you to look through the (often numerous) rows of a log and the visibility of columns can be configured to suit.

Logs can be displayed in Grid or List view type. The Search function helps to find and highlight keywords and text analysing all or only specific columns.

• Logging for stages within an Object should only be enabled during the development phases of a Blue Prism implementation. Logging Objects is not recommended for Production systems.
• Logging for stages within a Process can be enabled to whatever level is required to support debugging and tracing during Development phases, or for auditing and reporting purposes when the Processes are deployed in a Production environment. The level required will be determined by Blue Prism developers in Development and Testing phases, and by the business requirements in Production.
• Often a usable audit trail can be created by turning on logging only for the following stages within a Process:
  • All decision stages
  • All choice stages
  • All Errors
  • Work Queue – Get Next Item
• Turn off logging anywhere that might generate large amounts of log data. Look for:
  • Loops which use large data sets
  • Stages with lots of input or output parameters
• Some Data Item data types logged with parameter logging will consume larger amounts of disk space:
  • Collections - especially nested Collections o Markup for formatted text - for example, JSON data, decoded HTML web pages, or large XML strings
  • Text – large strings of more than 4000 characters in one Data Item
  • Images - pictures uploaded into Data Items. Lossless image formats will store more data.
  • Binary - strings of binary data
• Avoid running processes 24x7 and instead configure them to stop at least once every 24 hour period. This will avoid creating a huge single log that later cannot be archived by normal means. There are very few business requirements that cannot afford a few minutes of downtime each day, and if absolutely necessary a process could be designed so that each resource took its ‘rest’ at a different time.

Care should be taken during design and development to ensure that data protection policies (i.e. PCI compliance) are kept to by the Blue Prism solution. Logging should only be turned on to the minimum levels required to support the process and trace case routes and outcomes. For example, case IDs or customer IDs may be logged, but personal client information should never be logged.

Blue Prism recommends reducing the logged stages to a minimum level and to ensure that no client data is logged as part of that.

One method of ensuring this is to check that the option "Don't log parameters on this stage" is checked for all stages which may contain sensitive data.

When making the move from a Development or Test environments to one of User Acceptance Testing (UAT), PreProduction or Production environments then there is a definite and necessary change required in the logging levels for the Processes and Objects.

Blue Prism recommendation for moving into UAT, Pre-Prod or Production is to ensure the following:

• All stages within all Objects should be set to DISABLED, or ERRORS ONLY (if necessary)
• All stages within all Processes should be set to DISABLED or ERRORS ONLY, except for
  • Work Queue item 'Get Next Item' actions, if the queue item is not holding sensitive data.
  • Enable logging for all Decision and Choice stages within the Process
    • Logging for all Decision and Choice stages could be required only on the Main page if the process is based on a BP templates. Only Errors logging should be sufficient for all the other pages.
• All stages within all Processes should have parameter logging set to DISABLED
• Runtime Resources running Production processes should be set to log DEFAULT or KEY STAGES only
• It’s common practice to reduce production logging over time, e.g. on the first day of go-live logging levels are left high to facilitate audit of the ‘soft launch’ period. As the process settles down the logging is decreased, e.g. week 1 at 100%, week 2 at 50% etc

• Periodically run a ‘health check’ on the production environment. Logging settings might change after the implementation of a CR and then inadvertently left enabled at high level.

Blue Prism stores log details on the database in the [BPASessionLog_NonUnicode] or [BPASessionLog_Unicode] tables.

Number or rows generated by each process can be easily retrieved i.e.:

• select processname, count(*) from BPASessionLog_NonUnicode (nolock) group by processname
• select processname, count(*) from BPASessionLog_Unicode (nolock) group by processname

Space used by these tables can be retrieved i.e.:

• exec sp_spaceused N'BPASessionLog_NonUnicode'
• exec sp_spaceused N'BPASessionLog_Unicode'

Monitoring and auditing of these tables through third-party tools is allowed (verify you have the correct permissions to access the required table), however, please note, DB speed could be impacted by doing so.

Database Administrators can assist with the upfront and on-going assessment of the amount of disk space which your Blue Prism database may require. Monitoring the size of how much data is created for each Process in terms of the number of rows and space used by that data will form the basis for understanding:

• What size of disk is needed to support the data volumes in SQL Server during development phases.
• SQL Server TempDB database size: managing maintenance operations on large tables is recommended.
• How to configure Blue Prism's automatic archiving of log data and how to define an Archive and Backup policy.

Please refer to the Maintaining a Blue Prism Database Server data sheet available on the Blue Prism portal under the area: Release specific documentation -> Infrastructure and Installation as a guide for maintenance of a Blue Prism database.

Blue Prism provides a template document below to help you to assess and record the logging configuration and policy. 

A sensible archive strategy will allow the production database size to be kept to a minimum whilst still allowing retrieval of log files, should business units require them. Backing up production process and objects will provide some resilience to database failure for business-critical processes.

Blue Prism provides a template document to help customers to assess and record this kind of operational task. See below under Related Documents.