Managing Data

It's critical to have a data retention policy that explains what data must be held, why and where it's being held, and how long it should be held for. Some data is required to be retained by a company's internal rules. Other data is required by law to be retained for a certain period of time. Common types of retained data include files, email messages as well as digital worker audit and session logs.

Blue Prism Digital Workers record all their activity in a session log that captures a detailed history of their actions on a case by case basis. This record might include - for example – every button pressed, every piece of data that is read, every decision made, etc. Where sensitive data is concerned, the logged information can be masked or omitted if necessary, in order to comply with data standards such as PCI (Payment Card Industry).

The detail of the log relates directly back to the structure of the process diagram, making it easy to interpret by business users. Work items are typically captured in a Blue Prism queue, this provides Operations teams with immediate access to case status and outcomes, as well as management information on case times, creation and completion dates.

Over time records and documents that are no longer used for business purposes become irrelevant, but continue to take up valuable space in operational databases and computer disks. Archiving data to a cost-effective and secure tier is a best practice for freeing up space in applications and improving application and reporting performance in the organization. In some cases, information may need to be retained longer than the assigned policy, for example, if a legal hold is required to keep data available for litigation purposes or case specific data continues to hold value over time. Data and documents should be routinely deleted upon expiration of the retention period or expiration of the period mandated by the policy.

Compliance is one of the major reasons for a company to retain data. It is common for an organization to establish its own data retention requirements, in addition to this, there are several laws and regulations that a company needs to consider in forming its data retention policy.

This is particularly true for organizations operating within regulated industries. For example:

• Sarbanes-Oxley Act (SOX) data retention policy for companies publicly traded in the U.S.
• Health Insurance Portability and Accountability Act (HIPAA) for healthcare organizations
• Payment Card Industry Data Security Standard (PCI DSS) for organizations that accept credit cards
• General Data Protection Regulation (GDPR), which regulates data privacy laws across the European Union. Mandates apply to personal data produced by EU citizens, whether or not the company collecting the data is in the EU, as well as any people and organizations whose data is stored within the European Union.

Blue Prism simplifies the audit process by automatically generating a detailed audit history. Blue Prism automations are self-documenting and audited automatically. Blue Prism digital worker actions are recorded by date and time. They also record any exceptions, job failures or event types that need to be reported for compliance to HIPAA, GDPR, or other standards. How long audit records should be kept is dependent on the organization's internal policies as well as its industry compliance policies.

Blue Prism provides native archiving solutions for its digital worker session logs. Specific procedures need to be designed to archive files and email messages that are part of the automation: this aspect of data retention is often overlooked, but emails and files pile up quickly, and some take up a lot of space, so it's important for an organization to define an archiving policy that includes documents that are not directly saved on the Blue Prism database.

Blue Prism tools for archiving include:

• System Archiving (from Blue Prism client or from Blue Prism Command line): involves the transfer of session log data from the database to a file structure, freeing up database space and allowing permanent archiving of the old data. The archived data can also be restored back to the database at any time.
• Data Gateways (version 6.5 Enterprise Edition or above): provides an easy-to-use, centralized method of pushing data out of Blue Prism for use in external systems for monitoring and reporting, long-term data storage.
• Splunk integration with Blue Prism (version 6 Enterprise Edition or above): Splunk is a data collection and analysis tool which can be used to receive Session Log data from Blue Prism runtime resources via a HTTP endpoint.

As already mentioned, external documents (such as emails and files) which are part of the automation but not directly stored in the Blue Prism database must be included in the archive strategy.

A sensible archive strategy will allow the production database and disk size to be kept to a minimum whilst still allowing retrieval of log files -  should business units require them. Blue Prism provides a template document to help customers to assess and record this kind of operational task. See the Documents section of the Portal under the area: Robotic Operating Model -> Technical and Security -> Database Administration -> Archive and Backup Policy template (or at the bottom of this page). The Archive and Backup Policy Document gives guidance to archiving process logs. Find out more about logging levels.

A data retention policy should outline which media types are used for each category of data. Organisations need to focus on securing data throughout their life-cycle: the choice of where to store an archive is an important consideration to protect the integrity of the data against unauthorised access and natural disasters.

The organization should be able to search and retrieve archives easily within specified SLA’s.

Routine retrieval testing: It is recommended to verify the integrity of the archives by periodically testing the recovery process to test the readability of archiving media and helps to reinforce established retrieval procedures for the RPA COE.

Index and Search: It is critical to understand the business importance of indexing and searching. Retrieving needed files is crucial for compliance audits and litigation support activities. When a request for auditing is made, a company typically must provide the required data in a very short time. Retrieving important data months or years later would be problematic without a clear indexing policy.

The Blue Prism archiving process organizes the archive files automatically by date, process name and resource name making it easy to retrieve information at a later date. A similar indexing policy must be established for archiving of other documents (such as emails and files).

Security and Retention: Data retrieval from an archive should be restricted to authorized personnel. Blue Prism provides roles and permission to restrict access to archiving and data retrieval functions.