Best Practices for a Successful Go-Live

After installing or upgrading Forms Builder and developing form sequences in a test environment, it is important to observe some best practices to ensure a smooth transition to a production environment. The following recommendations are intended to help ensure a successful go-live.

Logging

alert Important
  Log files may contain confidential information such as user names and passwords, account information, etc. It is your responsibility to protect sensitive user and system data.

To mitigate the risks of exposing sensitive data, observe the following best practices:

  • Set the log level in production environments to the lowest, least detailed log level. Increase the log level in test environments only when needed. Reset the log level when testing is complete.

    The default logging provider used by Anthology Inc. products is NLog. NLog allows you to configure log targets, levels, rules, layouts, etc. through configuration. To configure logging, modify the NLog.config file in the application’s executing directory. For Web applications, this file exists alongside the web.config file.

  • When LogLine or LogObject workflow activities are used to capture entities that contain sensitive information, remove such activities as soon as testing is complete.

    We recommend setting the Level value to Information for any LogLine or LogObject activities. See Best Practices for Logging and Logging in Azure.

    If, instead you followed the recommendations, and the development machine NLog minLevel is set to “Info” and all logging is done at the “Information” level, and the production machine NLog minLevel is set to “Error” (default), then nothing needs to be done because the production machine will not log “Information” LogLine or LogObject activities. The additional benefit is that the logging is still available if a problem can only be seen in a production machine and lowering the NLog minLevel to “Info” temporarily (and restarting the app pool) will allow troubleshooting.

NLog Levels

In a test environment, the NLog.config might have been set to Info for debugging purposes. The Info level captures detailed messages written by LogLine workflow activities.

In a live environment, the NLog.config should be set to Error level so that none of the LogLine information appears in the log files and performance is improved. By changing the NLog level you don't need to remove LogLine activities from the workflows. If troubleshooting is required in a live environment, you can set the NLog level temporarily to Info.

  1. In the CMCFormsRenderer_V3 folder, locate the NLog.config file.

  2. In the <rules> section of the NLog.config file, set minLevel to Error.

    <rules>
       <logger
        name = "*"
        minLevel =
    "Error"
         writeTo = "file" />
    </rules>

  3. Save the NLog.config file.

alert In Forms Builder 3.5.1 and later, the ability to set NLog levels in the Settings workspace of Form Designer is removed to prevent conflicts with Azure log configurations. Azure logs are stored in customer-specific tables. If your Forms Builder deployment is in an Azure environment, contact Anthology Inc. obtain access to the Azure log tables or to request changes in the NLog settings.

In Forms Builder 3.6., several logger.debug statements and client-side logs are modified to Info level to make them available to help debug issues in an Azure environment since in an Azure environment the log level is set to Info level for all products. The Info level is set for logs related to:

  • Site Warmup
  • LookupUser
  • Account Controller
  • PDF creation and e-sign documents
  • Payment processing for PayPal, ACI, and IATS

Workflows

Use formInstance.ValidationMessages

Check the value of the ValidationMessages property on all workflow activities that have this property. The value should be set to formInstance.ValidationMessages to ensure that all form validation errors are captured in a live environment.

Don't Use Condition=True in Transitions

Set the value of the Condition field in transitions to Not formInstance.ValidationMessages.HasErrors or leave it blank. Do not use a Condition of True.

If the Condition of a transition evaluates to False, the transition will not occur. If the Condition is blank, the transition will occur. The value True can cause unexpected results.

Check the Placement of Custom Validations

Custom Validations using the CreateValidationItem activity should be placed in transitions after the WaitForFormBookmark activity and before the Condition.

Use Caution with Hard-Coded Values

When using hard-coded lookup values, any minor difference between test and live environment could cause errors with invalid or non-existent values. For example, if a DocumentType value is hard-coded as 43 instead of using a LookupReferenceItem activity on the DocumentType property, the value might not return the expected results in a different environment.

Place Save Activities in the Final Transition

Many of the save activities such as SaveDocument should be done in the final transition (not in the form/state itself). The save should occur after the WaitForFormBookmark activity. The transition's Condition for completion can then validate that no errors are returned by the activity before completing the sequence.

Initialize Values in the First State of a Workflow

Initialization of values based on an authenticated user should be done in Entry section of the first form/state in a workflow.

Remove the Back Transition in Complex Workflows

If a workflow performs multiple activities that create entities (such as enrollments) followed by other activities in later forms that rely on previous activities (e.g., registering for a class), remove the Back transition option to avoid duplicates (such as creating a 2nd enrollment).

Create Short Sequences and Simple Workflows

To avoid rollback issues with long sequences and complex workflows, it is best to have sequences with fewer forms and workflows designed to do only one specific thing.

Prevent DbUpdateConcurrency Exceptions

A DbUpdateConcurrency error occurs when an attempt is made to update an instance of an entity via a Save activity, but that instance has been modified by another user in the time from when the instance was initially retrieved in the workflow to the point in time when the Save activity executes. Closed

Example of a DbUpdateConcurrency exception in a Renderer log file:

2018-02-27 13:30:16.7645 54 Error Cmc.Nexus.Crm.Workflow.SaveDocument System.ServiceModel.FaultException`1[System.ServiceModel.ExceptionDetail]: Store update, insert, or delete statement affected an unexpected number of rows (0). Entities may have been modified or deleted since entities were loaded. See http://go.microsoft.com/fwlink/?LinkId=472540 for information on understanding and handling optimistic concurrency exceptions. (Fault Detail is equal to An ExceptionDetail, likely created by IncludeExceptionDetailInFaults=true, whose value is: System.Data.Entity.Infrastructure.DbUpdateConcurrencyException: Store update, insert, or delete statement affected an unexpected number of rows (0). Entities may have been modified or deleted since entities were loaded. See http://go.microsoft.com/fwlink/?LinkId=472540 for information on understanding and handling optimistic concurrency exceptions. ----> System.Data.Entity.Core.OptimisticConcurrencyException: Store update, insert, or delete statement affected an unexpected number of rows (0). Entities may have been modified or deleted since entities were loaded. See http://go.microsoft.com/fwlink/?LinkId=472540 for information on understanding and handling optimistic concurrency exceptions. at System.Data.Entity.Core.Mapping.Update.Internal.UpdateTranslator.ValidateRowsAffected(Int64 rowsAffected, UpdateCommand source) at System.Data.Entity.Core.Mapping.Update.Internal.UpdateTranslator.Update() at System.D...).

The best practice to avoid this error is to add a TransactionScope activity to the workflow. Use the defaults of IsolationLevel = Serializable, and a timeout of 1 minute.

Within that TransactionScope, add a GetEntity activity to retrieve the instance of the entity prior to the execution of the SaveEntity activity. Any property values that need to be updated prior to saving can be done so via Assign statements right after the Get activity and right before the Save activity.

A transaction locks the database to give the workflow a chance to read and update with no other process simultaneously doing the same. Read about the other less aggressive isolation levels as they may be adequate for the purpose based on the type of updates being done and produce less overhead. Google “TransactionScope IsolationLevel Activities”. A “RepeatableRead” may be sufficient.

This pattern will eliminate any chance that another user will update this record in between the execution of the Get and Save activities within the workflow.

Form Data

Avoid Null References in Workflows

To avoid null reference exceptions, ensure your forms accept only valid values. When working with entities, always use a CreateEntity activity (if data is initialized in following form) or use a GetEntity activity if looking up a known item. For example, use a GetEntity activity to retrieve a StudentEntity based on the User Id supplied when a user logs in to a sequence.

Use Form Designer Properties

For all fields and components in the Layout pane, take advantage of the given Form Designer properties to ensure good data. For example, use the Required property and specify values in the supported value ranges.

E-Sign Sequences

Place View Summary Before E-Sign Component

In sequences with e-sign component, insert the View Summary component before the e-sign component so that the end user can review the responses on all forms before the e-sign process is invoked.

Include RecipientStatus Activities

Include GetAdobeSignRecipientStatus, GetDocuSignRecipientStatus, or GetSignNowRecipientStatus activities in your e-sign workflows to handle all status changes (e.g., retry, completed, denied) and to recover from error conditions including connection loss.

Application Initialization

In Forms Builder 3.4 and later, Designer and Renderer can take advantage of Application Initialization which is available on Microsoft Windows Server 2012 and later as a standard part of IIS installation.

This means that when a server comes up, or an application pool is reset, or IIS is reset, the website automatically starts warming up as if a first user had launched a URL on the website. Further, when an application pool automatically recycles, IIS keeps the current process serving files while it warms up a new process. When warm, it redirects requests to the new process and kills the old one, resulting in a seamless recycle of the website.

Typically, a website that is hit by the first user has to start loading all the resources the website needs to serve webpages. This can take a while. On a website with Application Initialization, the loading of resources can happen automatically, and within a few minutes response times will be greatly reduced.

In addition, Designer and Renderer support caching. The caching is done during the warmup. This reduces trips to the database and significantly improves the user experience. Designer caching is on all the time and produces a noticeable performance increase when moving between panels and workspaces within Designer. Renderer caching is set in the Settings workspace. See Enable Renderer Caching.

Warmup is implemented in Anthology Student version 19.0.5 and later. CampusNexus CRM, Staff STS, and Student STS websites are not warmed up. If your workflow uses these sites, it may still take a while for the first user of a sequence to warm up these websites.

If your server has Application Initialization installed, there are two configuration items for IIS to get this to work fully.

  1. On each application pool that you are using (Designer and Renderer usually use different ones), go to Advanced Settings and set Startup Mode to Always Running.

  2. On each website (Designer and Renderer), go to Advanced Settings and set Preload Enabled to true.

Note: These settings will be configured by Installation Manager during the installation of Forms Builder 3.6 and later.

To test if this is working (non-Azure sites only):

  1. In Forms Builder Designer, go to Settings and change the NLog Level to Debug for both Renderer and Designer.

  2. At an administrator command prompt, type net stop w3svc. This will stop the IIS process.

  3. Delete the Designer and Renderer log files for today in C:\Logs.

  4. At the administrator command prompt, type net start w3svc.

  5. Designer and Renderer log files should be automatically recreated. They will contain Debug log lines with the words Starting Warmup.

  6. Reset your NLog Levels to their original settings and restart IIS.

For details on Application Initialization, see https://docs.microsoft.com/en-us/iis/get-started/whats-new-in-iis-8/iis-80-application-initialization.

Persisted Workflow Instances

It is quite common that students begin to fill out a Request For Information form and then change their mind and exit the sequence before completing. As a result, the durable instance table will accumulate rows for abandoned sequences that never reach an end state. For a workflow administrator it can become challenging to locate a specific workflow instance when browsing the list of persisted workflows in Workflow Composer. Closed

Persisted Workflows

A script example has been provided to remove persisted workflow instances for abandoned sequences from the Durable Instancing table. Run this script periodically as needed.

In Forms Builder 3.6 and later, persisted workflow instances can be deleted from the Sequence Designer workspace. For more information, see Delete Persisted Workflow Instances.

To run the script to remove persisted workflow instances:

  1. Open Microsoft SQL Server Management Studio.

  2. Connect to the server and database indicated in the status bar of your Workflow Composer.

    You must have administrator permissions to the database.

  3. Click New Query.

  4. Copy and paste the following script example into the query window:

    You may need to adjust the maximum number of days in the statement "DECLARE @MAX_AGE_IN_DAYS INT = 20".

    • If your form sequences use DocuSign, the time period should be at least 20 days.

    • If your form sequences do not use DocuSign, reduce the time period as appropriate.

    -- Delete instances older than the defined age CREATE PROCEDURE [dbo].[Sproc_DeleteWorkflowInstances_20DAYS] AS BEGIN DECLARE @MAX_AGE_IN_DAYS INT = 20 DECLARE @UsefulCursor CURSOR ,@SurrogateInstanceId BIGINT SET @UsefulCursor = CURSOR FOR SELECT i.SurrogateInstanceId FROM [System.Activities.DurableInstancing].[InstancePromotedProperties] p INNER JOIN [System.Activities.DurableInstancing].[InstancesTable] i ON i.id = p.InstanceId WHERE DATEDIFF(DAY, i.LastUpdated, GETUTCDATE()) >= @MAX_AGE_IN_DAYS ORDER BY i.LastUpdated DESC OPEN @UsefulCursor FETCH NEXT FROM @UsefulCursor INTO @SurrogateInstanceId WHILE @@FETCH_STATUS = 0 BEGIN EXEC [System.Activities.DurableInstancing].[DeleteInstance] @SurrogateInstanceId FETCH NEXT FROM @UsefulCursor INTO @SurrogateInstanceId END CLOSE @UsefulCursor DEALLOCATE @UsefulCursor END

  5. Click the Execute button to run the script.

    Expected result: Commands completed successfully.

  6. Once the script is saved to the database, use the following command to run the script whenever needed:

    exec Sproc_DeleteWorkflowInstances_20DAYS

    You can also put the exec command into a scheduled SQL Server Agent job, so it runs nightly unattended.