BbPatch 2.2.

5 Documentation
Contents
Contents
Features
Supported Products
About Updates
About Patches
Risks of Patches
Mitigating Patch Risks
Appropriate Request and Usage of Patches
Package Format
Package Scope
Minimizing Downtime
Rolling Maintenance Caveats
Resolving Package Conflicts
Package Repository
Configuring the Repository
Configuring a Proxy Server
Setting the Repository Temporarily Offline
Disabling the Repository
Using the Utility
Specifying Command Line Options
Configuring Settings
Data Storage
Hostname Uniqueness Requirement
Cloning Environments
Blackboard Upgrades
Managing Packages
Before You Begin
Applying Packages
Rolling back packages
Describing Details of a Package File
Listing Installed Packages
Check Installed Packages
Download Packages
FAQs and Problem Resolution

Features
BbPatch is a package management utility provided by Client Support to manage updates to Blackboard products, such as patches, patchsets and
security updates. It complements the Blackboard Installer by allowing small, reversible updates with minimal downtime.

The utility has the the following features:

Apply packages to a system, including version and platform compatibility checks, conflict detection and resolution
Packages can be downloaded from a secure online repository or applied from a local file
Robust rollback functionality that ensures a host can be restored to a previous state
An inventory that details all changes made to an environment, allowing listing of installed packages for one host or for all hosts across an
environment
The ability to checking the consistency of packages to ensure the packages are consistently installed and up to date

Supported Products
BbPatch can manage updates for the following products:

Blackboard Academic Suite Release 8.0

Blackboard Learn Release 9.0
Blackboard Learn Release 9.1

About Updates
Several types of updates are distributed using BbPatch packages:
 Patches - individual, rapid engineering fixes for critical product issues. See About Patches\ for important information about patch updates
Patch Rollups - a collection of individual patches that have undergone functional testing and resolve the most commonly reported issues
for a release
Security Updates - address one or more security issues advised via a security bulletin
Utilities - Support scripts, database fix scripts and utilities

If packages are installed, manual changes to files, in particular jar library files should be avoided.

About Patches
Patches are distinct from maintenance releases such as hot fixes and service packs. They are provided by Tier 3 Support for high impact issues
as a result of an escalation from the Tier 2 team and typically resolve one issue. Once a patch is provided for a given release, the issue
addressed by the patch is targeted for an upcoming Service Pack or Hot fix, depending on the severity.

If a patch is issued late in a release cycle, this could mean the patch is not included in the very next release, so please review
the resolved issues report for a release prior to upgrading to and contact Client Support should a patched issue not be listed as
resolved. A new patch may need to be issued, please request this well advance of upgrading to that maintenance release so the
patch can be integrated into upgrade testing.

Risks of Patches
Applying a patch may cause new issues. Patches are developed under urgent timelines and each patch is tested only to validate that it fixes the
specific issue it was designed to correct. No further regression testing is performed to identify potential impacts on other areas of the software.
Given this risk, patches should only be applied to address a specific, critical issue that is impacting the system and/or user community.

Mitigating Patch Risks

Except where supported Patchsets are provided, Blackboard Support does not recommend applying multiple patches to a Blackboard instance at
one time. Applying multiple patches at once can increase risk exponentially due to the aforementioned risks of applying an individual patch, added
to the fact that we do not test the potential impacts of a given patch on other patches. Problems arising from sequential application of multiple
patches can be very difficult and time-consuming to troubleshoot and correct. For these reasons, we do not provide a public list of all available
patches.

Appropriate Request and Usage of Patches

Blackboard Support recommends the application of patches that are provided through the normal Support process, that is, submission of case for
a specific, repeatable issue and confirmation from support that a particular patch addresses that issue. We create and provide patches specifically
for this purpose and can efficiently support any issues that may arise from the application of an individual patch. We are constantly striving to
improve the internal visibility of available patches.

Package Format
Packages that can be applied using BbPatch are identified by a .bbp extension and use the package identifier as the filename, for example
AS-162070.bbp. Platform specific patches have the platform appended to the filename, either Windows, Unix, Solaris or Linux. The key fields
described by the package are:

Identifier The unique identifier for the package. All commands that take an identifier as an argument refer to this field

Resolved This is the Known Issue(s) that this package addresses. For patches, this will be the identifier shown in the resolved issues report
bugs when the bug is resolved in a release. This field is also used for package conflict resolution. See Resolving Package Conflicts for
more information.

Compatible This indicates the build(s) with which the the package are compatible. It is not possible to install a package for an incompatible
with release.

Platform The platform that the package is compatible with. Most packages will be 'Any'
 Do not attempt to modify a package to allow it to be installed on a version or platform other than specified by the package
manifest. This may cause unintended behavior, system instability, functionality or data loss. If a package is not compatible,
contact Client Support for assistance.

Package Scope
Operations in packages generally apply to an individual host. However, operations that affect the shared content area and database are only
applied once per environment, that is, they affect the environment as a whole or are global. Depending on the operations performed by a
package, it may only be necessary to apply a package once per environment. This information is listed under 'Package deployment guidance'
section of the package details listing. There are three package scopes:

Host-only The package applies to individual hosts and must be applied/rolled back on all hosts

Host and The package has a mix of host and global operations. Packages that contain a mix of host and global operations, apply and
Global rollback the global operations on the first host to apply the patch, and the last host to be rolled back. Host operations are applied
normally as applied to each host.

Global-only The package applies to the environment globally. It only needs to be applied or rolled back once per environment. Packages with
only global operations only need to be applied once per environment. Attempting to apply the package on another host will
indicate that the package is already installed. These packages can be listed, checked or rolled back from any host

The automatic handling of operations global to the environment means there is no technical requirement for electing a host to handle these
operations, or to apply or rollback in any particular order.

Minimizing Downtime
Package operations are designed limit the service downtime requirements as much as possible. However, packages involving Java classes, as
most do, will not be effective until after an application restart. The deployment guidance section of a package listing will advise the user of the
downtime requirement for package maintenance. There are three possible categories:

Category Downtime Reason

Requirement

All Hosts Stop the The package makes updates to a shared resource. The change will be immediately active when applied or
application on all rolled back, and may be unsafe while the application services are running. The package may also include
hosts for the dependencies that have to be installed before the changes will function correctly. Operations trigger this
duration of apply or requirement include Building Blocks and SQL scripts.
rollback.

Current Stop the The package updates key libraries that cannot be safely updated while the application is running, or are
host application before running on a Windows platform.
apply and rollback
on the current
host.

None Package can be N/A

applied or rolled
back while the
application is
running

In load balanced environments, the latter two options allow packages to be applied during rolling maintenance, either rolling application
shutdowns or rolling restarts.

A restart is always required for changes to take affect and it is strongly recommended that restarts are always scheduled at the
same time as package maintenance. Some changes may take effect immediately.

Rolling Maintenance Caveats

While the Learn application supports graceful session fail-over, truly seamless fail-over in the event of an application shutdown or restart is highly
dependent on load balancer configuration and operational practice. Rolling maintenance should only be performed with an established and tested
configuration that ensures that end-user sessions fail-over cleanly to other application servers. Standard application server health checks should
not be so aggressive as to prevent application server restarts being apparent to end users, ad this can lead to application outages, poor
performance or crashes as load is prematurely moved from application servers due to minor responsiveness issues (aka 'load waterfalling').
Recommendations for rolling maintenance practice are outside of the scope of this documentation, however an approach might include:

1. Taking each application server offline manually on the load balancer before shutting down and restarting the application
2. Employing a health check method that allows a server to be removed from the pool based on the presence of a file, removing a file or
changing permissions on that file
3. Some load balancers support scripting, and multiple health checks per pool and in these cases, use the built-in health check URL as well
as a request to indicate the maintenance state of the host

In all cases, ensure that all end-user requests to the application sever have stopped before shutting down or restarting the application.

Resolving Package Conflicts

If conflicts are found, the utility will attempt to resolve the conflict automatically:

If the package to be applied resolves the same bugs as an installed package and is newer, as indicated by the package manifest, the
installed package will be rolled back and the new package automatically applied. All other conflicts will result in an error and no changes
being made to the system

Using --force option

For all the cumulative patches SP10 and above

Use --force option to remove the previous cumulative patches and replace with the new package.
For all the individual patches and patch sets
If the --force option is specified, the installed package and therefore the updates it made will be removed and replaced with
the new package. This option should be used with care as it may cause issues to be reintroduced

If the utility is unable to resolve a conflict and a package is required that rolls up two or more packages, please contact Client Support for
assistance.

Package Repository
The package repository is a central service for package distribution. It has the following advantages over legacy methods of distributing packages:

Secure - packages are distributed via SSL secured HTTP and metadata and packages are verified with SHA1 hashes. Communication is
limited to TCP/IP port 443 (or a configured proxy server port)
Centralized - packages are stored in a single, authoritative repository. The latest revision of a package is guaranteed
Convenience - once downloaded from the repository, packages are stored in a local replica and are available to all application servers in
an environment
Flexible - authenticated proxy servers are supported and the repository functionality of the utility can be disabled completely

The repository is used to retrieve packages automatically for any command for which a package identifier can be specified in place of a filename
(a filename being the 'legacy' option). For example the following command will download and describe the details of package LRNSI-1234:

bbpatch desc LRNSI-1234

If the repository cannot be used in an environment due to security policy or technical reasons:

The repository functionality can be disabled completely, avoiding errors on hosts that are unable to connect to the repository
The utility can be installed on any host. The download command saves packages to the current working directory, rather than to a
repository replica, retaining the secure transfer and hash verification benefits. Those packages can then be securely transferred to the
application server environment to be applied.

Configuring the Repository

Several bbpatch-config.properties settings are available to control the repository functionality. See the Configuring Settings section for
more information about the configuring the utility.

Configuring a Proxy Server

The utility supports HTTP proxy servers with optional authentication via the following settings:
 # Proxy hostname or IP address
proxy.host=
# Proxy TCP/IP port, defaults to 3128
proxy.port=
# Proxy username, optional
proxy.username=
# Proxy password, optional
proxy.password=

Setting the Repository Temporarily Offline

If for some reason application servers are unable to connect to the online repository, but packages are already available in the local replica, the
utility can be instructed to temporarily resolve packages from only the local repository by setting repository.online to false.

Well before scheduled maintenance, use the describe command to pre-stage packages. When run on an application server it
stores a copy of the package in the local repository replica. In the event of a problem with connectivity to the repository during a
maintenance window, the repository can be set offline and the already resolved package will be used.

Disabling the Repository

The repository functionality can be disabled completely by setting repository.enabled to false. This is recommended for any environment
where hosts are unable to connect to the repository.

Using the Utility

The utility is a text based stand-alone Java application and is run from a command prompt or terminal. A batch file or shell script is used to invoke
the utility. The Building Block plugin installs the shell scripts to the BLACKBOARD_HOME/tools/admin directory. Note the difference in
capitalisation when compared to the zip distribution:

Distribution Command

Building Block
BbPatch.bat/sh <action> [options]

Zip file
bbpatch.bat/sh <action> [options]

Before running the utility for the first time, using either distribution, verify the following requirements are met:

The utility requires that the application is running Java 1.6 as it uses the JAVA_HOME specified in bb-config.properties. This should
only be a concern on 8.0 and early 9.0 releases. All later releases require Java 1.6
On Windows platforms:
In load balanced environments, the utility must be run as a user that has permission to the shared content area. The service
account used for the application is recommended
On Unix platforms:
On Unix the utility must be run as root or bbuser. The shell script will switch to bbuser automatically when launching the utility

For the zip file based distribution:

The utility assumes that /usr/local/blackboard is the Learn installation location. It will prompt to set a BLACKBOARD_HOME
environment variable if that location cannot be found
The scripts may be put on the path and invoked from any location

Specifying Command Line Options

The utility takes an 'action' argument, followed by options specific to that action. Running the utility with no arguments outputs the list of each of
the available actions:
 Blackboard Patch Client version 2.1-SNAPSHOT

Package management tools for Blackboard Learn.

Available actions:
apply Apply packages.
check Check that installed packages are consistent and up to date.
describe Describe details of a package.
download Download packages from the remote package repository.
list List installed packages.
rollback Rollback installed packages.
upgrade Upgrade application data from a previous version.

Usage: bbpatch <action> [options]

Available options:
-?,--help print help information. Specify with an action for help
with that command
--version print version information

[SUCCESS] The operation completed successfully.

All command line options support a long and a short version of the argument. Use -? or --help in conjunction with any action for details of the
command-line options that are supported. Actions commands such apply may be abbreviated to a single letter, as long as a unique action
matches.

Configuring Settings
A bbpatch-config.properties file can be found in the config directory in the utility installation location. When using the Building Block
distribution, this location is in the shared content area and the configuration file is used by all application servers:

<bbconfig.base.shared.dir>/vi/<bbconfig.database.identifier>/plugins/bbcs-bbpatch/webapp/WEB-INF/config

The values for the shared directory and instance identifier will vary depending on the platform and configuration. Refer to the
bb-config.properties to confirm these values.

In addition to the configuration file, configuration properties can be set temporarily using command line arguments:

-Dproperty=value

For example, to temporarily set the repository offline, specify -Drepository.online=false. Multiple -D options may be specified in a single
command.

Each available option is documented in this guide. The default configuration values are correct for the majority of environments.

Data Storage
Utility data is stored in the Blackboard shared content area in a directory called bbpatch. You should not modify the content of this directory
manually unless specifically instructed by this documentation, a utility message or assistance from Client Support.

Hostname Uniqueness Requirement

The package inventory uses the default hostname of the host for the directory name used to store package metadata and rollback files:

bbpatch/inventory/appserver/<hostname>

On both Unix and Windows hosts, use the hostname command from a shell/command prompt to confirm the hostname that will be used. This
value must be unique for a particular environment.

Cloning Environments
When cloning an environment, additional steps are required to ensure the cloned application servers will reflect the previously installed packages:

Go to the bbpatch/inventory/appserver directory

If each application server was individually copied, note the source and destination hostnames that were used and copy and rename each
of the appserver-specific inventory directories to match that
If one application server was used as a template for all the others, remove all directories (if any) except the one for the source application
server and create a copy for each cloned application server, renaming the directories to match the destination hostnames
Update the local appserver's blackboard/system/bbpatch/target.hostname file to reflect the new hostname and thus tell bbpatch that this
appserver's inventory has been correctly updated. Altering this file without first correcting the inventory invalidates the inventory and will
lead to bbpatch malfunction!
Use the list and check commands on each application server to verify that the applied packages are correctly reflected and are
consistently applied

Blackboard Upgrades
When performing a release upgrade, the Blackboard Installer establishes a known-good baseline for all files and database objects, any changes
made by packages are removed. The utility will detect the change in application version following the upgrade and automatically backup and
remove the obsolete package inventory.

Managing Packages

Before You Begin

Although the utility apply and rollback behavior is robust, it should not be solely relied upon for rollback capabilities. Ensure that file
system backups are available for any software maintenance activity
The Blackboard Learn services must be stopped under most circumstances on Windows environments. Windows employs exclusive file
locking and will prevent files from being updated
Packages generally must be applied to every application server in an environment. See the Package Scope\ section for more information
Packages have varying downtime requirements. See the Minimizing Downtime\ section for more information

Applying Packages
The apply action applies the operations listed in a package. Before making any changes to an environment, the utility ensures that:

The package is compatible with the product version

The package is compatible with the platform
The already applied packages are in a consistent state and are not partially applied or rolled back
If the package updates Tomcat libraries, that the copy of the library in the systemlib directory is the same as the version in the Tomcat
library directory
If the package requires host downtime, that the Blackboard Learn services are stopped. This includes Tomcat, Collaboration and in 9.1
SP6 and later, the Exec service

After this validation is complete, no changes will be made until the utility receives confirmation that package should be applied unless the
--silent option is specified.

Apply Command Usage

The apply action supports any number of package identifiers or package filenames:

<bbpatch-command> apply /tmp/private-patch/LRNSI-123456.bbp LRNSI-654321 LRN-555222

The available options are:

-f,--force if any package conflicts with one already installed, force the removal of the installed
package(s)
-s,--silent perform the action without prompting for confirmation

Rolling back packages

The rollback action reverses the package operations, replacing the replaced files or objects with the originals copied to the rollback location
when the package was applied.

Rollback Command Usage

The rollback action takes any number of package identifiers as the only required argument:

<bbpatch-command> rollback LRNSI-123456 LRNSI-654321 LRN-5552

The available options are:

-s,--silent perform the action without prompting for confirmation

Describing Details of a Package File

The describe action shows the details of a package. See Package Format\ for a description of each of the fields listed:

Identifier : AS-162070
Description : Text and images are displayed distorted while navigating with IE9 and Arabic
Language Pack
Resolves bugs : AS-162027
Date built : 2011-07-12 15:46:53 +0100
Compatible with : 9.1.60230.0
Platform : Any
File operations:
Update frameset_jsp.class -> webapps/portal/WEB-INF/classes/blackboard/web/frameset_jsp.class
Update frameset_jsp.class ->
webapps/searchwidgets/WEB-INF/classes/blackboard/web/course/frameset_jsp.class
Update frameset_jsp.class ->
webapps/searchwidgets/WEB-INF/classes/blackboard/web/ext/frameset_jsp.class
Update subtabFrame_jsp.class ->
webapps/portal/WEB-INF/classes/blackboard/web/portal_005fjsp/layout/subtabFrame_jsp.class
Update viewPortfolio_jsp.class ->
webapps/bbcms/WEB-INF/classes/blackboard/web/portfolio/viewPortfolio_jsp.class
Update courseFrameset_jsp.class ->
webapps/blackboard/WEB-INF/classes/blackboard/web/course/courseFrameset_jsp.class
Update downloadWrapper_jsp.class ->
webapps/blackboard/WEB-INF/classes/blackboard/web/content/downloadWrapper_jsp.class
Update integrationFrameWrapper_jsp.class ->
webapps/xythoswfs/WEB-INF/classes/blackboard/web/IntegrationFilePicker/integrationFrameWrapper_jsp.class
Update filesframeset_jsp.class ->
webapps/xythoswfs/WEB-INF/classes/blackboard/web/filesframeset_jsp.class
Update gradecenter_005fframeset_jsp.class ->
webapps/gradebook/WEB-INF/classes/blackboard/web/gradebook2/instructor/gradecenter_005fframeset_jsp.class
Update contentWrapper_jsp.class ->
webapps/blackboard/WEB-INF/classes/blackboard/web/content/contentWrapper_jsp.class
Update frameset_jsp.class ->
webapps/searchwidgets/WEB-INF/classes/blackboard/web/user/frameset_jsp.class

Package deployment guidance:

Package scope: Host-only. Apply/rollback on all hosts
Downtime requirement: This host. Stop application before apply and rollback on this host

Describe Command Usage

The describe action takes any number of package identifiers or filenames:

<bbpatch-command> describe LRNSI-123456.bbp LRNSI-654321

Listing Installed Packages

The list action provides the ability to list the details of a the currently installed packages on an environment.

List Command Usage

By default, the list action lists all packages installed on the current host and those that apply globally to the environment. See Package Scope\ for
more information on the host/global terminology.

list

Blackboard Learn, Version 9.1.60230.0, C:\blackboard

Packages installed on this host:

Identifier Date Installed Date Built Bugs Resolved

AS-163058 2011-08-11 15:37:43 +0100 2011-08-09 01:00:16 +0100 AS-148787
AS-163080 2011-08-11 15:31:47 +0100 2011-08-08 20:28:57 +0100 AS-156226

Packages installed globally:

Identifier Date Installed Date Built Bugs Resolved

AS-162340 2011-08-11 15:32:31 +0100 2011-07-12 18:28:07 +0100 AS-157908

Detailed patch information and installed packages from all hosts in the environment can be generated by specifying the following command-line
options:

-a,--all show details for all hosts

-v,--verbose display additional information

Check Installed Packages

Use the check action to determine whether a specific package is properly applied and is consistent. There is the option to check an individual
package, or all packages that are installed on the system. The check command also checks the Package Repository\ to ensure that the latest
revision of a package is installed.

Checking of Oracle database objects is not currently supported.

Check Command Usage

By default, the check action checks all installed packages. Specify a package identifier to check an individual package.

Check uses md5 checksums to verify that files and database objects have not changed since the package was applied. If a change is detected
this generally indicates that files have been modified manually:

check

Blackboard Learn, Version 9.1.60230.0, C:\blackboard

Checking all installed packages...

Checking package AS-163058...

Checking rm_orphaned_loc_srch_in_navigation_item.sql at
C:\blackboard\system\tooldefs\system\LicenseUpdate\license-handlers\enterprise.contentsystem\database-changes\pre_updat
navigation_item.txt at C:\blackboard\system\database\vi\as_core\datatemplates\navigation_item.txt
[ERROR] File has changed since originally applied.
Checking manifest.txt at
C:\blackboard\system\tooldefs\system\LicenseUpdate\license-handlers\enterprise.contentsystem\database-changes\pre_updat
File has changed since originally applied.

Download Packages
Use the download action to trigger a manual download of a package from the repository. When run on an application server, the package is
saved to both the local repository replica and to the current working directory.

This command is normally only required when running the utility outside of your Blackboard Learn environment. The apply,
describe and check commands automatically download packages. See Package Repository\ for more information.

Download Command Usage

The download action takes any number of package identifiers as the only required argument:

<bbpatch-command> download LRNSI-123456 LRNSI-654321 LRN-5552

FAQs and Problem Resolution

I receive a 'Permission denied' error when running bbpatch.sh on Unix

The shell script switches to bbuser to ensure that any files touched by the utility have the appropriate permissions. Ensure that bbuser has
permission to the location where the utility was installed.

[root@localhost bbpatch]# ./bbpatch.sh

-bash: line 0: cd: /root/bbpatch: Permission denied
-bash: ./bbpatch.sh: No such file or directory

A patch includes a properties file in content\locale\en_US\messages but we use a different language pack. Will the
patch work?

Updates to language properties files indicate that the patch adds or updates a string, typically text that appears on the page. If a string is not
found in the language pack, the application falls back to the English string in the updated en_US language pack. Yes, the patch will work.
Translation is not provided for patches, however the string can be manually updated in the language pack per Locales and Language Packs
Overview. If the added strings cannot be determined, please contact Client Support for assistance.