Upgrade to DS 7: #1 What’s Changed

ForgeRock logo This is post 1 of a series of 3 on upgrading servers to ForgeRock Directory Services 7.

DS 7 is a major release, much more cloud-friendly than ever before, and different in significant ways from earlier releases.

To upgrade successfully, make sure you understand the key differences beforehand. With these in mind, plan the upgrade, how you will test the upgraded version, and how you will recover if the upgrade process does not go as expected:

Fully Compatible Replication

Some things never change. The replication protocol remains fully compatible with earlier versions back to OpenDJ 3.

This means you can still upgrade servers while the directory service is online, but the process has changed.

In 6.5 and earlier, you set up DS servers that did not yet replicate. Then, when enough of them were online, you configured replication.

In 7, you configure replication at setup time before you start the server. For servers that will have a changelog, you use the setup --replicationPort option for the replication server port. For all servers, you use the setup --bootstrapReplicationServer option to specify the replication servers that the server will contact when it starts up.

The bootstrap replication servers maintain information about the servers in the deployment. The servers learn about the other servers in the deployment by reading the information that the bootstrap replication server maintains. Replicas initiate replication when they contact the first bootstrap replication server.

As directory administrator, you no longer have to configure and initiate replication for a pure DS 7 deployment. DS 7 servers can start in any order as long as they initiate replication before taking updates from client applications.

Furthermore, you no longer have to actively purge servers from other servers’ configurations. The other servers “forget” a server that disappears for longer than the replication purge delay, and eventually purge its state.

These new capabilities bring you more deployment flexibility than ever before. As a trade off, you must now think about configuring replication at setup time, and you must migrate scripts and procedures that used older commands to the new dsrepl command.

Unique String-Based Server IDs

By default, DS 7 servers use unique string-based server IDs.

In prior releases, servers had multiple numeric server IDs. Before you add a new DS 7 server to a deployment of older servers, you must assign it a “numeric” server ID.

Secure by Default

The setup --production-mode option is gone. All setup options and profiles are secure by default.

DS 7 servers require:

  • Secure connections.
  • Authentication for nearly all operations, denying most anonymous access by default.
  • Additional access policies when you choose to grant access beyond what setup profiles define.
  • Stronger passwords.
    New passwords must not match known compromised passwords from the default password dictionary. Also in 7, only secure password storage schemes are enabled by default, and reversible password storage schemes are deprecated.
  • Permission to read log files.

Furthermore, DS 7 encrypts backup data by default. As a result of these changes, all deployments now require cryptographic keys.

Deployment Key Required

DS 7 deployments require cryptographic keys. Secure connections require asymmetric keys (public key certificates and associated private keys). Encryption requires symmetric (secret) keys that each replica shares.

To simplify key management and distribution, and especially to simplify disaster recovery, DS 7 uses a shared master key to protect secret keys. DS 7 stores the encrypted secret keys with the replicated and backed up data. This is new in DS 7, and replaces cn=admin data and the keys for that backend.

A deployment key is a random string generated by DS software. A deployment key password is a secret string at least 8 characters long that you choose. The two are a pair. You must have a deployment key’s password to use the key.

You generate a shared master key, and optionally, asymmetric key pairs, with the dskeymgr command using your deployment key and password. Even if you provide your own asymmetric keys for securing connections, you must use the deployment key and password to generate the shared master key.

When you upgrade, or add a DS 7 server to a deployment of pre-7 servers, you must intervene to move from the old model to the new, and unlock all the capabilities of DS 7.

New Backup

As before, backups are not guaranteed to be compatible across major and minor server releases. If you must roll back from an unsuccessful upgrade, roll back the data as well as the software.

When you back up DS 7 data, the backup format is different. The new format always encrypts backup data. The new format lets you back up and restore data directly in cloud storage if you choose.

Backup operations are now incremental by design. The initial backup operation copies all the data, incrementing from nothing to the current state. All subsequent operations back up data that has changed.

Restoring a backup no longer involves restoring files from the full backup archive, and then restoring files from each incremental backup archive. You restore any backup as a single operation.

The previous backup and restore tools are gone. In their place is a single dsbackup command for managing backup and restore operations, for verifying backup archives, and for purging outdated backup files.

Upgrade Strategies

When you upgrade to a new DS version, you choose between in-place upgrade, unpacking the new software over old, then running the upgrade command, or upgrade by adding new servers and retiring old ones.

Keep Reading

Now that you understand the key differences in DS 7, try a demo or two:

Upgrade to DS 7: #2 Add New Servers

ForgeRock logo This is post 2 of a series of 3 on upgrading servers to ForgeRock Directory Services 7.

If you’ve read the first post in this series, you understand the key differences that you need to know before upgrading.

One option when upgrading DS servers is to upgrade by adding new servers, leaving the service running during upgrade. Once you finishing adding new servers, and are satisfied with the result, you retire old servers and clean up the new servers to access the newest features.

This process makes it easier to phase out old systems, and might speed up rollback, if necessary. It involves more systems during the upgrade, requires initializing new replicas, and calls on you to reconfigure new servers to align with the older servers. You must manually enable new features after upgrade.

This demonstration installs two DS 6.5 replicas, adds three new DS 7 servers to the deployment, and then cleans up after upgrade, removing the DS 6.5 servers.

The demonstration is self-contained, and expected to run in a Bash terminal on your computer. It was written on Ubuntu 20.04 with OpenJDK 8 and 11, installing servers in /path/to. Edit the scripts, if necessary, to try the demo on your computer.

Install Two DS 6.5 Servers

Download the setup script so you can try this yourself. Before you run the setup as is, download DS 6.5 in .zip format.

The setup script installs two DS 6.5 servers that are both directory servers and replication servers. The script configures the servers with the DS evaluation profile (base DN: dc=example,dc=com), and uses Java 8. (DS 6.5 also supports Java 11, but DS 7 supports only Java 11 as of this writing).

After installation, the servers are running, and replicating with each other:

PathPortsCredentials
/path/to/ds-rs-111389 11636 14444 18443 18989cn=Directory Manager:password cn=monitor:password
/path/to/ds-rs-221389 21636 24444 28443 28989cn=Directory Manager:password cn=monitor:password
uid=Admin:password

Add Three DS 7 Servers

Before you run the script to add servers as is, download DS 7 in .zip format.

The script installs these servers that use Java 11:

  • /path/to/ds-rs-7: A DS 7 server with the evaluation profile that is a directory server and a replication server.
  • /path/to/ds-7: A standalone DS 7 directory server with the evaluation profile.
  • /path/to/rs-7: A standalone DS 7 replication server.

Server IDs

Notice that the server IDs are numbers. The existing deployment uses numeric server IDs.

New Security Model

The new servers’ keys rely on a deployment key. You generate your deployment key with the dskeymgr command and the deployment key password of your choice. Keep the deployment key password secret. The scripts use a demonstration deployment key that was generated with password as the password, and should only be used for this demonstration.

Configuration Compatibility

Before starting the new servers, the script adapts their configurations for compatibility with DS 6.5. In this case, with servers installed with the evaluation profile and no other changes, the only adaptations required are to enable the PBKDF2 and Slated SHA-512 password storage schemes. In DS 7, these password storage schemes are disabled in favor of strong default schemes.

Real-world deployments may well require additional configuration adaptations. DS 7 configures servers to be secure by default. To determine what you must do, compare your DS 6.5 configuration with a fresh DS 7 configuration. It may help to read the DS 6.5 config.ldif alongside the DS 7 config.ldif, entry by entry. This is perhaps the most difficult part of upgrading by adding servers.

Add to Deployment

The script adds the servers to the existing deployment, starts them, and initializes replication from ds-rs-1:

PathPortsCredentials
/path/to/ds-rs-731389 31636 34444 38443 38989uid=admin:password cn=monitor:password
/path/to/ds-741389 41636 44444 48443uid=admin:password cn=monitor:password
/path/to/rs-751389 51636 54444 58443 58989uid=admin:password cn=monitor:password

When initializing replication, the script overwrites DS 7 schema with 6.5 schema for compatibility.

While the upgrade is in progress, replication monitoring is split between the older servers, which use dsreplication status, and the newer servers, which use dsrepl status. Run both commands to get a more complete picture of replication status.

Remove 6.5 Servers and Clean Up

After you finish rolling out new servers, and are satisfied with the results, you retire older servers, and then clean up the configuration of your new servers. The cleanup process is optional, but recommended. It aims to bring the deployment in line with current best practices, and unlocks new features.

Removal of 6.5 Servers

The cleanup script stops and removes the two DS 6.5 servers.

Cleanup Command

Once only DS 7 servers remain, the script runs a dsrepl cleanup-migrated-pre-7-0-topology command. This is the first step of the cleanup process, limiting the dependency on the old security model based on cn=admin data and ADS keys:

Removing administrators from admin backend ..... Done
Cleaning and updating configuration of server localhost:34444 ..... Done
Cleaning and updating configuration of server localhost:54444 ..... Done
Cleaning and updating configuration of server localhost:44444 ..... Done
Removing servers from admin backend ..... Done
Removing instance keys from admin backend ..... Done

All servers have been cleaned

The command is conservative about cleanup. It does not remove cn=admin data, because the deployment depends on the secret keys in the backend to decrypt any data in the deployment that has been encrypted. This includes passwords stored with reversible password storage schemes, encrypted backends, and encrypted backup archives.

Schema Files

As described above, the previous script overwrites DS 7 schema with 6.5 schema for compatibility. The cleanup script reverses that process, replacing older configuration schema with DS 7 templates. Before you do this in production, review the differences in your schema files.

Admin Data

If you are sure that your deployment does not use encrypted data, then you can remove cn=admin data and the ADS keys as well, as shown in the script. Otherwise, re-encrypt the data before removing the old keys:

  • Deprecate password policies that use reversible storage schemes, and wait until passwords for all active users are hashed.
  • Export encrypted backends to cleartext LDIF, import with encryption based on DS 7 keys:
    • Make sure you have enough time to export and import in less than the replication purge delay.
    • Export the encrypted backend to LDIF.
    • Stop the server.
    • Remove cn=admin data and related configuration entries from the server configuration.
    • Import the backend from LDIF, overwriting existing data, and encrypting with a symmetric key protected by the shared master key.
    • Start the server.Replication online brings the server up to date for changes that happened while it was down.

Replication Status

After making the changes, the script checks replication status and displays the result.

cn=admin data can still appear in the output, even if you removed the data.

Go Further

At this point, the upgrade process is complete, and the service runs DS 7 servers. You can begin to use new DS 7 features.

You can now make additional changes to your DS 7 deployment, such as switching to string-based server IDs, or deprecating old password storage in favor of more secure options.

To stop and remove the servers you installed, you can use the teardown script.

Upgrade to DS 7: #3 In-Place Upgrade

ForgeRock logo This is post 3 of a series of 3 on upgrading servers to ForgeRock Directory Services 7.

If you’ve read the first post in this series, you understand the key differences that you need to know before upgrading.

The most straightforward option when upgrading DS servers is to upgrade in place. One by one, you stop, upgrade, and restart each server individually, leaving the service running during upgrade.

The in-place upgrade process is simpler to understand, and maintains compatibility with earlier settings. It is slower to roll back if you find a problem late in the upgrade process. You must manually enable new features after upgrade.

This demonstration installs two DS 6.5 replicas, upgrades them in place, and deploys keys required to use the new security model.

The demonstration is self-contained, and expected to run in a Bash terminal on your computer. It was written on Ubuntu 20.04 with OpenJDK 8 and 11, installing servers in /path/to. Edit the scripts, if necessary, to try the demo on your computer.

Install Two DS 6.5 Servers

Download the setup script so you can try this yourself. Before you run the setup as is, download DS 6.5 in .zip format.

The setup script installs two DS 6.5 servers that are both directory servers and replication servers. The script configures the servers with the DS evaluation profile (base DN: dc=example,dc=com), and uses Java 8. (DS 6.5 also supports Java 11, but DS 7 supports only Java 11 as of this writing).

After installation, the servers are running, and replicating with each other:

PathPortsCredentials
/path/to/ds-rs-111389 11636 14444 18443 18989cn=Directory Manager:password cn=monitor:password
/path/to/ds-rs-221389 21636 24444 28443 28989cn=Directory Manager:password cn=monitor:password
uid=Admin:password

Upgrade in Place

Before you run the upgrade and cleanup script as is, download DS 7 in .zip format.

In a rolling upgrade, the script stops each server, unpacks the new software over the old, and updates the configuration to use Java 11 before running the upgrade command.

To the extent possible, the upgrade command does not change the server configuration, or the LDAP schema. It aims to preserve compatibility, letting you make configuration changes later at your convenience.

On Cleanup

After you finish rolling out new servers, and are satisfied with the results, you retire older servers, and then clean up the configuration of your new servers. The cleanup process is optional, but recommended. It aims to bring the deployment in line with current best practices, and unlocks new features.

Schema Files

The cleanup process replaces older configuration schema with DS 7 templates. In this case, one of the newer schema files lets you use fully featured and replicated password policies. Before you do this in production, review the differences in your schema files.

New Security Model

When you install a new DS 7 server, you use a deployment key and password to derive at least a shared master key. The shared master key protects secret keys, which are then distributed in the replicated data they encrypt. When you upgrade, however, the servers carry on using the ADS instance key to protect secret keys, and replication of cn=admin data to distribute the keys.

The script adds keys based on a deployment key and password, and changes the server configurations to use them. You generate your deployment key with the dskeymgr command and the deployment key password of your choice. Keep the deployment key password secret. The scripts use a demonstration deployment key that was generated with password as the password, and should only be used for this demonstration.

Cleanup Command

When only DS 7 servers using the new security model remain, the script starts the servers, and runs a dsrepl cleanup-migrated-pre-7-0-topology command. This is only part of the cleanup process, limiting the dependency on the old security model based on cn=admin data and ADS keys.

The command is conservative about cleanup. It does not remove cn=admin data, because the deployment depends on the secret keys in the backend to decrypt any data in the deployment that has been encrypted. This includes passwords stored with reversible password storage schemes, encrypted backends, and encrypted backup archives.

Admin Data

If you are sure that your deployment does not use encrypted data, then you can remove cn=admin data and the ADS keys as well, as shown in the script. Otherwise, reencrypt the data before removing the old keys:

  • Deprecate password policies that use reversible storage schemes, and wait until passwords for all active users are hashed.
  • Export encrypted backends to cleartext LDIF, import with encryption based on DS 7 keys:
    • Make sure you have enough time to export and import in less than the replication purge delay.
    • Export the encrypted backend to LDIF.
    • Stop the server.
    • Remove cn=admin data and related configuration entries from the server configuration.
    • Import the backend from LDIF, overwriting existing data, and encrypting with a symmetric key protected by the shared master key.
    • Start the server.Replication online brings the server up to date for changes that happened while it was down.

Replication Status

After making the changes, the script checks replication status and displays the result.

cn=admin data can still appear in the output, even if you removed the data.

Go Further

At this point, the upgrade process is complete, and the service runs DS 7 servers. You can begin to use new DS 7 features.

You can now make additional changes to your DS 7 deployment, such as switching to string-based server IDs, or deprecating old password storage in favor of more secure options.

To stop and remove the servers you installed, you can use the teardown script.

ForgeRock Identity Platform 5.0 docs

ForgeRock Logo By now you have probably read the news about the ForgeRock Identity Platform 5.0 release.

This major platform update comes with many documentation changes and improvements:

Hope you have no trouble finding what you need.

User-Managed Access from ForgeRock

ForgeRock LogoLast week ForgeRock released a platform update, including many new features. One of those features is support for User-Managed Access (UMA). UMA is a profile of OAuth 2.0 that allows resource owners to share their resources with others in a standard way. UMA puts resource owners in charge of defining policies for accessing their resources. From that perspective, UMA could be seen as highly scalable delegated policy administration.

At a very high level, this short video on privacy introduces the ideas it.

At a level closer to the implementation, UMA describes how authorization servers, resource servers, and UMA clients interact to enable resource sharing. In the ForgeRock platform, OpenAM plays the role of authorization server. OpenIG plays the role of resource server. (These are currently the working parts of OpenUMA.) In the present implementation, the client side is for you to implement, although the OpenIG docs include an example client, and there are ForgeRock demos that cover the client side as well. You can download OpenAM 13 and OpenIG 4 from the ForgeRock BackStage downloads page.

To get started with OpenAM authorization server implementation, install OpenAM and then have a look at the Administration Guide chapter on Managing UMA Authorization.

To try OpenIG’s UMA resource server capabilities, install OpenIG and then have a look at the OpenIG Gateway Guide chapter on OpenIG As an UMA Resource Server.

The latter chapter is a tutorial that describes how to get everything working together including a minimalist, browser-based UMA client.

 

ForgeRock doc tools 3.1.0 released

ForgeRock doc tools 3.1.0 are out.

This is a minor release, compatible with 3.0.0. See the release notes for details.

ForgeRock doc tools 3.1.0 includes the following components:

  • forgerock-doc-maven-plugin
  • forgerock-doc-common-content
  • forgerock-doc-default-branding
  • forgerock-doc-maven-archetype

This release adds a few improvements and resolves a number of bugs.

One of the improvements is initial support for Asciidoc. The doc build plugin generates DocBook from Asciidoc source, and then processes the resulting output in the same way as other documents. At this time the doc build plugin does not allow you to mix Asciidoc and DocBook in the same document. For details, see the README.

Thanks to Peter Major for providing a new release of docbook-linktester, improving the link check usability with a more human-readable report, better supporting <olink> elements, and troubleshooting an issue related to throttling that affected link checks for some documents.

Thanks again to Chris Lee for a number of improvements to Bootstrap HTML output, and for fixing inter-document links in PDF (depends on the renderer, seen to work with Adobe Acrobat).

Thanks also to Lana Frost, Chris Clifton, David Goldsmith, Gene Hirayama, and Mike Jang for testing and bug reports.

ForgeRock doc tools 3.0.0 released

ForgeRock doc tools 3.0.0 is finally done!

This is a major release, and the build plugin configuration has changed. See the release notes for details.

ForgeRock doc tools 3.0.0 includes the following components:

  • forgerock-doc-maven-plugin
  • forgerock-doc-common-content
  • forgerock-doc-default-branding
  • forgerock-doc-maven-archetype

This release resolves 92 issues, with dozens of new features, fixes, and improvements.

Hats off to Chris Lee for his work to provide much better HTML, styled with Bootstrap, and to Gene Hirayama for his many improvements to PDFs.

Thanks also to Lana Frost, David Goldsmith, and Mike Jang for testing and bug reports.

Special thanks to Peter Major for docbook-linktester 1.3.0.

See the README for more about how to use the doc tools, and for details on the new features.

Time that won’t be wasted

Duke Nukem cover
Source: Wikipedia

Okay… this has nothing to do with the rest of the posts here… but we’ve been waiting so long, only to be disappointed.

A long time ago, back when Quake and even Doom seemed cool, there was a first-person-shooter that combined playability with humor so bad it was actually good. I’m talking about Duke Nukem 3D.

What I enjoyed about the jokes and sci-fi references was how the game went so over the top it seemed actually to make fun of itself, thus neutralizing the offensiveness. Duke Nukem 3D even seemed Kafkaesque at times, though in Fletch kind of way.

Looks like the one of the most delayed remakes in history is not worth the wait. So even if I did have one of the platforms it runs on, I would not be rushing to download a copy.

Gee, maybe I’ll have to read a book instead. Or write one.