About REST APIs and API Descriptors

ForgeRock Logo This post briefly describes the types of HTTP APIs available through the ForgeRock platform, and which ones come with live reference documentation.

The following categories of HTTP APIs are available in the ForgeRock platform:

ForgeRock Common REST (CREST) APIs

ForgeRock Common REST provides a framework for HTTP APIs. Each of the component products in the platform uses CREST to build APIs that do CRUDPAQ operations in the same ways.

ForgeRock platform component products generate live reference documentation in a standard format (Swagger, which has been standardized as OpenAPI) for CREST APIs. This is done through a mechanism referred to as API descriptors. You can use this documentation to try out the CREST APIs.

Standard HTTP APIs such as OAuth 2.0

Standard HTTP APIs are defined by organizations like the IETF for OAuth 2.0, the Kantara Initiative for UMA, and the OpenID Connect Working Group. These APIs have their own implementations and do not use CREST. They are documented where they are used in the product documentation.

The canonical documentation is the specifications for the standards. At present, the ForgeRock platform components do not generate live documentation for these standard APIs.

Non-RESTful, Challenge-Response HTTP APIs

Some APIs, such as the authentication API used in ForgeRock AM and the user self-service API used in ForgeRock IDM are not fully RESTful. Instead, they use challenge-response mechanisms that have the developer return to the same endpoint with different payloads during a session.

These APIs are documented in the product documentation.

The ForgeRock API reference documentation published with the product docs is, necessarily, abstract. It does not provide you a sandbox to try out the APIs. Unlike a SaaS, with its fixed configuration, the ForgeRock platform components are highly configurable. ForgeRock HTTP APIs depend on how you decide to configure each service.

Live Reference Documentation

It is your software deployment or SaaS, built with the ForgeRock platform, that publishes concrete APIs.

You can capture the OpenAPI-format docs, and edit them to correspond to the APIs you actually want to publish. A browser-based, third-party application, Swagger UI, makes it easy to set up a front end to a sandbox service so your developers can try out your APIs.

Note that you still need to protect the endpoints. In particular, prevent developers from using the endpoints you do not want to publish.

The following posts in this series will look at how to work with the APIs when developing your configuration, and how to get the OpenAPI-format descriptors to publish live API documentation for your developers.

Storing OpenDJ server keys on the Nitrokey HSM

ForgeRock Logo The Nitrokey HSM provides a PKCS#11 hardware security module the form of a USB key. The design is based on open hardware and open software.

This is a low cost option to familiarize yourself with an actual hardware HSM, and to test your procedures. With it, you can demonstrate that OpenDJ servers can in fact use the HSM as a key store.

In addition to the documentation that you can access through https://www.nitrokey.com/start, see https://raymii.org/s/articles/Get_Started_With_The_Nitrokey_HSM.html for a helpful introduction.

The current article demonstrates generating and storing keys and certificates on the Nitrokey HSM, and then using they keys to protect OpenDJ server communications. It was tested with a build from the current master branch. Thanks to Fabio Pistolesi and others for debugging advice.

This article does not describe how to install the prerequisite tools and libraries to work with the Nitrokey HSM on your system. The introduction mentioned above briefly describes installation on a couple of Linux distributions, but the software itself seems to be cross-platform.

When you first plug the Nitrokey HSM into a USB slot, it has PINs but no keys. The following examples examine the mostly empty Nitrokey HSM when initially plugged in:

# List devices:
$ opensc-tool --list-readers
# Detected readers (pcsc)
Nr. Card Features Name
0 Yes Nitrokey Nitrokey HSM (010000000000000000000000) 00 00

# List slots, where you notice that the Nitrokey HSM is in slot 0 on this system:
$ pkcs11-tool --list-slots
Available slots:
Slot 0 (0x0): Nitrokey Nitrokey HSM (010000000000000000000000) 00 00
 token label : SmartCard-HSM (UserPIN)
 token manufacturer : www.CardContact.de
 token model : PKCS#15 emulated
 token flags : rng, login required, PIN initialized, token initialized
 hardware version : 24.13
 firmware version : 2.5
 serial num : DENK0100751

The following example initializes the Nitrokey HSM, using the default SO PIN and a user PIN of 648219:

$ sc-hsm-tool --initialize --so-pin 3537363231383830 --pin 648219
Using reader with a card: Nitrokey Nitrokey HSM (010000000000000000000000) 00 00
Version : 2.5
Config options :
 User PIN reset with SO-PIN enabled
SO-PIN tries left : 15
User PIN tries left : 3

The following example tests the PIN on the otherwise empty Nitrokey HSM:

$ pkcs11-tool --test --login --pin 648219
Using slot 0 with a present token (0x0)
C_SeedRandom() and C_GenerateRandom():
 seeding (C_SeedRandom) not supported
 seems to be OK
Digests:
 all 4 digest functions seem to work
 MD5: OK
 SHA-1: OK
 RIPEMD160: OK
Signatures (currently only RSA signatures)
Signatures: no private key found in this slot
Verify (currently only for RSA):
 No private key found for testing
Unwrap: not implemented
Decryption (RSA)
No errors

The following example generates a key pair on the Nitrokey HSM:

$ pkcs11-tool \
 --module opensc-pkcs11.so \
 --keypairgen --key-type rsa:2048 \
 --id 10 --label server-cert \
 --login --pin 648219
Using slot 0 with a present token (0x0)
Key pair generated:
Private Key Object; RSA
  label: server-cert
  ID: 10
  Usage: decrypt, sign, unwrap
Public Key Object; RSA 2048 bits
  label: server-cert
  ID: 10
  Usage: encrypt, verify, wrap

The following examples show what is on the Nitrokey HSM:

$ pkcs15-tool --dump
Using reader with a card: Nitrokey Nitrokey HSM (010000000000000000000000) 00 00
PKCS#15 Card [SmartCard-HSM]:
 Version : 0
 Serial number : DENK0100751
 Manufacturer ID: www.CardContact.de
 Flags :

PIN [UserPIN]
 Object Flags : [0x3], private, modifiable
 ID : 01
 Flags : [0x812], local, initialized, exchangeRefData
 Length : min_len:6, max_len:15, stored_len:0
 Pad char : 0x00
 Reference : 129 (0x81)
 Type : ascii-numeric
 Path : e82b0601040181c31f0201::
 Tries left : 3

PIN [SOPIN]
 Object Flags : [0x1], private
 ID : 02
 Flags : [0x9A], local, unblock-disabled, initialized, soPin
 Length : min_len:16, max_len:16, stored_len:0
 Pad char : 0x00
 Reference : 136 (0x88)
 Type : bcd
 Path : e82b0601040181c31f0201::
 Tries left : 15

Private RSA Key [server-cert]
 Object Flags : [0x3], private, modifiable
 Usage : [0x2E], decrypt, sign, signRecover, unwrap
 Access Flags : [0x1D], sensitive, alwaysSensitive, neverExtract, local
 ModLength : 2048
 Key ref : 1 (0x1)
 Native : yes
 Auth ID : 01
 ID : 10
 MD:guid : b4212884-6800-34d5-4866-11748bd12289

Public RSA Key [server-cert]
 Object Flags : [0x0]
 Usage : [0x51], encrypt, wrap, verify
 Access Flags : [0x2], extract
 ModLength : 2048
 Key ref : 0 (0x0)
 Native : no
 ID : 10
 DirectValue : 

$ pkcs15-tool --read-public-key 10
Using reader with a card: Nitrokey Nitrokey HSM (010000000000000000000000) 00 00
-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAlxvdwL6PyRnOs58L0X8d
2z8/WcgA/beoR+p08nymN8KZ4KlWKUo93AKMcFBUW8Bl8zFC80P9ZlNIXM8NSmPr
cBR9Nmpi0nUQDgfTi8vIU51tD84UcYetxX9rSHbh+CKqUmmSk6f7JPIyT6RonrOo
QJQyFmIi4oV9/d0Op8WVCbL7omYaPFwYbdUPetM1MfVyLNpkhzVdvZJE0F46hXF8
Sspqjh4f9KkJWdozIOND8ZTFvxP5Cs1y/kvvuhfjWVAtii52E4LKXRr53SA5Spl2
v1oNu5sqoaEd/SNxjj/52iH6zeGm61I7wbcIgvcHCI5CKONKceSL3PkIYzHeJMu2
SQIDAQAB
-----END PUBLIC KEY-----

The following example self-signs a public key certificate and writes it to the Nitrokey HSM. The example uses openssl, and configures an engine to use the Nitrokey HSM, which implements PKCS#11. The configuration for the OpenSSL engine is stored in a file called hsm.conf. On an Ubuntu 17.04 laptop, the PKCS#11 library installed alongside the tools is /usr/lib/x86_64-linux-gnu/opensc-pkcs11.so as shown below:

$ cat hsm.conf
# PKCS11 engine config
openssl_conf = openssl_def

[openssl_def]
engines = engine_section

[req]
distinguished_name = req_distinguished_name

[req_distinguished_name]
# empty.

[engine_section]
pkcs11 = pkcs11_section

[pkcs11_section]
engine_id = pkcs11
dynamic_path = /usr/lib/x86_64-linux-gnu/openssl-1.0.2/engines/libpkcs11.so
MODULE_PATH = /usr/lib/x86_64-linux-gnu/opensc-pkcs11.so
PIN = 648219
init = 0

# Check the engine configuration. In this case, the PKCS11 engine loads fine:
$ OPENSSL_CONF=./hsm.conf openssl engine -tt -c(rdrand) Intel RDRAND engine
 [RAND]
     [ available ]
(dynamic) Dynamic engine loading support
     [ unavailable ]
(pkcs11) pkcs11 engine
 [RSA]
     [ available ]

# Create a self-signed certificate and write it to server-cert.pem.
# Notice that the key is identified using slot-id:key-id:
$ OPENSSL_CONF=./hsm.conf openssl req \
 -engine pkcs11 -keyform engine -new -key 0:10 \
 -nodes -days 3560 -x509 -sha256 -out "server-cert.pem" \
 -subj "/C=FR/O=Example Corp/CN=opendj.example.com"
engine "pkcs11" set.
No private keys found.

The openssl command prints a message, “No private keys found.” Yet, it still returns 0 (success) and writes the certificate file:

$ more server-cert.pem
-----BEGIN CERTIFICATE-----
MIIC/jCCAeYCCQD1SEBmUy8aCzANBgkqhkiG9w0BAQsFADBBMQswCQYDVQQGEwJG
UjEVMBMGA1UECgwMRXhhbXBsZSBDb3JwMRswGQYDVQQDDBJvcGVuZGouZXhhbXBs
ZS5jb20wHhcNMTcwODE0MTEzNzA0WhcNMjcwNTE0MTEzNzA0WjBBMQswCQYDVQQG
EwJGUjEVMBMGA1UECgwMRXhhbXBsZSBDb3JwMRswGQYDVQQDDBJvcGVuZGouZXhh
bXBsZS5jb20wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCXG93Avo/J
Gc6znwvRfx3bPz9ZyAD9t6hH6nTyfKY3wpngqVYpSj3cAoxwUFRbwGXzMULzQ/1m
U0hczw1KY+twFH02amLSdRAOB9OLy8hTnW0PzhRxh63Ff2tIduH4IqpSaZKTp/sk
8jJPpGies6hAlDIWYiLihX393Q6nxZUJsvuiZho8XBht1Q960zUx9XIs2mSHNV29
kkTQXjqFcXxKymqOHh/0qQlZ2jMg40PxlMW/E/kKzXL+S++6F+NZUC2KLnYTgspd
GvndIDlKmXa/Wg27myqhoR39I3GOP/naIfrN4abrUjvBtwiC9wcIjkIo40px5Ivc
+QhjMd4ky7ZJAgMBAAEwDQYJKoZIhvcNAQELBQADggEBAIX0hQadHtv1c0P7ObpF
sDnIWMRVtq0s+NcXvMEwKhLHHvEHrId9ZM3Ywn0P3CFd+WXMWMNSVz51cPn6SKPS
pdN9CVq6B26cFrvzWrsD06ohP6jCkXEBSshlE/k71FcnukBuJHNzj8O6JMwfyWeE
xT53WIvgz9t02B/ObZSYFlUNX+WApPCbILTHazEzYws3AN0hZPmv4Ng1Vt71nNiT
EZskTxvKsmuEuG5E8j79zO0TYvOrGCISzS3PFRrl7G83vNaSyzBIhTYF2Ilt2g7B
jnNc1/k8R/TXwskJR8gL7EFZyakQ6xUiboFDf6PWa4KMLJVNX5HsVGyLvP9FiVkY
82A=
-----END CERTIFICATE-----

The following example writes the certificate to the Nitrokey HSM:

# Transform the certificate to binary format:
$ openssl x509 -in server-cert.pem -out server-cert.der -outform der

# Write the binary format to the Nitrokey HSM, with the label (aka alias) "server-cert":
$ pkcs11-tool \
 --module opensc-pkcs11.so \
 --login --pin 648219 \
 --write-object server-cert.der --type cert \
 --id 10 --label server-cert
Using slot 0 with a present token (0x0)
Created certificate:
Certificate Object, type = X.509 cert
  label:      Certificate
  ID:         10

With the keys and certificate loaded on the Nitrokey HSM, prepare to use it with Java programs. If the Java environment is configured to access the HSM, then you can just use it. In testing, however, where you are trying the HSM, and the Java environment is not configured to use it, you can specify the configuration:

# Edit a configuration file for Java programs to access the Nitrokey HSM:
$ cat /path/to/hsm.conf
name = NitrokeyHSM
description = SunPKCS11 with Nitrokey HSM
library = /usr/lib/x86_64-linux-gnu/opensc-pkcs11.so
slot = 0

# Verify that the Java keytool command can read the certificate on the Nitrokey HSM:
$ keytool \
  -list \
  -keystore NONE \
  -storetype PKCS11 \
  -storepass 648219 \
  -providerClass sun.security.pkcs11.SunPKCS11 \
  -providerArg /path/to/hsm.conf

Keystore type: PKCS11
Keystore provider: SunPKCS11-NitrokeyHSM

Your keystore contains 1 entry

server-cert, PrivateKeyEntry,
Certificate fingerprint (SHA1): B9:A2:88:5F:69:8E:C6:FB:C2:29:BF:F8:39:51:F6:CC:5A:0C:CC:10

An OpenDJ server needs to access the configuration indirectly, as there is no setup parameter to specify the HSM configuration file. Add your own settings to extend the Java environment configuration as in the following example:

$ cat /path/to/java.security
# Security provider for accessing Nitrokey HSM:
security.provider.10=sun.security.pkcs11.SunPKCS11 /path/to/hsm.conf

# Unzip OpenDJ server files and edit the configuration before running setup:
$ cd /path/to && unzip 

# Set the Java args to provide access to the Nitrokey HSM.
# Make opendj/template/config/java.properties writable, and edit.
# This allows the OpenDJ server to start as needed:
$ grep java.security /path/to/opendj/template/config/java.properties
start-ds.java-args=-server -Djava.security.properties=/path/to/java.security

# Set up the server:
$ OPENDJ_JAVA_ARGS="-Djava.security.properties=/path/to/java.security" \
 /path/to/opendj/setup \
 directory-server \
 --rootUserDN "cn=Directory Manager" \
 --rootUserPassword password \
 --hostname opendj.example.com \
 --ldapPort 1389 \
 --certNickname server-cert \
 --usePkcs11keyStore \
 --keyStorePassword 648219 \
 --enableStartTLS \
 --ldapsPort 1636 \
 --httpsPort 8443 \
 --adminConnectorPort 4444 \
 --baseDN dc=example,dc=com \
 --ldifFile /path/to/Example.ldif \
 --acceptLicense

To debug, you can set security options such as the following:

OPENDJ_JAVA_ARGS="-Djava.security.debug=sunpkcs11,pkcs11 -Djava.security.properties=/path/to/java.security"

The following example shows an LDAP search that uses StartTLS to secure the connection:

$ /path/to/opendj/bin/ldapsearch --port 1389 --useStartTLS --baseDn dc=example,dc=com "(uid=bjensen)" cn

Server Certificate:

User DN  : CN=opendj.example.com, O=Example Corp, C=FR
Validity : From 'Mon Aug 14 13:37:04 CEST 2017'
             To 'Fri May 14 13:37:04 CEST 2027'
Issuer   : CN=opendj.example.com, O=Example Corp, C=FR



Do you trust this server certificate?

  1) No
  2) Yes, for this session only
  3) Yes, also add it to a truststore
  4) View certificate details

Enter choice: [2]: 4


[
[
  Version: V1
  Subject: CN=opendj.example.com, O=Example Corp, C=FR
  Signature Algorithm: SHA256withRSA, OID = 1.2.840.113549.1.1.11

  Key:  Sun RSA public key, 2048 bits
modulus:
19075725396235933137769598662661614197862047561628746980441589981485944705910796672312984856468967795133561692335016063740885234669000544938180872617609018349362382746691431903457463096067521727428890407876216335060234859298584617093111442598717549413985534234195585205628275977771336192817217401466821950358077667360760303781781546092776529804134165206111430903307470063770954498312408782707671718644473532565867636087296875111917369665456339790081809729622515754260638402122026793085096606980136589008904235094266835122846140853242190316629669042978441585862504373498978113550866427439699045924980942028978028000841
  public exponent: 65537
  Validity: [From: Mon Aug 14 13:37:04 CEST 2017,
               To: Fri May 14 13:37:04 CEST 2027]
  Issuer: CN=opendj.example.com, O=Example Corp, C=FR
  SerialNumber: [    f5484066 532f1a0b]

]
  Algorithm: [SHA256withRSA]
  Signature:
0000: 85 F4 85 06 9D 1E DB F5   73 43 FB 39 BA 45 B0 39  ........sC.9.E.9
0010: C8 58 C4 55 B6 AD 2C F8   D7 17 BC C1 30 2A 12 C7  .X.U..,.....0*..
0020: 1E F1 07 AC 87 7D 64 CD   D8 C2 7D 0F DC 21 5D F9  ......d......!].
0030: 65 CC 58 C3 52 57 3E 75   70 F9 FA 48 A3 D2 A5 D3  e.X.RW>up..H....
0040: 7D 09 5A BA 07 6E 9C 16   BB F3 5A BB 03 D3 AA 21  ..Z..n....Z....!
0050: 3F A8 C2 91 71 01 4A C8   65 13 F9 3B D4 57 27 BA  ?...q.J.e..;.W'.
0060: 40 6E 24 73 73 8F C3 BA   24 CC 1F C9 67 84 C5 3E  @n$ss...$...g..>
0070: 77 58 8B E0 CF DB 74 D8   1F CE 6D 94 98 16 55 0D  wX....t...m...U.
0080: 5F E5 80 A4 F0 9B 20 B4   C7 6B 31 33 63 0B 37 00  _..... ..k13c.7.
0090: DD 21 64 F9 AF E0 D8 35   56 DE F5 9C D8 93 11 9B  .!d....5V.......
00A0: 24 4F 1B CA B2 6B 84 B8   6E 44 F2 3E FD CC ED 13  $O...k..nD.>....
00B0: 62 F3 AB 18 22 12 CD 2D   CF 15 1A E5 EC 6F 37 BC  b..."..-.....o7.
00C0: D6 92 CB 30 48 85 36 05   D8 89 6D DA 0E C1 8E 73  ...0H.6...m....s
00D0: 5C D7 F9 3C 47 F4 D7 C2   C9 09 47 C8 0B EC 41 59  \..

When using an HSM with an OpenDJ server, keep in mind the following caveats:

  • Each time the server needs to access the keys, it accesses the HSM. You can see this with the Nitrokey HSM because it flashes a small red LED when accessed. Depending on the HSM, this could significantly impact performance.
  • The key manager provider supports PKCS#11 as shown. The trust manager provider implementation does not, however, support PKCS#11 at the time of this writing, though there is an RFE for that (OPENDJ-4191).
  • The Crypto Manager stores symmetric keys for encryption using the cn=admin data backend, and the symmetric keys cannot currently be stored in a PKCS#11 module.

Faster docs

ForgeRock Logo One of the things you have asked for is to see large documents load faster on the ForgeRock BackStage docs site. We recently switched from publishing HTML documentation through the BackStage single-page app to publishing separate, static HTML with JavaScript to provide BackStage features.

This allows browsers to use progressive rendering, and start laying out the page before everything has been loaded and styled. The result is that large documents feel faster in your browser.

If you have bookmarks to published HTML, notice that we have dropped the per-chapter view of published docs. Each document is now a single HTML page. So instead of a link to /docs/product/version/book/chapter#section, target /docs/product/version/book/#section. Also notice that we have consolidated documentation sets to make information easier to find, with only one set per major or minor release. Generally this means that you only have to read one set of release notes, no matter what maintenance version you have right now.

The latest docs are the ones for version 5 of the platform:

We still publish all the same docs as before, including docs for software that is beyond the end of its service life. Please check out the updated site. Open issues there for any problems you notice.

ForgeRock 3.2.2 doc tools released

ForgeRock Logo ForgeRock doc tools 3.2.2 were released today.

This is a maintenance release, compatible with earlier 3.2.x releases.

ForgeRock doc tools 3.2.2 includes the following components:

  • forgerock-doc-maven-plugin
  • forgerock-doc-common-content
  • forgerock-doc-default-branding

This release resolves bugs and includes several improvements. For details, see the release notes.

Big thanks once again for enhancements, for identifying problems, and for help debugging.

Thanks to Chris Lee, Cristina Herraz, Joanne Henry, Lana Frost, Lori Goldman, David Goldsmith, Gene Hirayama, and Mike Jang for fixes, improvements, testing and bug reports.

Thanks also to the ForgeRock BackStage team for their help and continued improvements to release documentation.

ForgeRock repos and mirrors

ForgeRock LogoYou might already know that ForgeRock’s platform is built on open source software. We have been using Stash, now BitBucket, to host the git repos and to manage the review process.

If you want to read the code or the doc source code, you will find the canonical copies on that server. For example—there are more repos on the server, but here are some of the most popular links:

If you want to contribute a patch or a feature, sign up, login, and follow the development process. That process involves working on https://stash.forgerock.org/.

You will also find read-only mirrors (and other goodies) at GitHub, under https://github.com/ForgeRock:

Either server is fine if you just want to read the code or clone a repo.

docbkx-tools 2.0.17 is out

Congratulations to Cedric on the release of docbkx-tools 2.0.17 earlier this week.

For those of you working with DocBook and Maven, docbkx-tools provides a plugin to generate output formats (HTML, PDF, etc.) as part of the Maven build, by applying the DocBook XSL stylesheets.

The 2.0.17 release adds some improvements, including support for DocBook XSL 1.79.1.

At ForgeRock, we have been relying on docbkx-tools since 2011. The next release of our doc build plugin is using 2.0.17. Upgrade was straightforward. The only issue that still needs fixing is olink support in chunked HTML.

ForgeRock doc tools 3.2.0 released

ForgeRock Logo ForgeRock doc tools 3.2.0 were released today.

This is a minor release, compatible with earlier 3.x releases, with one small exception. See the release notes for details.

ForgeRock doc tools 3.2.0 includes the following components:

  • forgerock-doc-maven-plugin
  • forgerock-doc-common-content
  • forgerock-doc-default-branding

This release brings several improvements and squashes a few bugs.

Big thanks once again to Chris Lee for enhancements to the Bootstrap-styled HTML (the default for draft, in-progress docs to read when trying nightly builds), for identifying problems, and for help debugging.

Thanks also to Joanne Henry, Lana Frost, Lori Goldman, David Goldsmith, Gene Hirayama, and Mike Jang for testing and bug reports.

ForgeRock doc tools 2.1.5 released

ForgeRock doc tools 2.1.5 is now available. This is a maintenance release, adding an option to stop the build after pre-processing DocBook XML sources and fixing some bugs. Thanks to Gene and Chris for their fixes, and to Lana for testing.

See the release notes for details about what has changed.

You do not need to make any configuration changes to move to this maintenance release from 2.1.4, except to update the version number in your POM.

See the README for more about how to use the doc tools.

Towards automating tests for examples in docs

LogoTM_verticalbigForgeRock documentation includes many (but never enough) examples. When reading about editing configuration files, you expect to see excerpts of the configuration files. When reading a developer guide, you expect to see code samples. When following a tutorial that involves the command line, you expect to see command line examples for each step. Most of us figure things out a lot more quickly given both a good explanation and also a working example.

Trouble is, examples can go stale and break when the software changes. Unless you have a test harness, this sort of breakage happens silently. If the doc source contains only example input and output, it can also take time to set the software up in order to reproduce the conditions for the example. And yet readers hardly want to search for the relevant part of the example in a mass of scaffolding code and configuration.

Some of that work can be done behind the scenes by quality engineers. They can set up a context that allows them to test examples in the documentation, and indeed some of the quality engineers at ForgeRock like Jean-Charles and Laurent are already starting to solve the problem that way. It would be better for them and for everyone else, however, if it were a lot easier to prepare the context, and not something separate from the examples.

So it would help both to include only the salient excerpts in the doc but also to link to all the material needed to set up the software to try or to test the examples. (Of course everything must be versioned alongside the software. If OpenAM changes between versions 11 and 12, then the examples must change as well.)

With XInclude and JCite support, this is already technically possible for XML and Java samples. We use JCite in the OpenDJ LDAP SDK Developer’s Guide to quote from code samples tested as part of the build. But throughout the docs we have samples that involve neither XML nor Java code.

For all the other samples, we are adding two things to the next major release of the forgerock-doc-maven-plugin: copying arbitrary resources into the documentation, and quoting from any text-based file.

In the next major version, copying arbitrary resources will involve setting <copyResourceFiles> to true. If the resources are not under src/main/docbkx/resources, you will set this by using the <resourcesDirectory> setting. There are some notes on this in the draft README for the nightly build version of the plugin. This way docs can include long example scripts and other files, without including the entire text in docs.

Quoting from any text based file will depend on a new plugin called xcite-maven-plugin. The basic idea is as follows. In the file you want to cite, you either add nothing (if you want to quote the entire file) or you add markers (if you want to quote only part of the file). The markers are just strings. So they could be in comments, but they could also be part of the text. Then in the file where you want the quote to appear, you add a citation, à la JCite. For example: [file.txt:start marker:end marker]

Suppose file.txt looks like this:

# start marker
This is a great quote & stuff.
# end marker

Then in your book.xml file, you might have the citation:

<para>[file.txt:start marker:end marker]</para>

After you run the plugin, the quote ends up in your book.xml file:

<para>This is a great quote &amp; stuff.</para>

The examples above are overly simplified. But you can imagine how this might work to include excerpts of a long shell script that involves some setup and configuration, then a number of example commands.

More to come…