OpenIG: A quick look at decorators

openig-logo

Guillaume recently added a cool new feature for the upcoming release of OpenIG: decorators. To use decorators before the release, take a nightly build, or build OpenIG yourself.

Decorators are objects that extend what other objects can do. For example, OpenIG now has a CaptureDecorator that lets Filters and Handlers capture requests and responses as they pass through, and a TimerDecorator that logs how long it takes for Filters and Handlers to do their processing.

You configure decorators in the heap alongside your other objects.

For example, to configure a CaptureDecorator itself that also captures the entity when logging a request or response, you add a configuration object that looks like this:

{
    "name": "capture",
    "type": "CaptureDecorator",
    "config": {
        "captureEntity": true
    }
}

To add decorations to other objects, you have two options. You can either set default “decorations” for all applicable objects in the heap, or you decorate individual objects. The scope of the former is everything in the heap to which the decoration applies. The scope of the latter is only the individual object.

You configure a decoration by using its “name” as a top-level property. For example, to add a “capture” decoration to all Filters and Handlers, capturing at all possible capture points, add the “decorations” to the heap like this.

{
    "heap": {
        "objects": [
            {
                "name": "capture",
                "type": "CaptureDecorator",
                "config": {
                    "captureEntity": false
                }
            },
            ... other objects ...,
            {
                "name": "ClientHandler",
                "type": "ClientHandler"
            }
        ],
        "decorations": {
            "capture": "all"
        }
    },
    "handler": "ClientHandler"
}

The configuration shown here results in a lot more capturing than when you use a CaptureFilter. A CaptureFilter captures only the request and response where it is configured in a chain. When you add capture decorations at all capture points to all your Filters and Handlers, then the capture happens for requests and responses as they enter and leave each Filter, and for requests as they enter and responses as they leave Handlers.

Also, notice that “decorations” is a heap property, rather than a global property of the server or the route.

Once you have narrowed down what you want to observe, and find for example that you only want to capture requests as they enter the “ClientHandler” and responses as they leave the “ClientHandler”, comment out “decorations” and decorate only the “ClientHandler”.

{
    "heap": {
        "objects": [
            {
                "name": "capture",
                "type": "CaptureDecorator",
                "config": {
                    "captureEntity": false
                }
            },
            ... other objects ...,
            {
                "name": "ClientHandler",
                "type": "ClientHandler",
                "capture": [ "request", "response" ]
            }
        ],
        "_decorations": {
            "capture": "all"
        }
    },
    "handler": "ClientHandler"
}

Notice what has changed in the configuration. “ClientHandler” now has the “capture” decoration, this time with an array of capture points rather than a single string. The field “_decorations” now employs Guillaume’s famous underscore commenting convention. (When OpenIG does not recognize a JSON field name, it ignores the field. The leading _ is a nice toggle.)

Decorators are likely to take the place of older, more cumbersome ways of configuring some capabilities. The CaptureFilter is deprecated starting in the next release, for example. For more about OpenIG Decorators see the draft OpenIG Reference.

Leave a comment

Filed under Access Management

ForgeRock doc tools 2.1.5 released

ForgeRock doc tools 2.1.5 is now available. This is a maintenance release, adding an option to stop the build after pre-processing DocBook XML sources and fixing some bugs. Thanks to Gene and Chris for their fixes, and to Lana for testing.

See the release notes for details about what has changed.

You do not need to make any configuration changes to move to this maintenance release from 2.1.4, except to update the version number in your POM.

See the README for more about how to use the doc tools.

Leave a comment

Filed under Docs, Tools

OpenDJ: LDAP Controls

OpenDJ LogoLDAP controls are a standard mechanism for extending basic LDAP operations. For example, you can use a control to ask the LDAP server to sort search results before returning them, or to return search results a few at a time.

OpenDJ directory server supports a fairly long list of controls. Let’s take a look at three of them.

“Only do this if…”

The Assertion Control tells the directory server only to process the operation if a specified assertion is true for the target entry. You can specify the assertion as a filter to match.

As an example, let’s replace Babs Jensen’s street address, but only if it is the one we are expecting. Notice the assertion filter passed to the ldapmodify request. If Babs’s street address is not “500 3rd Street”, the request does not have an effect:

$ ldapmodify \
> --port 1389 \
> --bindDN uid=kvaughan,ou=people,dc=example,dc=com \
> --bindPassword bribery \
> --assertionFilter "(street=500 3rd Street)"
dn: uid=bjensen,ou=people,dc=example,dc=com
changetype: modify
replace: street
street: 33 New Montgomery Street

Processing MODIFY request for uid=bjensen,ou=people,dc=example,dc=com
MODIFY operation successful for DN uid=bjensen,ou=people,dc=example,dc=com

“Make the modification, and shut up”

The Permissive Modify Control is handy when you want to make a modification no matter what. It lets you add an attribute that already exists, or delete one that is already gone without getting an error.

As an example, let’s make sure user.0 is a member of a big static group. It doesn’t matter whether user.0 was already a member, but if not, we want to make sure user.0 is added to the group.

$ ldapmodify \
>  --port 1389 \
>  --bindDN uid=user.1,ou=people,dc=example,dc=com \
>  --bindPassword password \
>  --control 1.2.840.113556.1.4.1413
dn: cn=Static,ou=Groups,dc=example,dc=com
changetype: modify
add: member
member: uid=user.0,ou=people,dc=example,dc=com

Processing MODIFY request for cn=Static,ou=Groups,dc=example,dc=com
MODIFY operation successful for DN cn=Static,ou=Groups,dc=example,dc=com

“Delete the children, too”

The Subtree Delete Control lets you delete an entire branch of entries.

As an example, let’s delete ou=Groups,dc=example,dc=com and any groups underneath. The user doing this needs an access to use the tree delete control, as in aci: (targetcontrol="1.2.840.113556.1.4.805") (version 3.0; acl "Tree delete"; allow(all) userdn ="ldap:///uid=user.1,ou=people,dc=example,dc=com";).

$ ldapdelete \
>  --port 1389 \
>  --bindDN uid=user.1,ou=people,dc=example,dc=com \
>  --bindPassword password \
>  --deleteSubtree \
>  ou=Groups,dc=example,dc=com
Processing DELETE request for ou=Groups,dc=example,dc=com

DELETE operation successful for DN ou=Groups,dc=example,dc=com

As mentioned above, OpenDJ directory server supports many LDAP controls. So does OpenDJ LDAP SDK. If you want to use one in your application, see the Dev Guide chapter on Working With Controls.

Leave a comment

Filed under Directory Services and LDAP

Towards automating tests for examples in docs

LogoTM_verticalbigForgeRock documentation includes many (but never enough) examples. When reading about editing configuration files, you expect to see excerpts of the configuration files. When reading a developer guide, you expect to see code samples. When following a tutorial that involves the command line, you expect to see command line examples for each step. Most of us figure things out a lot more quickly given both a good explanation and also a working example.

Trouble is, examples can go stale and break when the software changes. Unless you have a test harness, this sort of breakage happens silently. If the doc source contains only example input and output, it can also take time to set the software up in order to reproduce the conditions for the example. And yet readers hardly want to search for the relevant part of the example in a mass of scaffolding code and configuration.

Some of that work can be done behind the scenes by quality engineers. They can set up a context that allows them to test examples in the documentation, and indeed some of the quality engineers at ForgeRock like Jean-Charles and Laurent are already starting to solve the problem that way. It would be better for them and for everyone else, however, if it were a lot easier to prepare the context, and not something separate from the examples.

So it would help both to include only the salient excerpts in the doc but also to link to all the material needed to set up the software to try or to test the examples. (Of course everything must be versioned alongside the software. If OpenAM changes between versions 11 and 12, then the examples must change as well.)

With XInclude and JCite support, this is already technically possible for XML and Java samples. We use JCite in the OpenDJ LDAP SDK Developer’s Guide to quote from code samples tested as part of the build. But throughout the docs we have samples that involve neither XML nor Java code.

For all the other samples, we are adding two things to the next major release of the forgerock-doc-maven-plugin: copying arbitrary resources into the documentation, and quoting from any text-based file.

In the next major version, copying arbitrary resources will involve setting <copyResourceFiles> to true. If the resources are not under src/main/docbkx/resources, you will set this by using the <resourcesDirectory> setting. There are some notes on this in the draft README for the nightly build version of the plugin. This way docs can include long example scripts and other files, without including the entire text in docs.

Quoting from any text based file will depend on a new plugin called xcite-maven-plugin. The basic idea is as follows. In the file you want to cite, you either add nothing (if you want to quote the entire file) or you add markers (if you want to quote only part of the file). The markers are just strings. So they could be in comments, but they could also be part of the text. Then in the file where you want the quote to appear, you add a citation, à la JCite. For example: [file.txt:start marker:end marker]

Suppose file.txt looks like this:

# start marker
This is a great quote & stuff.
# end marker

Then in your book.xml file, you might have the citation:

<para>[file.txt:start marker:end marker]</para>

After you run the plugin, the quote ends up in your book.xml file:

<para>This is a great quote &amp; stuff.</para>

The examples above are overly simplified. But you can imagine how this might work to include excerpts of a long shell script that involves some setup and configuration, then a number of example commands.

More to come…

Leave a comment

Filed under Docs, Tools

OpenIG’s improved configuration files

openig-logoIn the last couple of weeks, Guillaume and Violette have introduced more slick configuration improvements into the latest nightly builds of OpenIG. To me these are the coolest features since Guillaume added routing with support for runtime configuration changes in OpenIG 3.0.0. The changes are backwards compatible, too, so you don’t have to move right away if you just want to upgrade for other reasons.

In OpenIG 3.0.0 you have to declare all your objects in the heap, and then use the “name” wherever you reference the object. Also, you have to declare the “config” on an object, even if it is empty. (You leave “config” empty when all the fields are optional, and you want default settings for everything).

{
  "heap": {
    "objects": [
      {
        "name": "Chain",
        "type": "Chain",
        "config": {
          "filters": [
            "CaptureFilter"
          ],
          "handler": "Router"
        }
      },
      {
        "name": "CaptureFilter",
        "type": "CaptureFilter",
        "config": {
          "file": "/tmp/gateway.log"
        }
      },
      {
        "name": "Router",
        "type": "Router",
        "config": {}
      },
      {
        "name": "ClientHandler",
        "type": "ClientHandler",
        "comment": "HTTP client for routes",
        "config": {}
      }
    ]
  },
  "handler": "Chain"
}

That example is small, so it is still pretty easy to see what you are doing.

If the configuration file gets any longer, it can be tough to see the relationships.

What Guillaume did is allow you to add the configuration objects inline. (In the case where the object is only used in one place in the configuration, why not just declare it right there?) Violette then made empty “config” fields optional.

Now you can write it like this:

{
  "heap": {
    "objects": [
      {
        "name": "Chain",
        "type": "Chain",
        "config": {
          "filters": [
            {
              "type": "CaptureFilter",
              "config": {
                "file": "/tmp/gateway.log"
              }
            }
          ],
          "handler": {
            "type": "Router"
          }
        }
      },
      {
        "name": "ClientHandler",
        "type": "ClientHandler",
        "comment": "HTTP client for routes",
      }
    ]
  },
  "handler": "Chain"
}

Notice how easy it is to see what’s happening in the “Chain”.

The draft, in-progress docs and examples have all been updated to use the new, inline style. Check out what this does for some of the more complicated examples.

My favorite? If the file above is your main config.json, then your default route can be succinct indeed:

{
  "heap": {
    "objects": []
  },
  "handler": "ClientHandler"
}

Leave a comment

Filed under Access Management, Docs

OpenDJ directory server & groups

OpenDJ Logo When you think about groups in general, you realize there are different ways to define groups.

You can define a group by:

  • Specifying attributes that all members have in common (“All the software developers in Grenoble”).
  • Listing the members (“Alice, Bob, Carol”).
  • Using a mix of both (all the software developers in Grenoble, plus Alice, Bob, and Carol).

Notice that when you define a group by attributes of its members, the size of the definition is not a function of the size of the group. “All the software developers in Grenoble” remains the same size, even if the number of software developers in Grenoble varies.

Also notice that when you define a group by listing its members, the size of the definition depends mainly on the size of the group. “Alice, Bob, Carol” is about half the size of “Alice, Bob, Carol, Dan, Elena, Francis.”

OpenDJ directory server has groups of each of these types.

  • OpenDJ represents groups as directory entries. The group entry’s objectClass tells you what type of group it is.
  • Dynamic groups work by specifying attributes that all members have in common. They identify members by LDAP URLs, as in memberUrl: ldap:///ou=Developers,ou=People,dc=example,dc=com??sub?l=Grenoble. This is like specifying members by LDAP search, as in ldapsearch --baseDN ou=Product Development,ou=People,dc=example,dc=com "(l=Grenoble)".
    Dynamic group entries have objectClass: groupOfUrls. Dynamic group entries generally remain small even when they have lots of members.
  • Static groups list each member. Static group entries can have several object classes in OpenDJ, including groupOfNames, groupOfUniqueNames, and groupOfEntries.
    For groups with objectClass: groupOfNames and objectClass: groupOfEntries the member attribute is member, as in

    member: uid=Alice,ou=People,dc=example,dc=com
    member: uid=Bob,ou=People,dc=example,dc=com
    member: uid=Carol,ou=People,dc=example,dc=com
    member: cn=Grenoble Developers,ou=Groups,dc=example,dc=com
    

    As you would expect, static group entries get larger as you add members.

    You can nest groups, too, by adding a group as a member of a group. (For instance, the last member in the example is the Grenoble Developers group.)

    When you delete an entry or rename an entry, you no longer want it referenced in static groups. OpenDJ has a referential integrity plugin to handle this.

  • When you look up a dynamic group in OpenDJ what you get is the memberUrl value. This is fine if you are ready to use the LDAP URL, but not so great if you just wanted a list of the member entries. OpenDJ therefore provides what are called virtual static groups. Virtual static groups have OpenDJ look up the members of a dynamic group, and then return the list when you read the entry.

Checking group membership

In general, avoid reading and writing whole static group entries, because static groups can be large. Why read an entire 1-million-member static group entry over the network when all you want is to do is check membership for one or two entries?

To make it easy to check group membership, OpenDJ provides a virtual attribute, isMemberOf. If you explicitly request isMemberOf with a search, the values OpenDJ returns are the DNs of groups the entry belongs to. Reading isMemberOf is the recommended way to look up group membership.

Updating static groups

You can update a large OpenDJ static group without reading the group, and without checking membership beforehand. Use the Permissive Modify control (--control 1.2.840.113556.1.4.1413).

If you have an application that needs notification of changes to groups, OpenDJ has you covered there as well. Instead of polling group entries, you can use the external change log mechanism. The external change log is related to directory replication.

Where to go from here

This discussion of OpenDJ groups only scratches the surface. To dig a bit deeper, have a look at these sections in the documentation:

Leave a comment

Filed under Directory Services and LDAP

New introduction to OpenIG

openig-logo Around the time OpenIG 3.0.0 released back in August, we decided that the introductory part of the main guide left much up to the reader. Indeed, it introduced all the concepts, but you had to play with OpenIG for a while until those concepts would sink in.

For those of you trying the nightly builds, notice that since then the aim has been to simplify the draft of the next Guide to OpenIG. Thanks to Christophe, Guillaume, Jean-Charles, Ludo, Matt, Simon, Violette, Warren, and all of you who have offered your suggestions and pointed out parts that needed attention.

Now the guide starts with a short chapter on Understanding OpenIG. That chapter covers the main concepts that help you get started: the Exchange that encapsulates the HTTP request and the response as well as state information, the configuration including routes, how Filters and Handlers are chained together and how the Exchange flows through a chain.

The rest of the guide aims to be hands-on & practical. From the Getting Started chapter that should have you protecting your first app in a few minutes, through Installation in Detail and a series of tutorial chapters and configuration templates, to scripting and troubleshooting, you pick up the ideas a few at a time while seeing OpenIG in action.

Once you feel pretty confident that you understand how OpenIG works, to get all the details or to find just the right tool for a particular job you can check out the draft Reference.

Please feel welcome to log issues against these and all of the docs that we are working on.

1 Comment

Filed under Access Management, Docs