How To

Auto Discovery and Auto Configuration in Plugins

Print

Auto Discovery and Auto Configuration in Plugins

The UrbanCode Air Plugin framework used by IBM UrbanCode Deploy has powerful auto-discovery and auto-configuration capabilities.  These were added in version 6.0. Most plugin authors don’t realize these are there. This document is designed to provide an overview of the technology and suggestions for how to apply it.

To see how discovery and configuration can work with plugins, check out this video featuring WebSphere Application Server.

Auto…. Whatnow?

There are three main capabilities of plugins being covered here:

  • Auto Discovery: An automatic discovery step is transparently run whenever an Agent Resource is added to the resource tree. The discovery process is generally used to discover information about the agent and either update its properties, or automatically generate and configure sub-resources.
  • Auto Configure: With Auto-Configure, an end user initiates a configuration process against a resource to configure it for deployments. Generally, the user passes in additional information that helps get more information. If Discovery determines an app server is installed, Configuration may be given admin credentials so that it can determine which apps are currently installed on which ports.
  • Resource Role: A resource role is basically a property sheet. Typically they are attached to discovered resources listing information that is relevant or useful at deployment time. The contents of the resource role may be populated by Discovery, Configuration or manually.

Reference Example

To see a production quality example of the discovery capabilities, download and unzip the Websphere Application Deployment plugin, and unzip it.

For the purposes of this document, we’ll use a simpler example. We will be making changes to improve the user experience for the Apache Tomcat plugin. If we look at a typical Tomcat step, we see that a few pieces of information are needed to configure it properly.

Disclaimer: The sample code provided is probably bad, poorly styled and incomplete. It was used to explore the capabilities of Discovery, not to build production code.

empty-values-2empty-values-1We need to know where Tomcat is installed, what URL it runs on, username and password information. If we can discovery any of this, we can make it so the designer of a process doesn’t need to fill any of this in.

Update the Header

We will be using schema namespaces and product features that are new in version 6.0.0 of UrbanCode Deploy. So we need to make sure the plugin is configured to use those, and prevent installation into prior versions.

So we update the first line of xml from:

<plugin xmlns="http://www.urbancode.com/PluginXMLSchema_v1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">

to:

<plugin xmlns="http://www.urbancode.com/PluginXMLSchema_v1" xmlns:server="http://www.urbancode.com/PluginServerXMLSchema_v1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">

Because this plugin also takes advantage of the optional property syntax new in 6.0.1, we set the minimum version to be 6.0.1.0.

<header>
….
<server:required-server-version>6.0.1.0</server:required-server-version>
</header>

Add Discovery

Now we can worry about some automation.  Auto-configuration steps are designed just like other steps. However, since they are not setup, they generally don’t take many inputs. Those that they have look for existing properties or values. A discovery step is marked as such by adding the “type” tag from the server namespace:

<server:type>AUTO_DISCOVERY</server:type>

I copied from the WebSphere example, and created resources using the UrbanCode Deploy Java rest client by including it in a “lib” folder in the plugin and adding it to the classpath. My full step definition is:

<step-type name="Tomcat Discovery">
    <description>This step will discover if Apache Tomcat is on an agent by checking common installation paths. 
       If it is, it will assign the Tomcat Server role to the resource and set the tomcat home property.
    </description>
    <properties>
      <property name="resourcePath">
        <server:property-ui type="textBox" default-value="${p:resource.path}" label="Resource" 
           description="The Resource to configure." hidden="true"/>
      </property>
      <property name="pathOverride">
        <server:property-ui type="textBox" default-value="${p?:agent/tomcat.path}"/>
      </property>
    </properties>
    <post-processing><![CDATA[
        if (properties.get("exitCode") != 0) {
            properties.put(new java.lang.String("Status"), new java.lang.String("Failure"));
        }
        else {
            properties.put("Status", "Success");
        }
     ]]></post-processing>
    <command program="${GROOVY_HOME}/bin/groovy">
      <arg value="-cp"/>
      <arg path="classes:lib/uDeployRestClient.jar:lib/jettison-1.1.jar"/>
      <arg file="discover.groovy"/>
      <arg file="${PLUGIN_INPUT_PROPS}"/>
      <arg file="${PLUGIN_OUTPUT_PROPS}"/>
    </command>
      <server:type>AUTO_DISCOVERY</server:type>
  </step-type>

Discover.groovy will be run on any new agent added somewhere to the resource tree. It can do pretty much anything (see “Uses for Discovery” below). The core of my routine checks common installation paths for tomcat. It then assumes poor security configuration (yes this is just an example) and grabs credentials for managing the server out of the tomcat-users.xml file. If getting key information like this isn’t doable without additional knowledge from an administrator, you should use an auto-configure step to gather the information rather than the auto-discovery.

Here’s the relevant bit of code (about 50 of the 150 lines of code in my step):

code

If you are went through the the IBM UrbanCode Deploy Learning Circle Lab Workbook you have a good test environment for this: a single virtual machine with two Tomcat instances representing integration and user acceptance test environments. Adding the agent to the tree as “My Demo Box” results in two sub-resources being created automatically:

my-demo-box

Each is also configured with its path, the name of a user in the manager role as well as its password. The plugin hasn’t discovered the manager URL, so that remains blank.

Configured Resource

Resource Roles

You may have noticed that the screenshot shows the properties within a window titled “Role Properties: Tomcat”. In the list above, this Tomcat role was added to the resource using the line:

addRoleToResource(tomcatResource, roleName, tomcatProps);

While there’s some wrapper code, the core of what’s going on there is a call to the UrbanCode Deploy REST library:

def weburl = System.getenv("AH_WEB_URL");
def user = apTool.getAuthTokenUsername();
def password = apTool.getAuthToken();
def finalURI = new URI(weburl);
def udClient = new ResourceClient(finalURI, user, password);
…
udClient.setRoleOnResource(resource.toString(), roleName, properties);

The Resource Role itself is defined within the plugin.xml file. You can have as many of these as you want.  The WebSphere Plugin I used as an example has five roles covering Cells, Nodes, Servers, Clusters and Portals.

For the this Tomcat example, I just used one:

  <server:property-group name="Tomcat" type="ResourceRole" specialType="AUTO_DISCOVERY">
    <server:description>
       Role for a resource that represents a Tomcat application server instance.
    </server:description>
    <server:property name="manager.url">
       <server:property-ui type="textBox" label="Manager URL" description="The URL to the Tomcat Manager tool."/>
    </server:property>
    <server:property name="manager.username">
       <server:property-ui type="textBox" label="Manager Username" description="The Username used to log in to the Tomcat Manager tool."/>
    </server:property>
    <server:property name="manager.password">
       <server:property-ui type="secureBox" label="Manager Password" description="The Password used to log in to the Tomcat Manager tool."/>
    </server:property>
    <server:property name="tomcat.home">
       <server:property-ui type="textBox" label="Tomcat Install Directory" description="The tomcat home directory."/>
    </server:property>
  </server:property-group>

The “specialType” attribute can be used to flag a resource role for use with auto-discovery.

Debugging Discovery

When your step is failing during development it can be a little tricky to find the logs if you  don’t know where to look. Eventually I found them on the main server, in the <main server directory>/logs/autoDiscovery/ directory. Each agent will get its own subdirectory under there when discovery is run.

Auto-Configure

Auto-configure steps are executed against existing resources with a given role. To define them create a new step. You’ll need to use a role tag to associate the configuration with a resource role the discovery can work against and the type role to flag the step as auto-configure.

Any example definition would be:

<step-type name="Tomcat Auto-Configure">
    <description>This dummy step would further configure Tomcat</description>
    <properties>
      <property name="moreInformation">
        <server:property-ui type="textBox" default-value=""
                label="Path to check" description="The path to the thing"/>
      </property>
    </properties>
    <post-processing><![CDATA[
        if (properties.get("exitCode") != 0) {
            properties.put(new java.lang.String("Status"),
                           new java.lang.String("Failure"));
        }
        else {
            properties.put("Status", "Success");
        }
     ]]></post-processing>
    <command program="${GROOVY_HOME}/bin/groovy">
      <arg value="-cp"/>
      <arg path="classes:lib/uDeployRestClient.jar:lib/jettison-1.1.jar"/>
      <arg file="configure.groovy"/>
      <arg file="${PLUGIN_INPUT_PROPS}"/>
      <arg file="${PLUGIN_OUTPUT_PROPS}"/>
    </command>
    <server:role>Tomcat</server:role>
    <server:type>AUTO_CONFIGURE</server:type>
  </step-type>

Now, we looking at subresources at any level of the resource tree, we have the option to run the discovery process against them and it will prompt us for what whatever additional information the step requires in its non-hidden properties.

auto-configure-now-an-option

Updating the Steps to Exploit Discovered Information

Our original goal was to streamline configuration. So far, we would still require users to go into each Tomcat step they add to a process and reference the new properties we added. Instead, we’ll update the both the plugin definition and the upgrade rules to put default values in anywhere blank.

So if we look at the “Start Application” step, we would find it like this:

 <step-type name="Start Application">
    <description>Start an already deployed web application on Tomcat.</description>
    <properties>
      <property name="tomcatManagerUrl" required="true">
        <property-ui type="textBox" label="Tomcat Manager URL" description="The URL of the Tomcat Manager interface."/>
      </property>
      <property name="tomcatUsername" required="true">
        <property-ui type="textBox" label="Tomcat Manager User Name" description="The user name to use to login to the Tomcat Manager interface."/>
      </property>
      <property name="tomcatPassword" required="true">
        <property-ui type="secureBox" label="Tomcat Manager Password" description="The password to use to login to the Tomcat Manager interface."/>
      </property>
      <property name="tomcatContext" required="true">
        <property-ui type="textBox" label="Context Name" description="The context that is being deployed."/>
      </property>
    </properties>
    ......

And upgrade it with default values by inserting a “default” attribute. I’m using the ${p?…} syntax here. The question mark instructs UCD to return an empty string as the property result if the property is not set. Without the question mark, the literal value of ${p:…} is returned if the property doesn’t exist.

 <step-type name="Start Application">
    <description>Start an already deployed web application on Tomcat.</description>
    <properties>
      <property name="tomcatManagerUrl" required="true">
        <property-ui type="textBox" default="${p?:resource/manager.url} label="Tomcat Manager URL" description="The URL of the Tomcat Manager interface."/>
      </property>
      <property name="tomcatUsername" required="true">
        <property-ui type="textBox" default="${p?:resource/manager.username} label="Tomcat Manager User Name" description="The user name to use to login to the Tomcat Manager interface."/>
      </property>
      <property name="tomcatPassword" required="true">
        <property-ui type="secureBox" default="${p?:resource/manager.password} label="Tomcat Manager Password" description="The password to use to login to the Tomcat Manager interface."/>
      </property>
      <property name="tomcatContext" required="true">
        <property-ui type="textBox" label="Context Name" description="The context that is being deployed."/>
      </property>
    </properties>
    ....

The result is that someone using this step only has to supply the application name or context to use it. The rest of the settings are already there and looked up automatically. However, they can override the values if they want to. If you want to discourage overriding discovered configuration, flag the discovered properties as “hidden” to bury them under the “show hidden properties” checkbox.

2014-04-16_1516

Uses for Discovery

Discovery can be used in a number of ways. Both my Apache Tomcat example and the IBM WebSphere examples find installations of applications servers and create resources accordingly.

It could also be used to check common installation paths for a testing tool. If found, record the installation path and tag the Agent as useful for running tests from.

Another option would be to check the server to determine its hostname and use that to query an in-house CMDB to configure the agent with.

Special thanks to the UrbanCode team in the Cleveland lab, especially Kensie Sturdevant, for answering all my  questions about these capabilities.