UrbanCode Deploy

UrbanCode Deploy 10 Minute Tips: Importing Component Versions – Part 1

There are several ways to import new component versions into UrbanCode Deploy each with its pros and cons.  In this article, we’ll explore this important topic and also look at some gotcha’s and ways to avoid them.
The most obvious way to import new component versions is using one of the many Source Code Plugins available either out of the box or for download from urbancode.com.  This is also the easiest way to get your deployment assets into UCD.

However, although this is really easy, it comes at a price, well several actually:

  1. You must nominate at least one agent (normally dedicated) to importing new component versions.
    • So there is a license cost straight away or you accept that the agent might also have to do other work which may slow down the import.  Also you need to consider what additional firewall ports you might need opened in this case.
  2. The component can be configured to require manual importing of a new version which means you must tell it when to do it:
    • If you forget to tell it to import or you don’t request all of the components in your application to update, you might have inconsistent versions between components.
    • This takes your time and introduces a further delay between the completion of the build and having a new version ready to deploy, slowing your CI/CD pipeline.
  3. You set the component to automatically import new versions.  A nominated agent now polls the repository every few minutes looking for any new versions to import:
    • Polling introduces a delay into your CI/CD pipeline, new versions will be delayed before they are discovered and made available.
    • When you scale up in a large enterprise with 100’s or even 1000’s of components this is not an insignificant load.  So you either need more agents or wait longer.
    • For the vast majority of poll cycles, the agent will find nothing for any given component so its wasted effort.

So, although its easy to do, just configuring the source code plugin, select your agent and tick a couple of boxes it has a cost.

Gotchas

With some of the source plugins you have to consider that the polling agent won’t know when a component version is complete, it will start downloading as soon as it sees a new version is available.  So, for example using the Versioned File System source plugin, as soon as the new directory appears and the poll cycle starts, it will start downloading.

So why should you care?  Well, if you’re still copying the files into the new directory and the agent starts downloading you might randomly get incomplete component versions because not all the files were placed or maybe even incomplete files depending on how your copy works.  To avoid this, you should create the directory someplace else on the same filesystem first.  Then you should move the folder into the correct location for the agent to discover it.  In this way the creation of the new component version becomes atomic – its either there or its not so you can avoid the possibility of it not discovering everything.

Is there a better way?

I believe there is.  I think using the udclient API to publish your build assets directly into UCD makes more sense.  There is no polling of a repository and therefore no delay introduced into your CI/CD pipeline.

You can trigger the import directly from your build if it is successful and it’s pretty easy to do.  We just add a few udclient commands to your build script.  This also gives you more control over what order things get imported in.

Some of the source code plugins do effectively the same thing. Any plugin that is a plugin to your build system/repository tool will likely push the assets into UCD in a similar manner to what is described.  eg Jenkins Plugin

There are several udclient commands we will make use of:

  • createVersion
    This creates the new version container to receive the version assets
  • addVersionFiles
    This command uploads the files into the version container.  You can add from several sources with multiple commands if you wish
  • addVersionLink
    Optionally add an http link back to your build result ensuring traceability back to your build.  You can add more than one link if you want.  You likely don’t have this option with most source code plugins
  • addVersionStatus
    Optionally add one or more UCD statuses to the new version.  Not available with all source plugins
  • setVersionProperty
    Optionally create properties values on your new version passing information about the build for example build reference.

This approach is quite flexible and optionally allows you to make use of more UCD capabilities all automatically from your build.  It also doesn’t tie up an agent and there is no delay in your CI/CD pipeline waiting for the next source import cycle.

Here are some examples of the various commands.  First we’ll create a new container for a component version and then add files to it:

[root@localhost]# udclient --weburl https://localhost:8443 --username admin --password admin createVersion -component 'test component' -name 1.0
{
  "id": "39940122-0fc5-4030-bf13-c0e3e4215167",
  "name": "1.0",
  "type": "FULL",
  "created": 1600683121371,
  "active": true,
  "archived": false,
  "sizeOnDisk": 0,
… …
  "totalSize": 0,
  "totalCount": 0
}
[root@localhost]# udclient --weburl https://localhost:8443 --username admin --password admin addVersionFiles --component  'test component' -version 1.0 --base v-files

You can perform several addVersionFiles calls if you wish.  However, once the version is deployed for the first time the contents are locked.  There are other parameters as well that allow you to be more specific about what files should be uploaded and also one for preserving the execute permissions of files if desired.  This might make your process simpler as well.

This example adds a link to my build result so that I can traverse from my component version in UCD back into my build tool.  This helps to maintain the traceability of my version back up the CI/CD pipeline.  I can add multiple links if I want.

[root@localhost]# udclient --weburl https://localhost:8443 --username admin --password admin addVersionLink --component  'test component' -version 1.0 --linkName "Build Result" --link http://mybuild/build/20200921-1123-1.htm
Operation succeeded.

Next Here I’m going to add a status to say that my build ran the code quality check tool

[root@localhost]# udclient --weburl https://localhost:8443 --username admin --password admin addVersionStatus --component  'test component' -version 1.0 --status CodeQualityCheck
Operation succeeded.

Lastly, I can add a new property or change the value of an existing one on a component version provided the version hasn’t been deployed yet.  This request requires information in a JSON file (or fed to standard input).  My file looked like this:

{
  "version": "1.0",
  "component": "test component",
  "name": "myVerProp",
  "value": "propVal",
  "isSecure": "false"
}

[root@localhost]# udclient --weburl https://localhost:8443 --username admin --password admin setVersionProperty propInfo.json
{
  "id": "174b04bd-2b31-65ec-325f-a4c540994112",
  "name": "myVerProp",
  "value": "propVal",
  "secure": false
}

Next Time

In the next part of this article, we will explore the udclient API further to discover what other features we have available for uploading new component versions.


Alan Murphy is an IBM services consultant who has worked with clients to help them adopt new tools and processes for the last 20 years. UrbanCode Deploy and DevOps, has been his focus for the last 5 years. He also develops tools to assist clients in tool adoption and blogs on an occasional basis.

Tags: