UrbanCode Deploy

10 minute tip: Migrating Assets from one server to another

There are occasions when it may be desirable to migrate assets from one UCD server instance to another.  This might happen for example if you have a separate instance to develop major changes to say a component template which would be potentially too disruptive to normal operations if it was done on the live system.

Under normal circumstances, though you would only have a single UCD instance for an entire environment pipeline.  See my article on the reasons for this:

How Many Instances of UrbanCode Deploy (UCD) Should I Have?” 

UCD contains export / import functions to allow you to migrate some assets between UCD servers and of course you could also export / import into the same server if you wanted to.  You might do this to create a copy of say an application to experiment on.  You could export the application, edit the resulting JSON to change the name of the application and perhaps its components to create an independent copy.

Alter adding an app
Quite often you will see alerts similar to this one.

 

In older versions of UCD you may see a less friendly message, but it means the same thing.  So why does this arise?

Representation of Secure Fields

If you create secure fields, when they are exported they are saved in the export file as encrypted values, like this:

The important parts of this are:

  • the encryption key alias.  In this example that’s
    aes128key7p9j
  • the encrypted value:
    llWZgVd2Sv1X2zBaoJc6Aw==|AcMeneArr+GGMbgGdLsxyw==

To be able to import this exported application, I need the encryption key of the UCD server that the application was exported from.

But what happens if that key isn’t available to you?  Maybe somebody shared the application export and they can’t or won’t share their server encryption key.  Well, all is not lost.

I could remove the complete encrypted construct from the file.  I can then import it, but of course, I’ve lost the original secure value.  But that may not be an issue.  To remove the encrypted phrase entirely, find all the values that look similar to this

“crypt_v1{AES\/CBC\/PKCS5Padding|aes128key7p9j|llWZgVd2Sv1X2zBaoJc6Aw==
|AcMeneArr+GGMbgGdLsxyw==}”

And replace them with “”.  So, the field in the JSON would then look like this:

“password”: “”

Another thing you could do if you know the password you want to be used is to put the required password in clear text into the import file.  The field in the JSON would then look like:

“password”: “myPassword”

UCD will take the plaintext password myPassword and encrypt it in the database using the importing servers encryption key.  Of course, you would have to be careful to dispose of your import file properly.

This would save you having to find each secure property through the GUI and editing the property value.

There could, be dozens and dozens of these and it would be painful to have to remove them all manually.  So how can I automate this?

Well if you have a Linux system, then it’s pretty easy.  Something like this will do the trick:

sed -e 's/crypt_v1{.*"/"/'  oldfile.json > newfile.json
OR
sed -i -e 's/crypt_v1{.*"/"/'  oldfile.json
to do the edit in-place.

Of course you could modify those phrases to replace encrypted strings with actual plaintext passwords as well.

On Windows, PowerShell provides some similar filter capabilities that you could use.

Harvest and Install Server Encryption Keys

If you want to preserve the secure values, of course, the only way to do that is to import the encryption key of the source server into the server you want to import the asset into.

Of course, you will want to carefully consider whether you should be copying the server encryption key of your production UCD instance.  That key gives you the power to decrypt any value held by the server that owns the key.  You will of course also want to carefully consider how you move keys as well.  A key is potentially vulnerable in transit.

OK, we’ve decided that copying the key is safe and we know how to copy it safely.  But how long do I need the key for?  Well, when UCD imports things and it finds an encrypted value it extracts the encryption key alias from the JSON, it then looks in its own encryption Keystore for that alias and if it finds it, retrieves the key and decrypts the value.  When it comes to storing the secure value in its own database, it is re-encrypted using the importing servers’ own encryption key.  So, after the import is done, the foreign encryption key is no longer required unless of course, you intend to import further assets from the same source.

Extracting the Server Encryption Key

The encryption keys are stored in a file called encryption.keystore which is stored in the UCD servers /appdata directory.  The exact location of this directory is dependent on how you configured UCD when it was installed.  If you look in the installed.properties file in your UCD install at the following path:
<<installdir>>/server/conf/server/installed.properties

You will find an entry that looks similar to this:

encryption.keystore=../appdata/conf/encryption.keystore

That’s where you’ll find the encryption keystore.  The path is relative to the UCD servers /bin directory.

Before you do anything else, make a safe backup of the encryption.keystore file – just in case.  If you lose or break this file, you have an unrecoverable problem.

Full instructions on how to proceed from here can be found in the online help.  The link below if for UCD version 7.1  Make sure you use the page for your UCD server version, just in case the process changes.

Be aware that sharing keys between UCD instances that use a JVM from different vendors may present additional challenges.

Sharing secured properties among servers

Plugin Usage

There is another challenge, that of plugin usage.  When importing assets from one server to another, the destination server must have installed all of the plugins that the import references.

 

Alan Murphy is a services consultant who has worked with clients to help them adopt new tools and processes for the last 20 years. UrbanCode Deploy and DevOps has been his focus for the last 5 years. He also develops tools to assist clients in tool adoption and blogs on an occasional basis.

Tags: