Migrating from Pre-Release to Core

1. Migrating from earlier versions of PolySync

If you have a pre-release version of PolySync installed─2.0.1-pr.1 to 2.0.8-pr.8─you must follow the migration steps before upgrading to Core.

These steps also work to move a pre-release installation from one supported architecture to another, while also upgrading to Core.

Not following these steps will result in missing configuration and logfile sessions.

1.1 Prepare your files for migration

PolySync has provided a migration script for upgrading from pre-release to Core.

#!/usr/bin/env python3
import os, shutil, sqlite3, re, getpass, calendar, time, sys, subprocess, glob

def get_binary_dir():
    return '/usr/local/polysync/bin'

def get_old_path():
    return '%s/polysync-dynamic-driver' % get_binary_dir()

def get_new_path():
    return '%s/polysync-core-dynamic-driver' % get_binary_dir()

def file_is_sdf(file):
    return re.compile(r'(.*)\.sdf$').match(file)

def cleanup(filepath, num_changes):
    if num_changes is 0:
        os.remove(filepath + '.backup')
        print('Removing %s.backup, no changes were made to sdf' % filepath)

def close_sql_connection(connection, filepath):
    num_changes = connection.total_changes
    print('Closing %s after updating %s rows' % (filepath, num_changes))
    connection.commit()
    connection.close()
    cleanup(filepath, num_changes)

def create_backup_file(filepath):
    backup_filepath = filepath + '.backup'
    print('Creating backup file %s' % backup_filepath)
    shutil.copyfile(filepath, backup_filepath)

def get_sdf_paths():
    sdf_paths = []
    for root, _, files in os.walk('/'):
        for filename in files:
            if file_is_sdf(filename):
                sdf_paths.append('%s/%s' % (root, filename))
    return sdf_paths

def update_cells_to_new_path_string(connection):
    try:
        connection.cursor().execute(
            'UPDATE parameter_values SET value=? WHERE value=?', (get_new_path(), get_old_path()))
        connection.cursor().execute(
            'UPDATE host_configurations SET rnr_base_dir=? WHERE rnr_base_dir=?', ('', '/usr/local/polysync/rnr_logs'))
    except:
        print("Found non-polysync *.sdf file, ignoring")

def update_sdf_core_driver_strings():
    sdf_count = 0
    for sdf_filepath in get_sdf_paths():
        create_backup_file(sdf_filepath)
        connection = sqlite3.connect(sdf_filepath)
        update_cells_to_new_path_string(connection)
        close_sql_connection(connection, sdf_filepath)
        sdf_count += 1
    print('Deprecated values have been updated in %s SDF files on the system' % sdf_count)

# prerelease installation directory
PSYNC_HOME = '/usr/local/polysync'
UTCTIME = calendar.timegm(time.gmtime())
IMPORT_STAGING_DIR = '/tmp/polysync_import/%s' % UTCTIME
BACKUP_FILE = '/opt/polysync-backup-%s.tar.gz' % UTCTIME
IMPORT_FILE = ""

REAL_USER = os.environ['SUDO_USER'] if os.geteuid() == 0 else getpass.getuser()

# help message
if len(sys.argv) > 1:
    if sys.argv[1] == '-h' or sys.argv[1] == '--help':
        print('\n\n')
        print('PolySync Prerelease Migration Script')
        print('\nThis script will migrate the contents of PSYNC_HOME=%s' % PSYNC_HOME)
        print('to the new user home directory PSYNC_USER_HOME=%s/.local/share/polysync.' % os.environ['HOME'])
        print('\nYou must run the script with sudo or as the root user.')
        print('\n  * Import and backup the current prerelease installation')
        print('    $sudo ./polysync-prerelease-migrate')
        print('\n  * Import from a prerelease installation tar file')
        print('    $sudo ./polysync-prerelease-migrate.sh /path/to/polysync-backup.tar.gz')
        print('\n')
        exit(-1)

# only allow root
if os.geteuid() != 0:
    print('This script must be run as root (sudo) user, exiting.')
    exit(-1)

# check for a provided backup file to import
if len(sys.argv) > 1:
    FILE = sys.argv[1]
    if '.tar' not in FILE:
        print('The import file provided is not an ordinary tar file, exiting')
        exit(-1)
    if os.access(FILE, os.R_OK) is False:
        print('The import file provided is not readable, exiting')
        exit(-1)
    if os.path.getsize(FILE) is 0: 
        print('The import file provided is empty, exiting')
        exit(-1)

    if os.path.isdir('/tmp/polysync_import'):
        os.removedirs('/tmp/polysync_import')
    os.mkdir('/tmp/polysync_import')
    os.mkdir(IMPORT_STAGING_DIR)

    print('Extracting import file')
    print('This may take awhile')
    subprocess.call(['/bin/tar', 'xzf', FILE, '-C', IMPORT_STAGING_DIR])

    if not os.path.isdir('%s/polysync' % IMPORT_STAGING_DIR):
        print('The import file provided did not extract into %s/polysync, exiting' % IMPORT_STAGING_DIR)

    IMPORT_FILE = FILE

# backup existing installation
if os.path.isdir(PSYNC_HOME):
    print('Making a backup of your installation in %s' % BACKUP_FILE)
    print('This may take awhile for large installations')
    WORK_DIR = os.getcwd()
    os.chdir(PSYNC_HOME)
    os.chdir('..')
    subprocess.call(['/bin/tar', 'cfz', BACKUP_FILE, 'polysync'])
    os.chdir(WORK_DIR)

update_sdf_core_driver_strings()

# import existing configuration if the user is known
if REAL_USER == 'root':
    print('This script could not determine the real user, skipping configuration import')
else:
    PSYNC_USER_HOME = '/home/%s/.local/share/polysync' % REAL_USER 

    # apply explicit host ID if license is currently valid so we import it
    if os.path.isfile("%s/bin/polysync-license-tool" % PSYNC_HOME):
        HOSTID = str(subprocess.check_output(". %s/utils/set_env.sh && %s/bin/polysync-license-tool -q | grep -i 'host id:' | awk -F\"[' ]*\" '{print $4; exit}'" % (PSYNC_HOME, PSYNC_HOME), shell=True, universal_newlines=True, timeout=5)).rstrip('\n')

        print("Importing persistent Host ID '%s'" % HOSTID)

        try:
            subprocess.check_output(". %s/utils/set_env.sh && %s/bin/polysync-license-tool -id %s" % (PSYNC_HOME, PSYNC_HOME, HOSTID), shell=True, timeout=5)
        except subprocess.CalledProcessError as e:
            # ignore if license hasn't been activated/validated yet
            ERROR = e.output

    #import from file if provided
    if IMPORT_FILE:
        PSYNC_HOME = IMPORT_STAGING_DIR + '/polysync'

    print('Importing configuration from PSYNC_HOME to PSYNC_USER_HOME')
    print('PSYNC_HOME = %s' % PSYNC_HOME)
    print('PSYNC_USER_HOME = %s' % PSYNC_USER_HOME)

    subprocess.call(['/bin/mkdir', '-p', '%s/config' % PSYNC_USER_HOME])

    if os.path.isfile('%s/db/psync.sdf' % PSYNC_HOME):
        if not os.path.isfile('%s/config/psync.sdf' % PSYNC_USER_HOME):
            subprocess.call(['/bin/cp', '-a', '%s/db/psync.sdf' % PSYNC_HOME, '%s/config/psync.sdf' % PSYNC_USER_HOME])
        else:
            print('SDF already exist, skipping over %s/config/psync.sdf' % PSYNC_USER_HOME)

    # import DDS configuration if one does not exist
    if os.path.isfile('%s/utils/set_env.sh' % PSYNC_HOME):
        if not os.path.isfile('%s/config/dds.xml' % PSYNC_USER_HOME):
            PIPE = subprocess.Popen(". %s/utils/set_env.sh; env" % PSYNC_HOME, stdout=subprocess.PIPE, shell=True)
            OUTPUT = PIPE.communicate()[0]
            ENV = {}
            OSPL_ARCH = ''
            OSPL_TARGET = ''
            for line in OUTPUT.splitlines():
                keyvalue = str(line).split('=')
                keyvalue[0] = keyvalue[0].strip('b\'')
                keyvalue[0] = keyvalue[0].strip(' ')
                keyvalue[1] = keyvalue[1].strip('\'')
                keyvalue[1] = keyvalue[1].strip(' ')
                if keyvalue[0] == 'OSPL_TARGET':
                    OSPL_TARGET = keyvalue[1]
                if keyvalue[0] == 'OSPL_ARCH':
                    OSPL_ARCH = keyvalue[1]
            if OSPL_ARCH:
                subprocess.call(['/bin/cp', '-a', '%s/utils/%s/etc/config/ospl.xml' % (PSYNC_HOME, OSPL_ARCH), '%s/config/dds.xml' % PSYNC_USER_HOME])
            elif OSPL_TARGET:
                subprocess.call(['/bin/cp', '-a', '%s/utils/%s/etc/config/ospl.xml' % (PSYNC_HOME, OSPL_TARGET), '%s/config/dds.xml' % PSYNC_USER_HOME])
            else:
                print("Failed to copy dds.xml configuration file.")

    # import db and license files
    if os.path.isdir('%s/db' % PSYNC_HOME):
        shutil.copytree('%s/db' % PSYNC_HOME, '%s/license' % PSYNC_USER_HOME, symlinks=True )

    # import RnR sessions
    if os.path.isdir('%s/rnr_logs' % PSYNC_HOME):
        shutil.copytree('%s/rnr_logs' % PSYNC_HOME, '%s/rnr_logs' % PSYNC_USER_HOME, symlinks=True)

    # import system log
    if os.path.isfile('%s/polysync.log' % PSYNC_HOME):
        shutil.copyfile('%s/polysync.log' % PSYNC_HOME, '%s/polysync.log' % PSYNC_USER_HOME)

    subprocess.call(['/bin/chown', '-R', '%s:%s' % (REAL_USER, REAL_USER), PSYNC_USER_HOME])

    # restore the real location, modified from import file
    os.environ['PSYNC_HOME'] = PSYNC_HOME

    # delete the current installation if we have successfully created a backup
    if os.path.isfile(BACKUP_FILE) and not os.path.isfile(IMPORT_FILE):
        shutil.rmtree(PSYNC_HOME)

# delete the staging directory
if os.path.isdir(IMPORT_STAGING_DIR):
    shutil.rmtree(IMPORT_STAGING_DIR)

The migration script is going to look in the default location of $PSYNC_HOME, /usr/local/polysync/ for the existing install and do two things:

  • Import the configuration and logfile sessions that are currently in $PSYNC_HOME to the expected locations in $PSYNC_USER_HOME, which is $HOME/.local/share/polysync
  • Creates a backup tarball of the existing installation
    • The backup tarball can be copied to any supported architecture and the migration script will place the files in the expected locations for Core
    • Note: the migration script will delete the contents of $PSYNC_HOME after creating the backup tarball
1.1.1 Run the migration script
  1. Create the pre-release migration script on the pre-release host
    • Open a new file with the preferred text editor
    • Copy and paste the Python script from above into the file
    • Name the script polysync-prerelease-migrate
    • Save and close the file
  2. Run the pre-release migration script on the pre-release host
    • $ sudo ./polysync-prerelease-migrate
  3. If you’re migrating to a new host, copy the generated /opt/polysync-backup-<UTC-timestamp-here>.tar.gz to a temporary USB storage device

2. Migrate to a new host

Follow these steps to install Core along with the existing pre-release resources on to a different host.

  1. Copy the polysync-prerelease-migrate script to the target host
  2. Copy the polysync-backup-<UTC-timestamp-here>.tar.gz to the /opt/ directory of the target host
  3. Run the migration script, and pass in a path to the pre-release backup archive
    • $ sudo ./polysync-prerelease-migrate /opt/polysync-backup-<your-UTC-timestamp-here>.tar.gz

The migration script will decompress the archive in a temporary location and then copy the configuration and logfile files to the expected locations on the system in PSYNC_HOME, and PSYNC_USER_HOME.

The script can take some time, especially if there were large logfile sessions on the pre-release host.

2.1 Replacing the original SDF

If the migration script detects an existing SDF it will not overwrite the SDF in $HOME/.local/share/polysync/config.

To install the original SDF on the new system extract the SDF file from the polysync-backup-*.tar.gz tarball and copy it to PSYNC_USER_HOME/config.

$ cd ~/Downloads
$ tar xf polysync-backup-<UTC-timestamp-here>.tar.gz
$ cp polysync/db/psync.sdf $PSYNC_USER_HOME/config/

3. Upgrade

Now that pre-release files have been migrated to the new system you can follow the standard installation steps.

4. Moving a license

Follow these steps to move a license from one host to another.

  1. Run the license tool on the pre-release host:
    • $ polysync-license-tool -a
  2. Copy the Host ID
  3. Upgrade to Core
  4. Run the license tool command once more
    • $ polysync-license-tool -a
  5. Send an email to help@polysync.io containing the Host ID output from both commands, and indicate that you’re migrating to a new host