Migrating On-premise PostgresSQL to Cloud SQL

In this post I'll cover what it takes to move an on-premise PostgresSQL database into a Cloud SQL instance.  I've already migrated a workgroup server, but I could have just as easily kept my workgroup server on-premise.  At the end of this I'll have all of my Content Manager infrastructure within a secure, private, fully-managed cloud infrastructure.

The first task is to generate a  SQL dump from PostgresSQL but with no extension related statements.  I'll need to install GnuWin so that I can leverage the sed command.  I executed the pg_dump command, piped the results through sed and then to disk (as shown below).

Next I created a bucket named "postgressql-backup" and uploaded the file.  Although I could have done this via the command line, I preferred to use the cloud console. 

Next, I go over to my Cloud SQL instance and select import. 

Then I can click Browse, navigate to the file in the bucket, and select it.

Lastly, I need to select the target database within my instance and then click Import.

After clicking Import, I returned to the Cloud SQL overview dashboard to monitor the progress.  Eventually I'll see a notification once the import has completed.

Now I connect to my Content Manager Workgroup Server and launch the Enterprise Studio.  Here I am registering a dataset because I've also decommissioned the original workgroup server.  Alternatively, I could have modified the connection string of an existing dataset.  I used all of the settings from the Cloud SQL instance  when configuring the connection string, as shown below.

I completed the wizard and was shown a message indicting success.

A quick check of the client shows me everything worked. 

Sweet!  Now my DemoDB dataset fully resides within the cloud!

Migrating the SQL DemoDB to PostgresSQL

The installation media for Content Manager 9.2 comes with a demonstration dataset that can be used for testing and training.  Although I think the data within it is junk, it's been with the product for so long that I can't help but to continue using it.  To mount the dataset you have to restore a backup file onto a SQL Server and then register it within the Enterprise Studio.

SQL Server is a bit too expensive for my testing purposes, so I need to get this dataset into PostgresSQL.  In this post I'll show how I accomplished this.  The same approach could be taken for any migration between SQL and PostgresSQL.

I'm starting this post having already restored the DemoDB onto a SQL Server:

If you look at the connection details for the highlighted dataset, you'll see that GIS is enabled for this dataset.  My target environment will not support GIS.  This inhibits my ability to use the migrate feature when creating my new dataset.  If I tried to migrate it directly to my target environment I would receive the error message shown below.

Even if I try to migrate from SQL to SQL, I can't migrate unless GIS is retained...

To use the migration feature I need to first have a dataset that does not support GIS.  I'll use the export feature of the GIS enabled dataset to give me something I can work with.  Then I'll import that into a blank dataset without GIS enabled.

The first export will be to SQL Server, but without GIS enabled.  When prompted I just need to provide a location for the exported script & data.

Once completed I then created a new dataset.  This dataset was not initialized with any data, nor was GIS enabled. The screenshot below details the important dataset properties to be configured during creation.

After it was created I can see both datasets within the Enterprise Studio, as shown below.

Next I switched over to SQL Server Management Studio and opened the script generated as part of the export of the DemoDB.  I then executed the script within the database used for the newly created DemoDB No GIS.  This populates the empty dataset with all of the data from the original DemoDB.  I will lose all of the GIS data, but that's ok with me. 

Now I can create a new dataset on my workgroup server.  During it's creation I must specify a bulk loading path.  It's used in the same manner as the export process used in the first few steps.  The migration actually first performs an export and then imports those files, just like I did in SQL Server.  

On the last step of the creation wizard I can select my DemoDB No GIS dataset, as shown below.

Now the Enterprise Studio shows me all three datasets.

PostgresSQL in GCP

If you're like me and are moving to the GCP stack, you'll find you have a few extra steps to undertake when preparing your infrastructure. 

First, you'll have to manually install the PostgresSQL ODBC drivers on your VM instance.  You can download them from the PostgresSQL website, locate here: 

https://www.postgresql.org/ftp/odbc/versions/

Second, you create your PostgresSQL instance within your GCP project.  This was super simple.... click, click, click, done.

Next I secured access between my PostgresSQL instance and the VM instance.  After that, it is necessary to create a schema for your Content Manager database.  Content Manager will throw up on the screen if you don't have one already prepared.  

To resolve the issue you'll need to manually create your schema within the database.  I did this via the gcloud shell, but you could do this locally from your gcloud SDK (or any database management tool capable of managing a PostgresSQL instance).

Only draw-back to this cloud instance of PostgresSQL is there is no GIS support...

Now that my Content Manager instance is fully in the cloud I can leverage all of the really cool features of the GCP stack!