Matillion Server Setup

Let’s Launch another EC2 instance that will house our Matilion server.  

Note: Keep in mind you will have to pay for this, so you may want to terminate this instance when you are not using it.  You can take a snapshot, create your own AMI, and restore the instance when you are ready to use it again.

*At the time of writing T2-medium Linux EC2 instance was about $.05 and hour, and Matillion outside of the 2 week free trial period was $1.37 and hour.

matillion setup launch instance

Go to AWS Marketplace.  Search for ‘Matilion Software’. Click Select.

matillion setup aws marketplace annotate

Chose an instance type.

matillion setup t2-med

Select Auto-assign Public IP.  Now let’s create a new IAM role, to give this Matillion server access to our S3 buckets.  Click ‘Create new IAM role’.

matillion setup - config ec2

Click Create Role.

matillion setup - create role

Select EC2 as the trusted entity type.

matillion setup -ec2 role

Search for S3, and check AmazonS3FullAccess.  This will allow the Matillion server to access our S3 buckets.

matillion setup - permission policies

 

Note:  If you plan on using other features outside of this tutorial you may want to add the following as well.

AmazonSNSFullAccess

AmazonSQSFullAccess

CloudWatchFullAccess

AmazonRDSReadOnlyAccess

Click ‘Next: Add Tags’.

Click ‘Next:Configure Security Group’

Optionally change the SSH & HTTP port to My IP.

Note:  I left port 443 open to 0.0.0.0/0, which means it can be accessed by any IP.  You may want to lock this down to more specific IP addresses.

matillion setup - security group my ip2

Click ‘Review and Launch’.

Click ‘Launch’.

Select existing key pair of MyPOCKeyPair. Check box to acknowledge private key.

matillion setup - select key pair

Click ‘Launch Instances’.

Click View instances in bottom right corner of screen.  Or you can navigate Services->EC2->Instances (in left hand pain).

Navigate in your browser to the public IP of this EC2 instance.  Copy the Instance ID, this is your initial Matilion Password.

matillion setup - instance id

Log in to Matillion by entering the Public IP of the Matillion ec2 instance into your web browser.  User name is ‘ec2-user’ and password is instance ID from above.

matillion setup - login

The first screen will prompt to create a project.  Add project names as well as your Snowflake details.  Your Snowflake Account is the code in your snowflake URL, right after the “HTTPS://” and before the first period.  For example if you link to snowflake is ‘https://ie03491.snowflakecomputing.com/console/login#/’, then your account is ‘ie03491’.

The database, warehouse and schema names are case sensitive.

matillion setup - proj setup redact

Next let’s reset our password.  Select Admin->User Configuration.

matillion setup - user config

Click on the ‘lock’ symbol next to your user name.

matillion setup - change password

Enter a new password and click OK.

matillion setup - reset password2

Server backup and Disaster Recovery

If you would like to suspend your server when not using, the following are instructions on creating a snapshot and AMI of your Matillion server.  You then launch this AMI when you want to pick back up your work. Be sure to use the same security settings you did when initially setting up the Matillion server.

Note:  I had issues logging in after restoring my AMI.  After working with Matillion support, it was found that for some reason my PW on the restored server was not being set properly.

You should copy the “/usr/share/tomcat8/conf/tomcat-users.xml” file from your Matillion server and store it somewhere safe.  Launching your AMI, if you have trouble logging in, replace the tomcat-users.xml file with this back up.  If you want to backup the existing tomcat user file before replacing, be sure to move it out of the ‘conf’ directory as this can cause issues.  Then restart your tomcat server using the following command:

> service tomcat8 restart

Additionally you may want to export a backup of your code to JSON for disaster recovery purposes.

JSON back up:

matillion ami - json backup

Create AMI.  Stop the instance (optional).

matillion ami - stop.png

Navigate to the Root device’s volume.  Select the instance that we will create the AMI of.  Click the ‘Root device’ link under the description tab in the bottom half of your screen.  Click the EBS ID from the resulting window.

Alternate:  You can click on root device and note the EBS ID.  Then navigate ‘Elastic Block Store->Volumes’ in menu on the left.  Then manually select the volume ID.

matillion ami - volume.png

Under ‘Actions’ select ‘Create Snapshot’.  Make sure your volume id is selected.

matillion ami - create snapshots.png

Navigate to ‘Elastic Block Store->Snapshots’.  Select your snapshot and under ‘Actions’ menu, select ‘Create Image’.

matillion ami - create image.png

Give your Image a name.  Select Hardware assisted virtualization as ‘Virtualization type’.  Everything else can remain as the defaults.

matillion ami - image details.png

If you want to launch your image, the process is it is very similar to how we launched the ec2 instance originally.  This time you are using one of your own AMIs and not one from the Market Place.

matillion ami - lauch ami.png

Finally.  Ready for the Fun Stuff?

 

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s