Deployment is not the Devil (Flask & Heroku Tips)

On a previous post I claimed deployment is the devil because after several successful (not always easy) deployments, pushing up my Sun Finder app proved elusive. I seriously wanted to scratch my eyes out at times with all the errors and issues. Still it was a good learning experience (one that I fought against but a good one all the same), and I did finally deploy as of last week! Check it out at sunfinder.io.

So deployment isn’t all bad but it sure can be frustrating if there is a ton more work that is needed to deploy after the long haul of web app development.

To help others new to deployment and especially working with Flask, here are some things I learned along the way.

First off, if you are working with Flask then use this help page as a starting point on how to develop and setup apps on Heroku.  On the left side of the page, there are links that provide similar support for other frameworks. Just make sure to setup a repository to store your project and configure remote access.

1. Local Database

The biggest deployment challenge I had was importing a pre-populated database from my personal computer (local).

Overview
Heroku provides a Postgres add-on from Heroku Postgres which adds a remote database onto the application. When a remote database is provisioned, a config variable is assigned to the repository which usually includes a color in the name. This variable is a reference to the URL where the empty database is accessible.

DB Remote Storage Option
In order to load a pre-populated database, it has to be stored remotely in a service provider like AWS S3 (Amazon Web Services Simple Storage Service) or Dropbox and then imported to Heroku. S3 is a popular storage solution because its been around for a while and is known for its optimization. I haven’t looked into Dropbox as a solution, but I suspect its a good option as well.

High-level directions on how to setup S3 for Heroku are provided at this help page. AWS also provides pretty extensive usage directions.

IAM: User & Permission Setup
After signing onto AWS, go to the IAM section under Deployment & Management section. In this area, setup a user and obtain security credentials. Make sure to capture the credentials  (Access Key ID and Secret Access Key) for later reference. Then setup a group and assign access permissions. Go for the admin permissions setup if its just one person. Once the group is created, click on it and look below the tab section for the option to add a user. Make sure to add the user to the group.

S3: Bucket Setup
From IAM, return to the main section by just clicking on the box in the top left corner and choose S3 under Storage & Content Delivery section. In this area, create a bucket (aka root folder) to store the database. Make sure to create the bucket in the region where the app is stored which for Heroku is the US Standard region. This is important because S3 is free for in region data transfer rates. Also, make sure the group is granted access permission to this bucket. At this point the database can be uploaded.

DB: Compression,  Upload & Configuration
In order to upload a database, compress the local copy first which is also referred to as dumping. Heroku’s directions on how to create a compressed version of the file can be found at this page.  Create the dump file and open the AWS bucket in order to access the upload option. Once the file is uploaded, add the AWS security credentials into Heroku configuration following the directions on the Heroku S3 help page.

DB: Import
I followed the import directions on Heroku’s help page that explained compression but there were a number of errors (like “invalid dump format: /tmp/…/sunfinder.dump: XML  document text”). In order to resolve these problems, I logged into the Heroku Postgres site.  It showed all of my Postgres provisioned databases and when I clicked on one of the databases, I was able to see connection settings and statistics. There is an icon of two arrows pointing in opposite directions in the top, right corner that provides a list of additional options. There I clicked on PG Restore and found more explicit command directions for import. The only part of the command that needed to be changed is to swap “your data file” with the dump file name that is inside the bucket. This resolved my errors and enabled database setup.

Just remember that any changes made to the local database need to be compressed, uploaded onto AWS again and imported in order for it to be seen in the remote application.

2. Static Content

When I first read the Heroku S3 help page, I mistakenly thought I had to store all of my static content on S3 (e.g. img, js, css). Granted previous deployments seemed to work and not require this, but I couldn’t get the css and js files to load correctly on my application.  I was getting a 403 error with a link to an XML page that said “Access Denied”. 

So I loaded all the static content on AWS S3 and made it public. This actually made the application work once I changed the static file references to the new AWS location and links. Then I finally figured out that the Heroku application key configuration was incorrect and thus, the problem. So I rolled back my changes to keep the static reference links internal vs. pointing at AWS.

Using AWS to store and reference static files is more useful in situations where there is a significant amount of content and/or users are loading content onto the application. There is a lot of literature out there that provides more details.

3. Heroku Configuration & Updates

In the process of setting up the Heroku repository, don’t forget configuration. It can make things go wonky like 403 errors if its wrong. In the errors around static page loads, my configuration of the Flask app secret key was incorrect on Heroku. Definitely read this link and make sure to load all the secret keys that are needed.

Additionally, make sure to git add and commit changes in order to push updates to Heroku. If a change doesn’t show on the remote site, it’s possible that it wasn’t committed before the Heroku push.

4. Not All Browsers are the Same

Another error/warning I found was in the he Chrome browser’s Inspect EIement console: “The page at … displayed insecure content for ….”. The dots represent a link that the warning referenced. My current hypothesis is this is because I loaded HTTPS Everywhere on my computer and some of the links in my site point to sites that do not use secure socket layer protection. Its just not an option at some sites. This is just a warning and does not prevent my application from functioning. If I learn more, I will update this post.

One thing that I was reminded of while troubleshooting my app is that not all browsers function the same way. So I opened my app in a different browser to check if some errors and warnings would go away. Its just one more way to test the application and help narrow down the problems. Granted there are many browsers and versions of browsers that can impact functionality and plenty of materials on how to develop for all those variations.

5. When All Else Fails – Reboot

Initially I made the first Sun Finder deployment attempt in June. When I returned to deployment in Aug., I tried working with the already established Heroku repository and configuration. At some point in my error tackling, I realized its better to just reset and restart. So I deleted the Heroku repository and created a new one. This didn’t resolve all my errors, but it did help clear out some of the more mysterious ones (e.g. the ones unknowingly created during the learning process).

For those out there working on Heroku deployment, I still stick by a previous post comment that Rails is an easy experience, but it is very doable to achieve deployment with other frameworks. If anything it can be a little more educational at times. Just don’t let the challenges keep you from that final hill to launch.

Comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s