Wow its been a while since I last posted. Lots going on with interviews last month and then deciding to teach myself Ruby on Rails and being obsessed with Julython. I completed and launched my first Rails app (technically second after the sample app from the tutorial) at the beginning of this week. It is a simple thing that literally is just a button and I couldn’t be prouder.
What is Python Trek?
Python Trek is a mashup of Stark Trek and Monty Python scripts, quotes, etc. I built a program to generate random sentence or sentences using a Markov Chain algorithm and post those to the Twitter account @pythontrek. I used Ruby to build the program that generates the tweets and I applied Rails to set up a webpage where anyone can press a button that will generate the random tweet and post it.
- Python Trek button site: http://python-trek-2013.herokuapp.com/
- Python Trek twitter site: http://twitter.com/pythontrek
Why did I do this?
One of my code challenges last month asked me to write a Ruby program and present it. The company asked me to build something small and didn’t have to be a web app. I only wrote the Markov Chain generator for the challenge and afterwards, I decided to take it a step further and learn Rails.
I am in learning mode, and since I want to learn all the things, I figure why not leverage this opportunity. The reality is that Ruby & Rails (usually referenced as Ruby on Rails) are in high demand since so many companies are using it in the Bay Area. I know it does not hurt at all to expand my knowledge and skills in this area.
For those who are new to Ruby & Rails, Ruby is the programming language and Rails is the framework to build web applications with Ruby. If you’ve read any of my previous blog posts on Sun Finder, you’ll see I use Python as my main scripting/programming language to build my app and Flask is the framework.
I actually understood some of the mechanics of Python better by trying to learn another language as well as web frameworks and how web stacks work by trying to understand Rails.
How did I do it?
Lots of help and lots of time. Thankfully the network I have now was able to give me some great coaching and that’s especially true for a good friend, Jason and my mentor Jeremy. I also just put a lot of time and effort into working through it. Still to give a little more detail to the process:
The week and half I had for the assignment back in mid June, I was juggling 2 other codes challenges, interviews and a couple other activities. Thus, I probably clocked about 4 days or so on doing the initial Ruby app.
I spent about half a day on the free Ruby course on Code School get some fundamentals. The concepts in general were pretty similar to Python. Afterwards, I made an exec decision to use a project we did in Hackbright and rewrite it in Ruby. I wanted focus on understanding Ruby and not be distracted with figuring out the program problem I was solving.
Another day or so (total time spread over several days) went to just googling how to do ‘x’ from Python in Ruby. It required some research and frustration at times when I could do one thing in Python that was not something Ruby allowed and vice versa. Like using list comprehension or setting default variables if a hash/dictionary key had not been added yet.
While building out the Ruby app, I also leverage Zed Shaw’s ‘Learn Ruby the Hard Way’ since it’s structured just like the Python version. That made looking up information very easy. Additionally, I found the Ruby-doc site a great reference document when trying to find how to write certain syntax, and as always, StackOverflow was my friend for many answers. But let’s face it, googling was just the way to go most of the time.
Once I had the program rewritten in Ruby and working on the command line, I started to play with the code to improve its functionality. I was working to align it to the Markov approach because we hadn’t really finished that algorithm while we were doing the project in class. Then I just got fancy creating two hashes/dictionaries for capital vs. lower-case and trying to define the end of sentences based on end marks that would pop up.
If you are asking what is Markov, check out the wiki article. To give you some sense of it, it’s an approach to create a random process characterized as memoryless. The next state only depends on the current state and nothing before that. It’s not the best solution for predictive text, but it is a way to approach randomly generated text.
Summary Random Text Approach
For the project, the goal is to take a text file full of content from two different areas of interest and mash-up the words by creating a hash/dictionary out of the text. The hash/dictionary is a key:value pairing between two words (in the stricter sense, a word and an array/list). So if ‘a’ is a key then all the immediate words that follow ‘a’ are put in an array/list as the corresponding value. Then to build a sentence, a random key is selected initially, then each new word is used as a key and the associated array/list values are looked up where one word is randomly chosen from the value array/list to add to the sentence. This continues till the sentence meets a limit of no more than 140 characters or less.
I’ve added a little more complexity to the approach that what I’ve mentioned above and if you don’t know about hashes or key/value pairs then the information then just ignore that last paragraph. All you need to know is the program stores words from Star Trek and Monty Python in a way that is able pull and create random sentences.
Why Python Trek?
Initially I used sample text to build the program and when it worked well enough, I switched over to figuring out what text I wanted to create for the official mash-up. I made the decision at like 1 in the morning after several hours of coding that day. I had been watching the original Star Trek while programming, and it’s quite probable, I heard the song, “Always Look on the Bright Side of Life”, sometime that day. So I was like, ‘wouldn’t it be fun to have a Star Trek and Monty Python mash up?’ I’ve definitely been a fan of both and they both hail from campiness. Thus, I spent a couple of hours copying and pasting quotes and script stuff into a raw text file. It really is a bit messy and could use some proper data munging, but I had other priorities to tackle.
Setting up the @pythontrek handle was the simplest part of this project, and I was a little surprised the handle wasn’t taken. Setting up the code to link to the API didn’t take too long because all the work I’d done with APIs before made it pretty easy. Plus, there was plenty of sources out there that shared the code and how to set it up.
Beyond the Challenge
Being the type A person that I am, I had to take this challenge further and learn more. I went through my code with my mentor and got help refactoring the solution from a functional structure to a class based structure. Also, we worked on modularizing the code because the functions were very long and weighty. Making those changes took a day with help. Still what really made this a valuable exercise was deciding to learn Rails so I could publish a page that would give other users access to generating random tweets.
I spent about a week 1/2 spread over 2 1/2 weeks trying to go through the Ruby tutorial my friend showed me, Ruby on Rails Tutorial. I had hoped it would just take a couple of days, but let’s face it…Rails is complicated. Its hard going from Flask to Rails because as they tell you, “Rails does a lot for you.” That is not necessarily a good thing when a lot of times you are trying to understand what exactly it is doing for you.
When building out Rails, some key files and sections to be aware of:
- Gem file lists all the gems/packages/libraries needed for the application. This is like a Flask requirements file that lists out all the programs and packages that must be loaded in order for the program to work.
- Routes file defines paths to views similar to the decorators (@app) put above view functions in Flask. Basically the decorators are being listed in a separate file that help generate views based on corresponding controllers and actions.
- Controller folder/section defines the view structure with specific actions for the views (e.g. page render or redirect). If there is any data manipulation that should take place elsewhere and content in this section should stay focused on concepts around building views, passing data, maintaining sessions, etc.
- Model section is focused on structuring database content and interactions
- Views section includes the templates for pages that are rendered.
It’s a little tricky to the get hang of Rails because there are a lot of other files and folders that help and support the web application as well as short-cuts that are supposed to make it easier, but once you work with it enough, it will make sense.
Test Driven Development (TDD)
The tutorial I used spent half the time going through how to do test driven development in Rails. I had wanted more exposure to TDD and this was a nice way to integrate it. There were times I hated RSpec (Rails TDD) because it liked to throw all kinds of errors that I had a hard time with because I had a hard time understanding what Rails was doing. It did force me to really work with Rails.
Heroku & Postgresql
Something else the tutorial went through is how to launch an app on Heroku. It took me a solid day of wrestling with Heroku and Postgres but I finally got the hang of it. The tutorial doesn’t cover everything you will need, but I can say that both Postgres and Heroku provide all kinds of documentation and there are plenty of people who post about it.
A couple of things I learned are that when setting up Heroku with Postgres, you don’t need a password in your database yaml file. You do need a user name that has been granted access for creating databases. If you get errors regarding the user, look-up how to setup a user in Postgres, and just open a terminal to do it (again stressing to make sure to grant the user the create db rights).
A useful database.yml file that I leveraged for the format when creating my file can be found a this link. Also, note you need a local test db for Postgres, but that does not get posted to Heroku. Additionally, don’t forget to rake create:db and migrate when there are changes to the databases.
Python Trek Button
Eventually (last weekend) I got the sample app from the tutorial to work; thus, I finally switched gears to build the web app for my Python Trek. As mentioned, I was just trying to build a page with a button that anyone can push to generate a random sentence and post to Twitter. Seems easy enough and seems like a lot of work after the fact to accomplish it. It really would have taken me a weekend or less if I had used Python and Flask.
Text File Alternative
What made this application a little tricky was that I was not building out a database(db) model. I wanted to use my text file solution where I just open and read from the text file and generated a couple hashes through a class object. Most tutorials and any helpful code to leverage is written from the perspective of using a db model. The way I tackled my solution was to store my class structure in the library section (lib) as well as the text (source) file and the tweet method I defined.
With help, I learned how to reference those files through the controller and pass the information needed into the web page. I also applied Ajax to post up the tweet that was generated after the button was pushed on the same page as the button. Thankfully that just went into a script tag on the main page template.
Stake in the Ground
The good news is I finished the app and it works. It was not easy learning a new language and framework considering I’ve only been doing this since late Feb. Still it was a great exercise that I would recommend to anyone who is going through the programming learning process.
This week, I’ve been switching gears back to some stuff I want to do in Python for a little while. There is a lot more I can do with this app (hell my friends are already asking that I let them create user accounts and track who pushes the button more) and I would like to continue to leverage it as a learning ground. Still I’ve done what I’ve wanted to do so far. Additional next steps will have to get in line.
Side Note – Seriously?
The morning of my interview last month, I opened the project and was surprised to see a file labeled python. Initially thinking it was a mistake since I had written the project in Ruby. It finally dawned on me how I had shortened the name Monty Python and by using them in my mashup I was inadvertently (probably subliminally) putting a reference to the Python language in my application. It wasn’t till I showed the project to my mentor after the interview that he made me aware of the fact that Python actually is named after Monty Python. Seriously, no clue.