Moving my Blog

I am in the process of failing over my website from GoDaddy to AWS. Trying to tune WordPress to run a bit more optimally using RDS, proper caching, etc. The plan is to eventually migrate away from WordPress to my own Django Blog Engine but that is still in development.

You might see some bumpy things happening, but I hope for this to be relatively painless for all of you!

EDIT: And we are back online! This page is smoking fast now and SEO optimized! Finally broke away from GoDaddy! Right now, I am running in a VPC in AWS using RDS! That’s a 2015 goal complete!

Secret Santa – Python Flask Webapp

Hey all, sorry for the past quiet months. It has been quite hectic with my company moving downtown, changing positions, and ultimately trying to crank out and learn so really cool and exciting technology. As every with every winter, I expect to have a bit more downtime and hopefully I can share some cool things I have been working on in my spare time!

Today, I am excited to share with all of you a fun, but very basic (and I mean verrry basic) web app I wrote while trying to digest an exorbitant amount of Thanksgiving dinner. I was tired after Thanksgiving dinner and want to learn something new as well! Some backstory; every year, my family and I get together for Thanksgiving dinner and we do a yearly “Secret Santa” pool. With this being said, this is usually quite problematic as:

  1. Sometimes people draw their own name, thus making it public who is left in the pool
  2. Someone isn’t able to make it to dinner due to other arrangements so we need to draw the name and just tell them (but that is no fun…)
  3. Someone spills the beans on who they got thus potentially breaking the secret chain of who got who

In comes my idea; how about we do secret santa online this year? It sounded like a great idea, I would whip up a simple app that collect names and email addresses and shoots random emails out using Amazon SES. This would have worked if it wasn’t the fact that not everyone at dinner had an email address (primarily my grandparents). So I had to come up with a solution that solves all the three above existing problems, but also allowed folks without email to sign up as well.

With the goal set, I created a simple flask app that allows people to register their name and a unique passphrase. Once registered, users can check their status to see who they have been matched with. When it has been determined that all people have registered, the admin (me in this case) would go to a randomizer url and scramble all the people’s matches. Until the randomizer piece happens, users are told to check back later or poke the admin with a stick to kick off the match pairing. In my situation, I told everyone to register by noon Friday, I would do a shuffle at that time. Once the randomizer was complete, people can then check their pairing.

So after about 2 hours, I came up with the solution, implemented it, and had people up and running on AWS. It worked surprisingly well and even got a chance to show my younger cousin how it works! Best of of all, I have made the code open for you all to use and implement. I added a lot of “bootstrapping” files such as apache configs, sqlite3 setups, and even a sample WSGI file. Below you will find a list of technologies I used, as well as a number of improvements that could be done.

Hope this helps someone else out there as well! Happy Holidays all!

Technologies Used:

  • Python
  • Flask
  • Sqlite3
  • AWS (for EC2 primarily)

Potential Improvements:

  • Admin Interface
  • Secure Passwords
  • Allow for multiple pools
  • SQLAlchemy
  • Various Flask and Pythonic improvements

Sourcehttps://github.com/jvasallo/flask-secret-santa

March Madness Payout Calculator

So it’s that time of year again, March Madness has come around! This means people spending countless hours researching teams, creating brackets, joining pools, etc. At work every year we start a pool for the scores of each round. Basically, you get assigned a random winning number and losing number. You look at the final box score of a given game, and if they match your assignment you win X amount of dollars based on the round! Simple right?

This year, I found myself busier than normal (basically I have other things to do than watch College Basketball all day) so I decided to write a quick app to determine how much I will win. I am leveraging a unOfficial NCAA API so the data might not be 100% accurate, but it will be good enough for my purposes! :)

Feel free to check out the source code below so you can use it too. Note: you might need to tweak payouts, and your magic numbers to your specs!

Link: https://github.com/jvasallo/madness-calc

Download an SSL Certificate using a one-liner!

Had a task today to sync up a SSL certificate, only problem is I didn’t have the certificate nor could I find it anywhere. I knew I could use openssl and get a print out, but I wanted a actual file in my tmp directory. So obvious start with the basics…

openssl s_client -connect HOST:PORTNUMBER

Simple, straight forward…next how to I pipe this to a file. My gut told me:

openssl s_client -connect HOST:PORTNUMBER > /tmp/out.cert

Well…That worked…sort of. I want to be able to terminate this so I don’t have to hit ctrl+c. Well what if I pipe it to echo…

echo | openssl s_client -connect HOST:PORTNUMBER

Bingo! That worked marvelously! But wait…drats…extra stuff about the cert chain. Googled around and found someone already did the heavy lifting using sed!

echo | openssl s_client -connect HOST:PORTNUMBER | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > /tmp/websitessl.cert

Awesome! Now I have a clean importable cert. That will save the certificate to /tmp/websitessl.cert.

Thanks Sean and Vinay from Serverfault. They actually had some extra tips I picked up too!

1) You can use -showcerts if you want to download all the certificates in the chain. But if you just want to download the server certificate, there is no need to specify -showcerts

2) echo -n gives a response to the server, so that the connection is released

So TLDR is, just use:

echo -n | openssl s_client -connect HOST:PORTNUMBER | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > /tmp/websitessl.cert

Source: http://serverfault.com/questions/139728/how-to-download-the-ssl-certificate-from-a-website

No wget, No Problem!

I don’t post enough, so I am going to try to make some day to day blog posts here and there.

So today, I had an interesting experience with some hardware at work running a custom Linux OS. It was pretty locked down, and there is no simple “yum/apt-get install.” To make matters worse, most “core” packages were custom and didn’t exist. Some of these missing packages were wget, rsync, or curl.

I was tasked with putting a rather large file on this system, and was told the best way was to take a USB stick and plug it in. See…I am lazy (it was also -25 degrees outside to get to Building B). This means I needed to waste my time walking in the tundra I call Chiberia, when I could have been hammering away at something else.

First thing I do, login the box. Alright…so I have remote access. Let’s see, I can resolve internal addresses. Typical. Hmm…what about other addresses? Hmm interesting, I can reach them as well. Telnet to 80? Cool, that works too! wget failed…as expected…meh, worth a shot. Hmm, this is custom Linux, I mean…they really shouldn’t touch python. I wonder…BINGO! I got a python shell! In the shell, I ran the below script to essentially “wget” a file onto the box. Took a while, but it worked.

import urllib
urllib.urlretrieve("http://www.example.com/bigoldfile.tgz", "bigoldfile.tgz")

Sure this works, but again…i’m lazy; this is a bit of typing anyways. I want something scriptable that I can just call with a url. Behold, my wget.py. I know, nothing fancy, but it works. Just call, python wget.py <some_url>. Enjoy it for what it’s worth.

#!/usr/bin/python
import urllib
import sys

url = str(sys.argv[1])
file_name = url.split("/")[-1]
urllib.urlretrieve (url, file_name)

New Year, New You! New Portfolio Design!

Before I start, happy holidays and have a safe and awesome holiday season! Looking forward to all the adventures 2015 brings!

So I know I haven’t updated this page in a while, but I figured what better time than now! 2014 is coming to an end, and 2015 is coming fast our way. For now, enjoy this new template “Twenty Fifteen!” It is a stock theme that comes with WordPress. By switching I not only afforded myself a nice looking website but also a responsive look and feel that looks great on mobile.  Looks very sharp, and required only some minor changes to the core PHP theme files and CSS to my liking, such as:

# echo Joel loves using these to simulate a terminal!
# Joel loves using these to simulate a terminal!

Aside from a crazy year @ gogo with a lot of new and exciting projects, all has been well! Still learning, still automating, still Joel! Still sprucing up the page, along with my update Resume, and projects I can share. Check back soon!

In terms of a short article, how many of you have seen “The Interview”? The movie was first being pulled by movie theaters in fear of being compromised, next Sony is forced to pull the movie, Obama criticizes Sony for pulling the movie, and Sony ends up releasing the movie to theaters that want to play it! Crazy ride! Aside from all the political hoopla, the movie is very crude yet very entertaining. Don’t expect a less than heavy hitting comedy, and sometimes it even goes over the top.

Though the entire banning/unbanning story is not as important (in my eyes) to what Sony did next. Sony also made the movie available by means of streaming straight from your home! That’s right! You can rent the movie just like any other movie from Google Play, Xbox Video, and other providers! I feel that this is going to quickly become a trend if this movie does well. As a movie company, why would I lose potential profits to a movie theater (who can restrict, refuse to show, or dictate when and how long my movie is available for) when I can go straight to the consumer? Heck! I would suggest they decrease the rental period from a 24 hour window to a 6 hour window!

Let’s face it, the movie theater is a great experience, but the fact that a movie ticket alone would cost me nearly the price of the movie at retail is a bit absurd. The movies biggest draw used to be the fact that people had tiny televisions with less than optimal quality. Today, we have 7.1 surround sounds in our homes with 4K resolution television sets that can even do 3D! I don’t feel that movie theaters should cease to exist, but rather movie studios should work to making BOTH options available to the consumer market. Well, enough ranting! What are your thoughts? I for one am for it, how about you all?

 

Google IO 2014 Here I Come!

So thanks to an awesome individual, I got a invite code to Google IO 2014! Booked my flight, booked my (EXPENSIVE) Hotel, and got all my bags packed. Looking forward to stepping a bit outside my comfort zone and focus on my development skills more.

Looking forward to learning more on the Chrome side and also picking up some more info on the wearables. Thinking the Moto 360 and/or LG G watch are gonna be hot coming this fall. Once Apple lets their iWatch out, this is going to be big. We just need to wait and see.

Wish me luck, hope to learn a lot!

PyCon Day 4/5: The Synopsis

So…about those daily posts. :) As you know, the blog life can be sometimes hard. I have been too busy learning and meeting some awesome people.

Day 2 (Saturday of Testing and Best Practices)

Keynote:

Great talk about getting Python into schools. Its just there for the taking! Very nice Keynote for the day. It was given by Jessica McKellar, and I even got to meet her! Very nice person, and I got a signed Twisted O’Reilly book! :)


Introduction to Docker

I think I am officially in love with Docker. Create images to make containers. These containers are not VMs but essentially spin up and just run. Much faster to run and very compressed. Also you can make containers as images and even use them as deployment mechanisms! Definately something I am going to look into!

So you want to build an API?

It was very flask heavy, was hoping for more generic based talk. It was pretty good none the less.

Getting Started Testing

Yes. Yes. Yes. Test. Test. Test. One of the best talks of the afternoon. Can’t wait to add some test cases for my code. Even covered some mocks testing too!

REST is not enough: Push notifications for Android and iOS

Gave a good overview of what one might do to get started with Android and iOS. Good news. Tons of support in Python! 😀

In Depth PDB

Very solid talk and highly recommended. Stop using the darn print statement already! :)

Python Packaging simplified

Good talk about packaging your app for the world.

Its Dangerous to Go Alone: Battling Imposter Syndrome

I will admit. For a while I hit a wall. I burned out. I was experiencing imposter syndrome. There I said it. This was an excellent talk for anyone out there trying to get better and feel good about coding. Don’t listen to anyone else, go and be the best you can be. Stop telling yourself you aren’t good enough. You may not understand something…yet. 😀 Great talk and 100% recommend.

 

Day 3 (final day):

KeyNote:

Guido essentially had a txt file with a list of titles. He basically live coded a simple little script and had his talk on random topics. Quite fun, and kind of laid back….the most important thing:

download

 

Time to move to Python 3! Its official! Go out and help y’all! Also become a member of the PSF and join our awesome community!

Deliver your Software in An Envelope

Hitchhikers Guide to Open Source

Very fast paced talk but a feel good talk about committing to open source.

Set your code free: releasing open source code

Very nice presentation that really helped me structure my project. Ran through everything step by step, from License to Testing! :)

And that concluded the regular event! Wow that was a quick 7 days! I officially leave later today but just an amazing experience once again. Met a lot of smart and super talented individuals, and got to hang out and have fun with some great people too. I can’t wait til 2015! :)

PyCon 2014 Day 3: The big day is here!

So I apologize right now, if you consider these blog posts too much! I don’t usually get much time to write some blog posts but I thought I would share some great talks and my notes on a few of them.

Today, was the “official” first day of PyCon with some awesome Keynote speakers and also big talks. Again, this is my first PyCon so I was a bit new to talk tracks. However, I think I found all of the talks interesting that I attended :)

KeyNote

The main keynote was given by John Perry Barlow and it was a fun and pleasurable talk about privacy and policy. Had me chuckling, and at times made some very good points. Talked about how we should have a lack of secrecy, and how we should not be afraid about large controlled corporations. Nice talk to listen too.

Computer Science Fundamentals for Self-Taught Programmers

Took this talk to get a grip of the big O notation as that has been one of my bigger weaknesses when it comes to programming. Big O notation (O(n)) is a way developers talk about the theoretical measure of the execution of an algorithm. At the end of a day, algorithm is just a fancy way of saying a recipe (a step by step guide on how to do something). While big O notation is very important, for really small sample sizes, it can not just matter. Always remember that theoretical speed is different than practical speed. In short, unless you are coding for large datasets this is probably not going to make your app faster. With all this being said it is still something you should be familiar with because it allows you to discuss your code in an organized format with other developers. I still need some work in this domain regardless but I honestly just need to refocus because I was able to do this roughly a year ago. That’s what happens when you don’t hone your craft! :)

So you want to be a full stack developer?

This talk briefly touched a ton of tools available for a full stack setup. I consider myself a full stack developer in that all (almost all) of my code is essentially written and maintained by me from beginning to end. This talk was useful not in the sense of planning, but finding out what is out there in the world! I got a list, and I am not going to bore you with a long winded list, but if you wish. Please reach out and I can share!

Python Web Scraping Showdown

In a short demo, @kjam, talked about different web scraping utilities in Python and how they differ in speed. In short, lxml with xpath was the fastest, lxml with css was a close second, and finally BeautifulSoup came in last. With that being said, at the end of the day, the results weren’t that much different. I would say, use what is easiest to read and understand.

Link: https://github.com/kjam/python-web-scraping-tutorial

Ansible – Python Powered Automation  #3

Things I like about Ansible:

  • Agentless (no remote resources used)
  • Keeps things simple
  • Auditable
  • Batteries are included (Close to 250 plugins in core)

I think the only issue I see is that we have committed to puppet and salt, but I definitely don’t disregard Ansible as a powerful tool. In fact, it may become a leader in this domain. It has BLOWN up on Github in the past year.

Puppet Modules: Apps for Ops  #2

I attended this talk to see if there were any gotchas in our setup. I use puppet at the moment, but I think I was recently won over by Salt. :)

Link: https://speakerdeck.com/jbronn/puppet-modules-apps-for-ops

Getting Started with SaltStack  #1

I recently fell in love with salt after the heartbleed mess in the past week. I was tasked with finding out all our server config to see what needed to be patched. In a short salt command on our master, I ran the following command and did a complete inventory of our openssl versions:

sudo salt * cmd.run “openssl version” >> out.txt

Aside from this, I like the whole Jinja2 (coming from a django background) and yaml based config files. I also like the level of granularity (haha a pun…go figure :p). I am also interested in their Salt Reactor. Basically you can listen in to events that occur, and “react” to them in a proper fashion. It uses Zeromq implementation which in itself if kind of interesting. I have not read much about it but looks like a cool setup. 😀

Link: https://speakerdeck.com/ipmb/getting-started-with-salt

Reactor: http://docs.saltstack.com/en/latest/topics/reactor/

Application Deployment State of the Onion

Watch this video….Fast…and less than 30 minutes. Probably the greatest talk of the day in my book. No nonsense and straight to the point.

Writing Idiomatic Python

I got around to meeting up with Jeff Knupp, author of the book Writing Idiomatic Python, and we had a little meetup in one of the huddle rooms. Discussed a bit about writing a book and some of the challenges, and we got to discuss what it means to be idiomatic. There were a few people who had some comments on some of the idioms of Python, but as Jeff pointed out. The book is not based on his opinion and he might share the same views as someone, but he is simply writing a book about the idioms that are accepted by Python as a whole. Very interesting talk and also got a signed book out of it.

All the PyCon videos and talks can be found at: http://pyvideo.org/category/50/pycon-us-2014

Day 4 recap coming tomorrow night!

PyCon Day 2

Come one, come all to day 2 of my PyCon experience!

Today was a lot more social, got to finally meet-up with @jeffknupp, writer of Writing Idiomatic Python, and a few other individuals including smortaz from reddit (who is from the Microsoft team and wrote the open source Python Tools for Visual Studio!)

In terms of tutorials and talks, I am going to follow the same format as last night. Today on the agenda? Flask and Intermediate Python!

 

Topic: Flask by Example

Speaker: Miguel Grinberg

General Overview:

In short, the speaker went over how to create a sample website like his using only flask and its various add-ons. Coming from a django perspective, this is exactly what I needed. I needed a quick browse over how to create forms, db models, routes, etc. The slides take you step by step from hello world to something pretty solid. Talked a lot of best practices and overall was a great talk! Very fast paced with TONS of information!

Plan:

I want to take this flask knowledge and see if I can improve upon my automation portals that I have created for work. By doing this, I can make faster and more lightweight pages than my existing django setup. I love my django, but I think that at times it can be a tad “overkill.” There are tons of one off scripts I would love to just run from a GUI for end users, and this might help. An avenue for investigation is whether or not I can maintain user session persistence across both implementations (flask and django). I have been tinkering the idea of using redis to maintain a user session that can be referenced in both. Interesting stuff.

Links

 

Topic: Hands On Intermediate Python

Speaker: Matt Harrison

General Overview:

In a very long, and full of content tutorial, Matt covered tons of material ranging from list comprehension, decorators, generators, you name it! Working through examples, talking about real world scenarios, this talk really clicked for me! Nothing really to share other than finally going to start using generators and decorators a lot more now that I understand them. The generator piece is huge as there are tons of code that I could actually improve on.

Plan:

Get better at Python! This talk was awesome and broke a few misconceptions and barriers I had. Sometimes I would look at Python repo’s and be lost in the code. After today, I got some great take aways that I am going to look to take advantage. Time to get those generators and yields going! 😀

Links

 

Talk Schedule for Tomorrow:

  • Breakfast!
  • Opening Keynote
  • Break!
  • Computer Science for self taught programmers
  • So You want to be a full stack dev?
  • Import-ant decisions -or- Python Scraping Showdown
  • Lunch!
  • Ansible -or- Twisted
  • Puppet Modules
  • Getting Started with SaltStack
  • Break!
  • Distributed Computing -or- App Deployment
  • External Talk: Writing Idiomatic Python
  • Lightning Talks!