Packer/Ansible: Unable to acquire dpkg lock

Today I ran into a rather odd issue attempting to patch a base image using Ansible and Packer. Randomly and sporadically, my playbook would fail with the error of:

Could not get lock /var/lib/dpkg/lock-frontend

If you ever try to run Ansible on Ubuntu 16.10 and later, be aware that Unattended Upgrades is enabled by default. On boot of new Packer bake instances, I noticed that it would sometimes lock apt to do security updates by default. I spent a good part of my day tracking this down as to why sometimes it would work and other times it would not; turns out it was a race condition (could apt finish faster then next step). Seeing as it’s 2018 and we should not be afraid of security fixes, I didn’t want to disable this because this is useful for security and such.

To get around this, I created a role called prerun, which does the following task:

# Check for unattended-upgrades
- name: Wait for automatic system updates to complete
  shell: while pgrep unattended; do sleep 10; done;

After including this in roles that used apt, my error went away. One of my builds took almost 30 seconds to get past this; which would have otherwise failed. Hope this helps another poor soul out there. 🙂



Another way as I was shown with Packer (to avoid adding additional Ansible roles) is to run a pre and post script in between your Ansible run! 

In your base Packer JSON provisioners section, you would do:

  "script": "scripts/",
  "type": "shell"
  "extra_arguments": [
  "playbook_dir": "playbooks",
  "playbook_file": "playbooks/example.yml",
  "type": "ansible-local"
  "script": "scripts/",
  "type": "shell"

In, you would need a section to do whatever you need (most likely install ansible and such):

# Ensure to disable u-u to prevent breaking later
sudo systemctl mask unattended-upgrades.service
sudo systemctl stop unattended-upgrades.service
# Ensure process is in fact off: echo "Ensuring unattended-upgrades are in fact disabled" while systemctl is-active --quiet unattended-upgrades.service; do sleep 1; done

Finally, in (and whatever else you need, like delete ansible dir and such):

sudo systemctl unmask unattended-upgrades.service
sudo systemctl start unattended-upgrades.service


Another solution could be to add this in you Ansible roles before package installation:

- name: Wait for /var/lib/dpkg/lock-frontend to be released   
shell: while lsof /var/lib/dpkg/lock-frontend ; do sleep 10; done;

Thanks Gordon Kirkland for the additional solution!

Hope one of these many solutions helps others stuck with the same problem!

The Five Phases DevOps: A Retrospective on a Retrospective

A bit delayed news but I wanted to share some exciting news with everyone!

Recently, I accomplished one of my many personal goals; speaking at a local conference. On August 30th, I took to the stage at DevOps Days Chicago to talk about the Five Phases of DevOps! Since that time, I also gave my talk at Windy City DevCon (September 16th) sponsored by the local Google Developers Group!  I haven’t gotten a chance to publicly say this, but I want to give a huge thank you to all the conference organizers and speakers, my friends and family, and of course my team at gogo for helping me get prepared for this!

The talk was titled “The Five Phases of DevOps”, in which I took the audience on a journey as to how we incorporated DevOps at my current employer (gogo). This talk was a bit out of my comfort zone since it really wasn’t a technical talk; which is my day job. I didn’t show off any build tooling, I didn’t have Python code examples, and didn’t even “do the DevOps” to magically create a presentation.

I was a bit worried early on, however, I felt passionate about the amazing work we did in the past year so I felt I needed to do this talk; if we can do it, so can you. I haven’t gotten a chance to publicly say this, but I want to give a huge thank you to all the conference organizers and speakers, my friends and family, and of course my team at gogo for helping me get prepared for this!

I wanted to give a brief retrospective of how I prepared for my first conference talk. I hope this post has some tips to help out folks about to give a talk! Everything I am about to say is my personal opinion!

Lessons Learned:

When submitting a proposal, it’s better to have one solid topic than many general topics.

When trying to pick a topic, pick an idea that you feel passionate about and run with it. Early on I was tempted with submitting many small proposals but found myself struggling to generate enough content for the time needed. The final talk I submitted was actually stitched together from three separate topics! See if you have similarities across the topics you want to submit; you may be able to incorporate them into one!

A helpful practice would be to come up with bullet points for each topic you want to submit and see how it flows together. Flow is the most important thing when speaking to a large crowd. When taking your journey to Middle Earth, you want it to be as free flowing and smooth as possible; avoid trolls and goblins if at all possible.


Practice, practice, practice

This one is commonly stated but prepare your slide deck in advance and take the time to refine it! Once you feel you have a starter set of material, present it to a small audience and ask for honest and critical feedback. This is when you can enhance your content to deliver a solid talk for your real audience!

A helpful tip is to practice in front of both a technical and non-technical audience. By doing this, you can see if you are technically on point, but also you can see if you can hold the attention of someone who might not understand the topic as well.


Use text to supplement your talk; not drive it

A great piece of advice I was told by another speaker is that

“Your audience is there to hear you speak; not read your slides.”

Don’t load up your slides with a bunch of text and just read them. You are a speaker, not a reader! Have key points on your slides, and follow up with clear thoughts and explanations. Also remember to have a good balance of images and text.

Early on, the most common feedback that I received was that I was reading my slides too much. Remember, speaker notes are your friend! Put key points you want to mention in them for each slide.


Don’t have your text and images fight for attention

Try to have a solid balance of both in your slide deck. Too much text and your audience may get bored. Too many images and you may distract your audience.

Don’t have big blocks of text with many images. Realize that your audience will need to focus on three things at the same time; your text, your images, and you. Use your content to supplement you!

It may be helpful to use large image slides to talk to the audience and explain my points. Use text slides to deliver takeaway quotes or supplement what you are talking about.


Tie it back to real world examples

When trying to explain something, it is really helpful to show real use cases or examples to help drive the concept home. You probably have already seen this done in many technical talks. Traditionally this is done by showing code snippets, technical implementations, or service level examples.

As I mentioned this was my first non-technical presentation. I couldn’t just toss random code examples or show off a live demo. I found that my best ally was to leverage facts and metrics. Try to explain your message in an easily relatable way; pictures, stats, stories. (Everyone loves story time)

The goal is to show that by doing X you can see result Y, which means Z. Since my talk was about a year’s worth of progress, I found a timeline was the best thing to use. I took each phase of DevOps, related to a quarter of the year, and explained technical details there.


If you have live demos, record a video

When giving a tech talk, we all want to do a live demo in front of an audience. There is a rush of excitement we all get showing off technology just working. With that being said, realize the odds are forever not in your favor when it comes to live demos at a conference. Shitty WiFi? Displays not working right? Typos while being nervous from speaking?

I always suggest to always have a backup video recorded just in case something happens. You can try your live demo, and the second you see oddities flip to the video. No one wants to hear you justify issues, fiddle settings, or fix bugs/typos; they happen. Whether it is your fault or not, no one will care. Remember the goal is to supplement your talk and key points with a live demo, not distract from it!


Understand the logistics! Ask Questions!

Some things you should know in advance:

  • Is it a lightning talk or a full-fledged talk?
    • You probably already know this from your application but it never hurts to confirm. Also knowing when you are scheduled to talk early helps as well.
  • How much time do you have? Does that include questions?
    • I totally messed this one up and assumed my 30-minute slot had additional time for questions. Let’s just say I had to do some slide surgery the night before to accommodate this. Don’t let this happen to you.
  • What display and power connectors will be provided?
    • Make sure you can present all that great material you have. Bring your own adapters if needed as well. If you want to set up ahead of schedule, ask your event organizers!
  • What display resolution and how many screens will you have?
    • Design your slides for what you will be presenting on! A 70+ inch screen combined with 12pt font might lead to a bad time for folks as they digest your content; especially the crowd out in the back! A tip is to make sure you can easily read your material from at least 7 feet away when writing them!
Take it slow, breathe, and have fun!

Realize that no matter how much you practice, you may/will trip up. If you feel yourself getting overwhelmed….STOP! Breathe, drink some water, focus, and compose yourself. It is not the end of the world and a small flub here and it is normal!

A tip shared to me from another speaker was that as you progress with your talk, you may enter a zen-like zone. You will stop worrying about the crowd, you will find your words flowing naturally, and ultimately you might actually say things you don’t remember or didn’t practice. I found that all my key points and audience takeaways were delivered best during these phases. Relax, have fun, and go with the flow!

Review yourself and share your work!

So your talk is over, it’s time to share what you did! Self-promotion is one of the best things you can do, so make sure to include your relevant contact information at the beginning and the end of the slides. By including them at the end, you don’t need to awkwardly shift around to show people your contact info! When you are all done, make sure you share your slide deck on the interwebs, and also write a blog post about it (hopefully much faster than I did!).

If you have a chance, record yourself or have someone record you during your talk. Watch the video or listen to a recording of the talk as it was delivered.
Make notes and determine where you felt you needed more material, or where you need to practice more. Just because you gave this talk once doesn’t mean you won’t be able to give it again!


Well, that is a wrap for now. I have way more tips to share, but I also can’t type forever. If you need any advice or want to talk about my presentation feel free to reach out to me via Twitter or e-mail! I am planning on writing another article about the phases of DevOps with hopefully a bit more context and examples; look forward to that!

Below you will find the recording of my talk and also the slide deck for your viewing pleasure!