Concepts for a Modern Planet-Scale Application

12 min read Gossip Protocols Gossip protocols, also known as infection or epidemic protocols, are a category of protocols used for peer-to-peer communication that provide additional information about the network which they know about and accept but aren’t the authoritative source of. Some distributed systems use peer-to-peer gossip to ensure that information is efficiently disseminated to all members of a group. A really good use-case for this is network discovery and state maintenance in a very large network, where direct chatter between all nodes would waste a lot of bandwidth, and especially useful…

Signing Git Commits Using YubiKey on Windows

8 min read There are several things we need to do in order to achieve end-to-end security in our release pipeline. In this post, I’ll explain how to set up signing git commits and store the private key on a YubiKey using it as a smart card. Signing our commits is especially important in public projects like those on GitHub, to avoid people impersonating us. For private projects and later on in the build pipeline, we can validate that all our commits are signed by trusted parties, and add gates to protect against unauthorized code making it into our products.

Authentication Ideas

6 min read Security, and particularly around authentication, authorization, and auditing, is my favorite part of software development. It’s the stuff that not just lets us be safe, but rather, the reason I like it so much is that it’s by far the broadest part of software development. It requires us to understand the full breadth of the field, from hardware security components like TPM (Trusted Platform Module) chips to IETF standards-based protocols that not only make things safer but open the door to creating simpler, better, and more integrated systems. Historically it may not have always been the case, and security was at odds with other fields like performance and usability. Those problems have long been addressed now, once we realized that thinking of systems as having behavior emergent from the interaction of many systems and focusing on the end problem we’re trying to solve, instead of trying to fit the problem into an isolated individual system.

This new way of thinking gave way to new fields such as Systems Engineering, where the focus moves to focus on discovering the real problems that need to be resolved and identifying the most probable and highest impact failures that can occur. The domain of security, and organizations like (ISC)², OWASP and NIST have recognized and pushed the application of this understanding very well over the years, and standards have changed and become better.

One concrete example of this I think is NIST’s update to NIST 800-171 to remove periodic password change requirements, and drop the password complexity requirements in favor of screening new passwords against a list of commonly used or compromised passwords.

Observability using Elasticsearch and Neo4j

10 min read Elasticsearch continues to add features at an astonishing rate, and people find really creative ways to use them and enhance it even more. What Neo4j can do is just way too cool to pass on. So we’ll look at how to ingest data with elasticsearch and analyze the data with neo4j. Combining the two helps us achieve some really powerful solutions.

I originally was intrigued by elasticsearch for log aggregation and its capability to instantly aggregate and search over millions of records. We could ship logs from all sorts of data sources like application logs, web server logs (Nginx, IIS). Then we can filter through those logs in Kibana’s Discover, choose the columns we wanted to see for particular use-cases and create saved searches. This immediately made it useful to us, the engineering team. We then use query-based filtering to add restrictions on documents people should access, and with field-level security, we can control which fields they even see inside each document. All of a sudden we have the ability to give our level 1 support real-time visibility into customer issues, without overloading them. On top of this, we add Windows event logs and Syslogs and create some alerts.

Change Desktop Wallpaper with a USB Rubber Ducky

5 min read Rubber Ducky is the “technical” name of a USB device which looks like a USB thumb drive, but presents itself to the computer as a USB keyboard, which, once plugged in, starts typing away pre-recorded keystrokes at super-human speeds. It can bypass many IT safeguards since the computer detects it as a keyboard, and it can easily fool people. Someone might think they’re just plugging in a harmless USB thumb drive, or even just a harmless iPhone charging cable. These can be disguised as just about anything these days, and they can be microscopic.

Monitoring DNS Lookups against your DNS Server using Elasticsearch

7 min read DNS is one of the most fundamental components of today’s internet, one of the simplest technologies at the same time, but it’s also one of the biggest risks, and can be one of the best indicators when something is at risk. There are many cyber attacks that either have their root in DNS, or just use DNS just as a part of the structure of the attack without even thinking about it. Understanding your organization’s DNS traffic is a very solid step in protecting the overall organization. Monitoring DNS lookups with Elasticsearch and analyzing with machine learning can significantly reduce risk around several types of attacks. Here are some of the threats based in DNS and how to know about them.

Keeping Most of the Software on Your Computer Up-To-Date Automatically

5 min read This idea came to me when I was configuring Puppet Server to manage our Windows VMs. One thing that annoyed me for the longest time was having Notepad++ installed everywhere, and getting that popup that an update is available. Disabling the update check is not the solution. I realized that our server deployments are good. We build containers and automate installation of programs with package managers like Chocolatey on Windows and Yum on CentOS, and fully orchestrate servers and services. We can do something similar to keep our own personal computers’ software up-to-date using chocolatey, and without all the orchestration tech we usually needed. Basically just a scheduled task that runs a chocolatey script to keep our stuff up-to-date.

Common OpenSSL Commands and Using Let’s Encrypt

2 min read OpenSSL is a toolkit for generating and working with certificates, as well as a general-purpose cryptography library. While a very powerful tool, it also means that there are a lot of options, so here are a few commands I commonly find useful. We’ll cover some common OpenSSL commands to convert between certificate formats and containers, and getting a Let’s Encrypt certificate installed.

Combine a private key (.key) and a public key (.crt) into a password protected certificate archive / PKCS #12 format (.pfx)

openssl pkcs12 -export -out -inkey site_com.key -in site_com.crt

Hardware Security Modules (HSMs)

4 min read A hardware security module (HSM) is a physical computing device that safeguards and manages digital keys for strong authentication and provides cryptoprocessing. These modules traditionally come in the form of a plug-in card or an external device that attaches directly to a computer or network server. Hardware security modules are physical devices with auditing and encryption capabilities and are used to protect sensitive information such as encryption keys, as well as for their performance in encrypting/decrypting information. Once you load an encryption key into it, it can only be accessed/used…

Application Lifecycle Management / World-Class DevOps

9 min read

Development process

Scrum team, planning work in 2 week increments, reacting fast to change, following best practices in research, planning, architecture and writing software. The team’s focus is on building great software, so we want them focused on what’s important.

  • As a developer, I take on a new feature or bug that’s in the sprint and ready for development, and marks it as development in progress.
  • Then I create a new branch off of develop, and do the work.
  • Then commit one or more times, and mention #[Work Item Number] in the commit message, along with a description of what changed
  • Once I consider the work complete and “tested” locally on my computer, I’ll push the branch, go into TFS and click the shortcut to create a PR

* develop is locked, so all changes have to enter via Pull Requests
  • On the next page I’ll make sure everything looks good, by doing one more check of the changes in the code, and click Create
  • Done and move on to greater things …