Today I Learned

A Hashrocket project

26 posts by dillonhafer

Find host and port in development

Rails 5.1 is a little different the pre 5.1

require 'rails/commands/server/server_command'

In rails 5.0 or below

require 'rails/commands/server'
# lib/host_and_port.rb

def __host_and_port__
  options = Rails::Server::Options.new.parse!(ARGV)
  options.values_at(:Host, :Port)
end

You can then find the host and port for various configuration files.

# config/initializers/carrier_wave.rb
require Rails.root.join('lib/host_and_port').to_s

CarrierWave.configure do |config|
  config.asset_host = "http://" + __host_and_port__.join(":")
end

or

# config/environments/development.rb
require Rails.root.join('lib/host_and_port').to_s

host, port = __host_and_port__
config.action_mailer.default_url_options = { host: host, port: port }

Rails/PG Statement Timeout 🐘⏰

By default, Rails does not set a timeout on database statements. For example, this will run for a full day, even if your ruby process goes away.

ActiveRecord::Base.connection.execute(<<~SQL)
  select pg_sleep(86400);
SQL

The good news is that Rails provides a way to prevent this from happening from the database.yml with the statement_timeout variable.

default: &default
  adapter: postgresql
  ...
  variables:
    statement_timeout: 5000

I’m starting to think that this should always be set low. And then explicitly set higher on a per-query basis, when one is expecting something to take a long time.

group by 1, 2, 3 🔢

Postgres allow group by and order by to reference column order in a sql statement. This is particularly useful when an aggregate result needs to be referenced in the group by or order by statement.

-- group by aggregate
select
  (created_at at time zone 'UTC' at time zone 'America/Chicago')::date,
  count(*)
from posts
group by (created_at at time zone 'UTC' at time zone 'America/Chicago')::date
order by (created_at at time zone 'UTC' at time zone 'America/Chicago')::date
;

becomes

-- group by 1, 2, 3
select
  (created_at at time zone 'UTC' at time zone 'America/Chicago')::date,
  count(*)
from posts
group by 1
order by 1
;

Open images in vim with iTerm 🖼

iTerm 3 has a built in shell script called imgcat for displaying images in the terminal. With one simple autocmd in my vim configuration I can open images*:

:autocmd BufEnter *.png,*.jpg,*gif exec "! ~/.iterm2/imgcat ".expand("%") | :bw

In my command I wipe the image buffer(:bw) because I don’t want large images sitting around in buffers, but this is easy to change.

imgcat

*imgcat does not work with tmux

Use PostgreSQL socket in database.yml 🐘🔌

When using the host: configuration option in the database.yml set to localhost or 127.0.0.1, I would need to add an entry to PostgreSQL’s pg_hba.conf file to allow my ip address access. But, if you give the host: option the directory of your PostgreSQL sockets, rails will be able to use the socket, without needing to add an entry to the PostgreSQL configuration file.

production:
  database: fancy_things
  host: '/var/run/postgres'
  ...

Drop connections with Nginx

You can ignore requests sent to nginx by responding with the status 444

server {
  listen 80;
  listen [::]:80;
  listen 443;
  listen [::]:443 ssl http2;

  server_name dillonhafer.com;
  location /no-response {
    return 444;
  }
}

The primary use for this is to prevent processing requests with undefined server names:

# block for requests with undefined server names
server {
  listen 80;
  listen [::]:80;

  return 444;
}

Respect Do Not Track 👁‍🗨

Before I put google analytics, adwords, or other third party scripts on my sites I can respect the DNT headers a user has set. Here’s how I can do it with Google Analytics:

<script>
  if (window.navigator.doNotTrack !== '1') {
    (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
    (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
    m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
    })(window,document,'script','https://www.google-analytics.com/analytics.js','ga');

    ga('create', 'UA-xxxxxxx-5', 'auto');
    ga('send', 'pageview');
  }
</script>

In the above snippet I check if the user has enabled DNT. If they haven’t set the header or they have turned it off the script will run. Checking for window.navigator.doNotTrack !== '1' ensures that the user purposely set the header.

Chaining TLS Certificates 🐱

Apache allows you to declare an intermediate TLS certificate along with your regular certificate in your configuration, but many web servers only allow you to provide one certificate option. Like Nginx, go, or heroku.

In those cases, you will need to concatenate the entire certificate chain into one certificate file. This may sound daunting, but the process is very simple. Let me introduce cat -- concatenate and print files.

cat is normally used for printing files, but in this case we actually want to concatenate files. Below is a simple example on how we can do this:

cp example_com.crt example_com.chained.crt
cat AddTrustExternalCARoot.crt >> example_com.chained.crt
cat COMODORSAAddTrustCA.crt >> example_com.chained.crt
cat COMODORSADomainValidationSecureServerCA.crt >> example_com.chained.crt

To shorten it, we only need to use cat once:

cp example_com{,.chained}.crt &&
cat AddTrustExternalCARoot.crt COMODORSAAddTrustCA.crt COMODORSADomainValidationSecureServerCA.crt >> example_com.chained.crt

Verify TLS cert with private key

Hopefully you’re never in a situation where you don’t know what private key you used to generate your TLS certificate, but if you do… here’s how you can check.

Note: this is better than uploading the certs to production to check on them 😉

Assuming we have generated a private key named example.com.key and a certificate named example.com.crt we can use openssl to check that the MD5 hashes are the same:

openssl x509 -noout -modulus -in example.com.crt | openssl md5
openssl rsa -noout -modulus -in example.com.key | openssl md5

To make things better, you can write a script:

#!/bin/bash
CERT_MD5=$(openssl x509 -noout -modulus -in example.com.crt | openssl md5)
 KEY_MD5=$(openssl rsa  -noout -modulus -in example.com.key | openssl md5)
 
if [ "$CERT_MD5" == "$KEY_MD5" ]; then
  echo "Private key matches certificate"
else
  echo "Private key does not match certificate"
fi

Trust Issues 🤔

Our computers trust a scary amount of Root Certificate Authorites, and sometimes I have trust issues with some of them. Most recently being the StartCom bug, which allowed anyone to get a certificate for any domain they wanted.

I can’t trust them. Period. And I don’t have to.

Here is how you can revoke trust for any Root CA in OSX:

  1. Open Keychain Access.
    open /Applications/Utilities/Keychain\ Access.app
  2. Click on System Roots from the left Keychains sidebar.
  3. Typestartcom in the search bar.
  4. Select all the root certificates and press ⌘i.
  5. Expand the Trust section
    and change the option
    When using this certificate
    to
    Never Trust.

Put -L in your ssh config

If you’ve used ssh you’ve probably had to use local port forwarding before. If you use complex flags often, you will find remembering what you need is tiresome. Stop. Write it down so you can forget all about it. Save your memory.

Instead of doing this:

ssh -L 3000:localhost:3002 not_dillon@example.com

Add those flags to the ~/.ssh/config file instead:

# /home/dillon/.ssh/config

Host example
  User         not_dillon
  Hostname     example.com
  LocalForward 3000 localhost:3002

From now on, all you need to type is:

ssh example

The Case of the Default 🕵

In Apple’s Swift language switch statements must be what apple calls “exhaustive”. I’ve felt the term to be very literal. Literally exhaustive?

Example that does not work:

let count = 42

switch count {
case 1:
  print(1)
case 7:
  print(7)
}

The above statement does not work because it’s missing a default case. Why? What if I don’t want to do anything else? Why do I need to write something that won’t be used? Don’t worry, there is an amazing and less “exhaustive” way to handle these situations; simply default: ()

Correct example:

let count = 42

switch count {
case 1:
  print(1)
case 7:
  print(7)
default: ()
}

-y apt-get?

When installing packages on Ubuntu, you may find it really tiring to constantly confirm ‘yes’ all the time. I know I did. And when it comes to scripting your installs… that really becomes a nuisance. Today I learned that apt-get has a -y flag:

Automatic yes to prompts;
assume "yes" as answer to all prompts and run non-interactively.

Rails Sandbox 🏖

When you are working with a complicated database structure, and find yourself needing to debug a complex or dangerous (delete) action, you might be hesitant to experiment. Keep experimenting! Don’t want to setup all that data again? No worries. You can use a sandbox $ rails c -s

Usage: rails console [environment] [options]
  -s, --sandbox      Rollback database modifications on exit.

The sandbox flag will keep all database changes in a database transaction when you start the rails console and automatically issue a rollback when you quit the console.

Keep mina fast

One thing I’ve loved about mina is the speed of deployments. One way mina achieves fast deploys is by avoiding unnecessary tasks.

I recently started using webpack with rails and soon found I needed that same performance boost. Because I was using mina/rails I had a nice little macro already available for me 😁

Example of #check_for_changes_script

desc "Install npm dependencies"
task :install do
  queue check_for_changes_script \
    check: 'package.json',
    at: ['package.json'],
    skip: %[echo "-----> Skipping npm installation"],
    changed: %[
      echo "-----> #{message}"
      #{echo_cmd %[NODE_ENV=#{ENV['to']} npm install]}
    ],
    default: %[
      echo "-----> Installing npm modules"
      #{echo_cmd %[NODE_ENV=#{ENV['to']} npm install]}
    ]
end

Encrypt a zip archive

When you’re using the zip CLI on your machine or a remote server and you need some extra security you can use the -e flag to encrypt a zip archive. Be sure to use a super long random password for this.

zip -e secure-files.zip ~/Documents/*.pdf

-e       
--encrypt
       
  Encrypt the contents of the zip archive using a password which is entered on
  the  terminal  in response to a prompt (this will not be echoed; if standard
  error is not a tty, zip will exit with an error).  The  password  prompt  is
  repeated to save the user from typing errors.

You have new mail.

Sometimes you forget to give that cron task somewhere to output its results… so it dumps them in your mail box… 📬😣 all 30,000 times it failed to run.

You don’t have to delete all those emails one by one 😉

echo 'd *' | mail -N

d * means delete all mail. The -N means:

-N      Inhibit the initial display of message headers
        when reading mail or editing a mail folder.

Use rsync with a custom port

You can rsync over a custom port for cases when you have custom ssh tunnels setup using rsync’s -e flag. The -e flag tells rsync to use a specific shell, in this case it’s still ssh, but with the addition of the -p flag.

Setup a custom tunnel

ssh -N -L 2222:10.0.0.2:22 user@remote-machine

Use rsync over the custom tunnel

rsync -avz -e 'ssh -p 2222' localhost:/files /myfiles

Limit SSH by IP address

SSH has some nice security features like password-less authentication. For the sysadmin who wants even more fine-grained control, the ssh configuration has a setting for what users are allowed to authenticate based on their IP address.

# /etc/ssh/sshd_config
AllowUsers admin@127.0.0.1 deployer@127.0.0.1 git@*

This configuration says that admin & deployer are only allowed to login from 127.0.0.1, but git can login from any IP address. You can also use wildcards the other way:

AllowUsers *@127.0.0.1

The most useful thing here is the ability to use * to match users and hosts.

h/t Chris Erin