Today I Learned

A Hashrocket project

9 posts by chasemccarthy @code0100fun

Migrate from MySQL to Postgres

Today I wanted to see if a large client app would see any performance improvement using Postgres vs. MySQL.

Luckily there is a great tool called pgloader that can quickly scan and import a MySQL DB right into Postgres.

Homebrew FTW!

$ brew install pgloader

Create an import script to define the connection strings to each of your databases and configuration the import options you want.

-- import_mysql.load
LOAD DATABASE
FROM mysql://root@localhost/db_name
INTO postgresql://localhost/db_name

 WITH include drop, create tables, no truncate,
      create indexes, reset sequences, foreign keys;

Run the import:

$ pgloader import_mysql.load

If you get this error:

An unhandled error condition has been signalled:
   MySQL Error [1055]: "Expression #6 of SELECT list is not in GROUP BY clause and contains nonaggregated column 'information_schema.rc.UPDATE_RULE' which is not functionally dependent on columns in GROUP
 BY clause; this is incompatible with sql_mode=only_full_group_by"

Then create a MySQL configuration file (if you don’t have one already) and open it:

$ sudo cp $(brew --prefix mysql)/support-files/my-default.cnf /etc/my.cnf
$ sudo vim /etc/my.cnf

Make sure ONLY_FULL_GROUP_BY is not in the mode list:

# /etc/my.cnf
sql_mode=STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION

Now restart MySQL and try to import with pgloader again.

Testem supports parallel execution!

It actually has for a while but it had some limitations. It’s recently been updated and can speed up your tests greatly! LinkedIn used this feature to take their test suite from 40 minutes to 2 minutes! 🚀😆❤️

You can define multiple test pages to split up the tests by filters:

{
   "test_page": [
     "tests/index.html?filter=acceptance",
     "tests/index.html?filter=integration",
     "tests/index.html?filter=unit"
   ]
 }

You can even set the number of parallel workers manually or set it to -1 to have testem max out whatever your CI server can handle:

{
  "parallel": "-1"
 }

If you want to do this in Ember you’re in luck! ember-exam makes this super easy! Thanks Trent Willis!

$ ember exam --split=<num> --parallel

How to test Facebook login with Ember + Torii

Configuring Torii is super simple. The only trick is that the name of the session conflicts with ember-simple-auth. I renamed mine to torii-session.

Configure

Calling this.get('torii').open('facebook-oauth2') triggers a Facbook dialog to show in another window. There may be a better way to get the token into simple auth but I wrote a synchronous method that calls some private APIs… don’t judge me.

User Login Service

Logging into Facebook triggers a postMessage to notify Torii of success or failure. Torii then POSTs the code to your backend using the adapter.

Torii Adapter

The adapter is simple enough to mock with ember-cli-mirage. I just return a token if the parameters exist.

API stub

The more challenging part (usually) is stubbing the Facebook authentication. Here is what I wanted to write:

Acceptance Test

Notice how ember-cli-page-object make the tests read so pretty!

Not surprisingly, Ember’s dependency injection and some great patterns in Torii make swapping the original facebook-oauth2 service for our stubbed service straight forward.

Stubbed OAuth2 Provider

There were only slight changes needed to test Facebook auth failure. Similarly, adding Google or Twitter authentication won’t require many changes.

Can't find variable: Symbol

Have you ever had an Ember CLI test that passes in Chrome but not in Phantom? Of course you have.

Did it have an error that looks something like this?

✘ ReferenceError: Can't find variable: Symbol

ReferenceError: Can't find variable: Symbol

Don’t (╯°□°)╯︵ ʇno dıןɟ ! You just have to enable the Babel Symbol polyfill.

// ember-cli-build.js
...
var app = new EmberApp(defaults, {
  babel: {
    optional: ['es6.spec.symbols'],
    includePolyfill: true
  }
});

Lemme fix that for you (ヘ・_・)ヘ┳━┳

All green!

How to donate to charity for free!

If you are like me, you make all of your big purchases on Amazon. You also would like to donate to charity but are not sure what charity to choose or how to be sure it’s a legitimate organization.

Choose a charity

You first need to sign up for Amazon Smile choose the charity you would like to support.

Choose a charity

Amazon will now donate 0.5% of the purchase price for every item to the charity you selected! That may not sound like much but that is $1.50 for every $300 you spend, and you have to do LITERALLY nothing. Well… almost nothing. You still have to remember to shop through smile.amazon.com, and honestly I never remember. So today I figured out a way to make it truly effortless.

Install Switcheroo

Amazon has a Chrome extension that’s supposed to give credit automatically but it also shows ads and I’m sure it tracks every page you visit. There is an open source too called Switcheroo that lets you establish custom redirects for any URL. Install Switcheroo

Configure Redirects

Configure Switcheroo

Now you will be redirected automatically and your charity will benefit from your last minute holiday spending!

The pitfalls of `rootURL` and `baseURL`

rootURL

Notice that the rootURL parameter has a trailing /. This is required and will raise an error otherwise. There is also an internal assert in Ember that raises if any URL does not start with the rootURL. For our example, if a user tries to navigate to /app/foo the app will throw an error.

fix

Sadly, the fix for this issue must to be implemented server-side.

baseURL

The problems with baseURL are subtle but have caused the core team many issues with testing, styles, history, etc.

fix

Most of the problems are actually related to the HTML spec and are unrelated to Ember.js such as certain SVG issues. As such, the only real fix for the <base> tag is to kill it and use broccoli-asset-rev to actually change the URLs at build time.

The difference between `rootURL` and `baseURL`

These properties do very different things yet they can be used to accomplish very similar goals. For this reason they are confusing and oftentimes used interchangeably. But beware, they also have very distinct pitfalls.

baseURL

//config/environment.js
 ENV.baseURL='/app/foo';

Results in:

<!--index.html-->
<base href="/app/foo">

So when the index.html loads an asset, the browser will internally prepend baseURL:

<script src="assets/vendor.js"></script>

is treated like:

<script src="app/foo/assets/vendor.js"></script>

rootURL

//app/router.js
var Router = Ember.Router.extend({
  rootURL: '/app/foo/'
});

Unlike baseURL this will not affect the asset URLs. This parameter tells Ember that the root index route starts at /app/foo/. So even if you define a route that has a path of '/' such as:

this.route('dashboard', { path: '/' });

Ember will recognize /app/foo/ as the dashboard route.

Would you like to know more?

Read about the pitfalls of rootURL and baseURL.

How to remove newly git ignored files

While developing a Cordova app I realized I was checking in files that are generated at build time. For example, the www folder is copied into each platform (ios, android). I was not sure what I could safely ignore until I looked at ember-cli-cordova’s gitignore.

So I changed my .gitignore to look something like:

...
# android
platforms/android/assets/www
platforms/android/bin
platforms/android/gen
platforms/android/local.properties
platforms/android/ant-build
platforms/android/ant-gen
platforms/android/CordovaLib/ant-build
platforms/android/CordovaLib/ant-gen
platforms/android/CordovaLib/bin
platforms/android/CordovaLib/gen
platforms/android/CordovaLib/local.properties
platforms/android/.gradle
platforms/android/res/xml/config.xml
...

But these folders are already checked in so now I have to git rm each of them.

Note: Don’t forget to use the --cached or you will actually delete the files

$ git rm -r --cached platforms/android/assets/www
$ git rm -r --cached platforms/android/bin
... and so on

This is super tedious and I could easily miss a file or directory. There has to be a better way!

$ git rm -r --cached . && git add .

Git already knows how to ignore the files in .gitignore so just untrack all files in the project and add them back.

The staged changes should now be the new lines in .gitignore and the deleted files that accompany the changes.

Force Ember Data to skip cache with reload: true

In Ember Data 1.13 many of the store methods for retrieving data changed to become more constent. However I missed a key aspect of those changes and somehow it has not effected my everyday Ember until now.

The way findAll and findRecord behave in regards to caching was changed so that the behavior would facilitate the most common use case. The behavior that the Ember Data team has implemented is:

  • First time store.find is called, fetch new data
  • Next time return cached data
  • Fetch new data in the background and update

This means that your app might seem to behave fine when you migrate to Ember[-Data] 1.13 or 2.* but there could be subtle places where hard refreshes on certain show pages makes navigating back to an index page render without all the other records loaded. This just happened with the EmberWeekend site.

The solution was to simply add { reload: true } to the options of your findAll queries.

store.findAll('user', { reload: true }); //enforces getting fresh data