Column Encryption for a database in an Always On Availability group

We created a master key with password etc, like this article:

https://docs.microsoft.com/en-us/sql/relational-databases/security/encryption/encrypt-a-column-of-data?view=sql-server-2014

I created the master key on all 3 always on servers.

All works fine on the primary server but can't seem to find a way it will work correctly if we fail over to the secondary. When trying to just open the symmetric key with the certificate, we get this error:

Msg 15581, Level 16, State 7, Line 31
Please create a master key in the database or open the master key in the session before performing this operation.

I tried opening the master key and then opening the symmetric key:

Msg 15581, Level 16, State 7, Line 31
Please create a master key in the database or open the master key in the session before performing this operation.

This makes me suspicious, this encryption may not work when on the secondary becomes the primary...
Is there a better way to encrypt a field in Always On?