Deployed a mastodon instance today

As of today I run a mastodon instance, targeting Dutch speakers, although everyone could sign up if they want to.

The instance can be found at https://olifantje.net (olifantje is small elephant in Dutch).

Feel free to sign up or to sign up at another mastodon instance and follow me at https://olifantje.net/@eelco.

The Friendica instance at https://hollandsepod.nl will remain too, love the RSS integration in that one.

Voor de Nederlandse en Vlaamse lezers : voel je vooral welkom om aan te melden voor een account op olifantje.net. Het zou leuk zijn om een voornamelijke Nederlandstalige node te krijgen in het netwerk 🙂

Running encrypted backups with duplicity

This is just a short note on my experiences running backups with Duplicity.

Duplicity is an open source package that allows you to do incremental backups, complete with proper indexing, to remote storage. This can be a modern ‘cloud’ storage like S3, but I prefer to run it over a simple SSH link.

Next to properly working incremental backups, it also provides data security by using GPG to encrypt the data. And it has a lot of stuff you would expect : configurable full dump cycles, purging of old  backups. There is a windows / C# implementation too (haven’t tried it though)

The only thing lacking may be deduplication, which is kinda hard given that all data is encrypted.

It took me some time to get all the parameters right, but after some initial fiddling, I wrapped it all in some puppet code that gets deployed to all new machines / nodes.

So every new machine is backup up automagicly using duplicity by only applying my basic puppet profile to the host.

I also did an extensive restore test during the implementation phase with went fine.

Highly recommended little know tool in some dark corner of the Internet : http://duplicity.nongnu.org/ . Don’t let the HTML 1.0 web design turn you off, this tool is maintained and stable.

Choosing a distributed social platform is not simple

Fiddling around with open source / free alternatives for social networking has reconfirmed the one major problem with open source : there are so many forks to choose from.

To help someone choose a linux distro to run will end op in a short questionnaire about the needs of the person asking for the advice. And then it still remains a shot in the dark. Just get the live CD and see if you like it.

The same seems to hold true for open social networking. There are plenty of tools around, all with their own focus, maturity and limitations. And choosing which one is right for you requires actual test-driving of the tool. Which for a social networking platform is not as easy as a Linux distro.

Does federation work as I want it? Can I consume other content? Will it integrate with platform X or Y? You can only find out by actually running it. And than you can either risk your real account / network to integrate with or spend more time setting up a sandboxing environment. Both are not really attractive options when you just try to spin the wheel to get a feel.

During the last couple of weeks it became clear to me that there are options that I could use if I tried. But the solutions I’ve seen are nowhere near something that my wife or kids would like to use. Or any other people in my network so it seems 😉

After a small experiment yesterday night with Hubzilla, I just switched my Pod back to Friendica, as the stuff broken/missing in Hubzilla is more annoying to me than the stuff broken/missing in Friendica.

There is work to be done.

Released puppet module for ispprotect to the forge

This week I wrapped up version 0.3.0 of the ispprotect puppet module I created for the Erasmus University Rotterdam. The module manages the installation and scheduling of ISPProtect, a php malware scanner that also scans for outdated versions of popular PHP applications. The module can be found at https://forge.puppet.com/eelcomaljaars/ispprotect. More information in ISPProtect can be found at https://ispprotect.com #puppet #php #security

Friendica forums and community pages · friendica/friendica Wiki · GitHub

Pagina’s / forums maken over een onderwerp op hollandsepod.nl

Op veler verzoek (1) heb ik even uitgezocht hoe je een pagina/forum kunt maken. Volgens de documentatie op de wiki van friendica is het zoveel als een extra account maken en vervolgens je eigen account rechten delegeren, zodat je namens de pagina kan optreden. Zie ook de gekopieerde instructies :

To make forum management easier if you are creating a forum on the site you normally use (i.e. for your personal account), do this:

Log off.

Register a new account for the forum using the same email address. Wait for the confirmation email, then log on using the new identity.

Now issue a contact request to your other identity – your personal one. Then log off again.

Go to your personal identity and accept the contact request. Log off yet again – and log on once more with your forum identity. Visit http://YourDomain.com/delegate (replacing YourDomain.com with your own domain name). You should now be able to appoint yourself (as a person) to help moderate your forum.

Sounds complicated – but it’s really quite straightforward, and there’s a significant advantage to the approach: In future, you will only have to log on with the forum identity to change settings. You can perform all moderation tasks from your personal account (where you will discover a new Manage tab for the purpose).

Of course, from now on you must use nicknames rather than your email address to log on to your accounts – either the forum nick or your personal nick, depending on ‘who’ you need to be. If you try to log on using your email address, the system can’t know which account you want to use.Friendica forums and community pages · friendica/friendica Wiki · GitHub

friendica – Friendica Communications Platform

Puppet – Server Error: hiera configuration version 3 cannot be used in an environment

Screenshot of failing puppet-agent runToday I found my homelab puppet setup failing. I mostly run the latest released puppet code on my setup and this time it bit me in the ass :

Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: hiera configuration version 3 cannot be used in an environment on node buzz.maljaars-it.nl
 Warning: Not using cache on failed catalog
 Error: Could not retrieve catalog; skipping run

When looking it at the issue, I found that my setup had automatically been upgraded to puppet code version 4.9. This version supports a newer hiera config file version without maintaining backward compatibility apparently.

Looking for documentation on the new format, it seems to be missingat this time. Searching the history on the puppet community slack seems to confirm this.

For now I downgraded the puppet-agent package on the puppetserver machine and pinned it to the previous release. After restarting the puppetserver service everything was fine for now.

More info on this issue and other new features of puppet 4.9 : https://docs.puppet.com/puppet/4.9/release_notes.html

Third party apps crashing on MacBook Pro Late 2016

The experience with my MacBook Pro late 2016 have been mostly positive. There is an annoying thing going on with third party apps, including Firefox and KeepassX, crashing. The crashes always report themselves when I unlock my laptop. So I suspect it occurs during sleep or wake of the laptop.

It only happens to non-native third party applications it seems, so my hunch is that is somehow related with GUI libraries in combination with the new graphics chip/driver.

I sent off crash reports to both Apple and Mozilla. Will send a report the the KeepassX developers too today.

For now I will run these applications in an Ubuntu VM.

Small update : the list also includes VLC media player.

Update 24-jan-2017: The latest release of MacOS (10.12.3) seems to have fixed the issue. Make the Mac Great Again!

Validating puppet code changes using octocatalog-diff

During Puppetconf 2016 Github announces it was releasing one of its internal test tools called octocatalog-diff as open source. What the tools basicly does is compile the catalog for a certain machine using your old and new puppetcode to show you the diff output. This allows you to see the impact of your code without actually deploying and running it on puppet clients.

As this could potentially save me a lot of time, I decided to delve into the tool to see what it could bring me. This post describes to the proces of setting the tool up and the surprises I came across.

Environment

For my initial testing I used a vagrant provisioned box running Ubuntu 16.04. Next to that I installed the puppet agent 3.8.7 that is in the standard Ubuntu repositories.

Preparation

The following gems were needed during my trails. Just run ‘sudo gem install [GEMNAME]’ to get them on your system. When using Puppet v4 take note to use the gem binary in /opt/puppetlabs/puppet/bin/gem

  • hiera-eyaml
  • r10k
  • bundler
  • rspec

The tool can query the puppetdb instance of your setup, or you can use the facts in yaml format for the node(s) you are testing. As the puppetdb did not work for my at this point, I copied the contents of /var/lib/puppet/yaml/facts from my puppetmaster to my local test environment.

The tool also needs some develop utilities upon install. Get them using your systems package manager :

  • cmake
  • pkg-config
  • ruby-dev

Installing the tool

After trying the gem, I’ve installed the source version following the instructions on https://github.com/github/octocatalog-diff/blob/master/doc/installation.md . The rake test gave me a a couple of errors that I reported.

Setting up

There were three files I added to my puppet repo :

  • A hiera.yaml configuration valid for my setup
  • A bootstrap script to run r10k on my Puppetfile and to symlink local modules to the common modules dir (see below)
  • A .octocatalog-diff.cfg.rb configured as per the instructions on github Please note the the current example file has a key settings[:hiera_yaml_file]  that should read settings[:hiera_config] . (Github issue)

Running the tool

/vagrant/octocatalog-diff/bin/octocatalog-diff -f production -t [your current branch] -n [NODENAME] --fact-file /vagrant/puppetdb_facts/[NODENAME].yaml --to-fact-override vagrant_puppetrole=nagiosserver --from-fact-override vagrant_puppetrole=nagiosserver --bootstrap-script repo-bootstrap.sh

Installation paths may vary. Note: the fact override thing is my way to assign a role to host without a real ENC / foreman. This value is used by a small piece of code in my site.pp :

node default {
  ## This is a small hook to support local vagrant
  ## development. This special var get set as part
  ## of the vagrant provisioning process.
  if $vagrant_puppetrole != undef {
    class { "roles::${vagrant_puppetrole}": }
  }
}

When removing a single package from my ‘baseline’ it resulted in the expected output:

screen-shot-2016-11-03-at-14-26-40

 

 

 

diff production/NODENAME fea-puppetv4/NODENAME
*******************************************
  Package[tcpdump] =>
   parameters =>
     ensure =>
      - present
      + absent
*******************************************

Contents of  repo-bootstrap.sh

r10k puppetfile install Puppetfile
for i in site/*; do BLA=`echo $i |sed -e 's#site/##'`; ln -s ../site/$BLA modules/$BLA; done

Final notes

When trying to use the tool, I started out using Puppet 4.8. I run into some trouble with puppetlabs firewall module 1.8.1 (( [Puppet Error] Could not autoload puppet/provider/firewall/iptables: undefined method `value’ for nil:NilClass) . As soon as I downgraded to puppet 3.8.7 the firewall module stopped producing errors. Not sure if this is related to the puppet version or the combination with octocatalog-diff.

I will use the tool the coming weeks. The next step could be integrating it in the build pipeline(s).

 

Improving security by sandboxing work for different purposes on my laptop

During my freelancing time over the last couple of years, I’ve been trying different setups that would satisfy my wish to use a single laptop for different  clients whilst maintaining proper security.

After all, I do not want to explain my client that their servers got infected with Crapware because I felt the need to run Spotify on my laptop or visiting an infected site that one time.

Naturally as a infrastructure guy, I run a lot of virtual machines. The Vagrant + Virtualbox is a very good fit for puppet development work.

I tried running some of the ‘production’ work in VM’s, but it never really panned out. Having a dedicated homebanking VM or a ‘Client X VPN VM’ just was too much a different workflow to feel natural and effective.

So now I decided to split workloads in set of task with a similar security need / boundary and create VM’s for that.

Virtual machine setup on my bring your own device setup.

To improve the security in the virtual machines, there is only limited VMWare filesharing between the VM and the host OS, only a specific folder. Also, I disabled drop and drop + copy and paste support in the VMWare options.

I try to minimize risk in the host operating system, by perfoming the least amount of tasks directly in this layer. I suspect I can further reduce the tasks that I run in the host OS with a future generation of hardware.

Currently I still do puppet development / testing in the host OS. Running the virtualbox VMs nested in a VMware VM leaves me too much of a performance penalty.

The biggest limitation I’ve run into is video support in the OS X virtual machine. Video does work OK enough but it lacks proper retina support, so screen elements are really tiny or kind of blurry as before retina. Not a show stopper though.

Will report in a couple of weeks or so how well it went.