Skip to content

Introducing: Vault Project or How to setup an Encryption Server

April 01, 2016

Let’s talk about credentials. Credentials! Who doesn’t work without any secret data which should never go public? Sure, sure, we’d never share those secrets public, but then there’s git, and github and.. BAAMM.. credentials exposed. Upps!

To avoid this, it’s common sense to NOT checkin any credentials. NEVER. EVER. We make use of .gitignore, cfg templates and placeholders. But nevertheless it’s a hassle, especially when working in a team, where a credential exchange is sometimes required. In past, I preferred the solution where an encryption server in a save environment took care about encrypting/decrypting data. Once the data is encrypted, it’s save to checkin, store and share in the same way like any other data. Only users (or systems) with access rights for the encryption server can decrypt the data. Instant WIN! But how to setup such a server?

Introducing: Vault Project

I recently came across the Vault Project which exactly meet all my needs. In addition, it’s open source, simple to use and comes with very good documentation and tutorials. Nevertheless, I couldn’t find all the steps required to setup an encryption server in one place, so here they are:

How to setup an Encryption server with Vault

First, setup a new vault server on a remote machine by following those steps:

    1. Create a config file docs

touch vault.cfg

For our case we just need a simple file backend and expose the vault via tcp

backend "file" {
  path = "/Users//.vaultstore"
}


listener "tcp" {
  address = "127.0.0.1:8200"
  tls_disable = 1
}

See the vault config docs for other options.

    1. Now start the server …

vault server -config vault.cfg

    1. … and run the init procedure: docs

vault init -address=http://127.0.0.1:8200

This will output five keys and a root token. Make sure to keep those keys save, once lost you wouldn’t be able to unseal your vault, and therefore gain access. The root token is needed to authenticate against the server. To remove the need of passing in the host address all the time, you can set the value with VAULT_ADDR too: export VAULT_ADDR=http://127.0.0.1:8200

On start, the vault is sealed, and can only be opened with the keys generated at the very beginning. Unseal is done with:

vault unseal

Execute this three times, and enter a different key each time.

DONE! Now the server is up and running and you can connect from you local machine to the vault.

    1. Authenticate with the server

The vault only accepts authenticated connections, for that create a auth token with:

vault auth

Enter the root token-key obtained from step 1. Later, I’d strongly recommend to use a non-root token.

    1. Enable github auth backend docs

In order to allow other people access to the vault and decrypt data, it’s the easiest to enable github authentication. In this case, every user part to a specific github team is able to obtain an auth-token themselves.

Enable github auth:

vault auth-enable github

Register github org:
vault write auth/github/config organization=

…and a team:
vault write auth/github/map/teams/ value=root

Now, any team member can get access to the vault with a github token:
vault auth -method=github token=

Whereas `` only requires the ‘read:org’ scope to be granted. Once auth is complete the actual vault token will be stored in ~/.vault-token and allows the user to connect to the vault.

7.Enable transit backend docs

Per default, vault stores data associated with a key. But for our case, we want to encrypt data on the fly and manage storage within our SCM instead. Encryption only is enabled with the ‘tranist backend’:

vault mount transit

Add a key name to generate an encryption key:

vault write -f transit/keys/

The actual encryption key can be retrieved via:
vault read --format=json transit/raw/

DONE! Now the encryption server is ready to encrypt. For this I used json as response format and jq to extract the data.

  • 1.Encrypt:

    echo -n “” base64 vault write –format=json transit/encrypt/ plaintext=- jq -r .data.ciphertext
  • 2.Decrypt:

    vault write –format=json transit/decrypt/ ciphertext= jq -r .data.plaintext base64 -D

Finally, let’s create some bash functions to make life easy:

export VAULT_ADDR=""
export VAULT_KEY=""

function _encrypt {
  base64 | vault write --format=json transit/encrypt/$VAULT_KEY plaintext=- | jq -r .data.ciphertext
}

function _decrypt {
  vault write --format=json transit/decrypt/$VAULT_KEY ciphertext=- | jq -r .data.plaintext | base64 -D
}

# encrypt data.
# Usage: encryptd ""
function encryptd {
  echo $1 | _encrypt
}

# decrypt data.
# Usage: decryptd 
function decryptd {
  echo -n $1 | _decrypt
}

# encrypt file. will overwrite existing ones! 
# Usage: encryptf 
function encryptf {
  cat $1 | _encrypt > $1.enc
}

# decrypt file. will overwrite existing ones! 
# Usage: decryptf 
function decryptf {
  cat $1 | _decrypt > `basename -s .enc $1`
}

# decrypt file and open for edit. on close encrypt changes.
# Usage: editcrypt 
function editcrypt {
  cat $1 | _decrypt > $1.tmp && $EDITOR $1.tmp && cat $1.tmp | _encrypt > $1 && rm $1.tmp
}

Sweeeeeett!

On thing to mention is adding the non-encrypted files to you .gitignore. This ensures the file wont end up in your repo by accident:

$ cat .gitignore
# only checkin the encrypted version
/database.yml
/!database.yml.enc

I hope this helped setting up your own vault server. These are just the first rough steps. Vault allows way more, like very granular access management, various auth and storage backends etc. Again, I strongly recommend to check their docs https://vaultproject.io/docs and follow the interactive tutorial https://vaultproject.io/#/demo/0.c

Keep your data save!

[proofread by Daniel - thanks!]

Testing Deepdive Sumup

February 16, 2016

Here my follow-/sum-up of a Testing deep dive I gave:

Motivation for writing Tests

  • To avoid manual process (Out of pure laziness)
    • Don’t repeat your manual work flow :D
    • test setup/preconditions are exactly the same
    • ideal you never have to start irb / application server to test
  • To verify the code aligns & fulfils all the assumptions
    • gain confidence
  • To indicate other what’s important
    • to secure that new code doesn’t change/break current state
    • to avoid surprises
    • give confidence
  • Documentation
    • allows other to understand you code quickly
    • good example -> https://www.relishapp.com/rspec/rspec-core/docs/example-groups/basic-structure-describe-it
  • Avoid Bugs before they even exists ;)

Test Types

Follow the Pyramid (http://2.bp.blogspot.com/-YTzv_O4TnkA/VTgexlumP1I/AAAAAAAAAJ8/57-rnwyvP6g/s1600/image02.png)

  • (little) End2End (Acceptance),
    • no stubbing
    • full scenarios
  • (some) Integration
    • cover the full interface your code integrates (different call/inputs, return values, exceptions)
    • static data
    • some stubbing
  • (at lot of) Unit
    • Low level
    • stub external
    • all public method, each condition/parameter combination should be a test case
  • refactor usually break unit, but shouldn’t integration and for sure not End2End

Test Structure

  1. Setup static State (be aware of Time)
  2. Execute
  3. Assert

-> http://betterspecs.org

  • Prefer copy & past over magic/bad abstraction
  • be simple, explicit, verbose

Little Helpers

GitPrePush Hook: https://gist.github.com/rngtng/ea4c265704cdc04a384c

  • copy this in .git/hooks of every project
  • skip with git push --no-verify

See all test statuses:

Download http://ccmenu.org, create a access token here https://circleci.com/account/api and use this url in ccmenu:

https://circleci.com/cc.xml?circle-token=<token>

Keep in mind

  • make test should always work on your local machine
  • Avoid introducing code/merging PR when dependencies build are broken
  • Make sure new code integrates nicely with dependencies/or update dependencies asap.
  • Rebase branch on current master(HEAD) before merge

Pls share your opinions/experience/ideas on how to kill all the bugs and to maintain a simple & obvious development process for everyone!

Important Git Commands

December 18, 2015

Useful Git commands

  • git pull equals git fetch + git merge (all)

better to know what your’ doing:

git fetch
git merge

add

git add -p # select specific lines to stage
git add .
git add -u 
  • git stash
  • git push
    • push hooks
    • tracking branch
  • git remote
    • origin

see changes

git diff 
git diff --cached

… or stick to gui

gitx - http://gitx.laullon.com/

cleanup

git clean
git reset --hard master
git remote prune origin

rebase

git rebase master

interactive

git rebase -i master

if merges within branch use Hannes lye:

git lye - https://gist.github.com/855956

git merge --squash
git comit -v

Sugar

git hub
git compare

local:

git whatchanged master..head

http://stackoverflow.com/questions/53569/how-to-get-the-changes-on-a-branch-in-git/2831173#2831173

config

[push]
  default = current
  • git amend

Berlin Maker Fair 2015 - Findings

November 28, 2015

I went to Berlin Maker Fair 2015

Here some Projects which caught my attention the most:

Brick´R´Knowledge - https://www.brickrknowledge.de/en

  • an innovative plug system which can be used to tranfer electronical knowledge very easily, fast and playfully

Seeedstudios RePhone - https://www.kickstarter.com/projects/seeed/rephone-kit-worlds-first-open-source-and-modular-p

VaiKai - http://vaikai.com

  • intuitive connected toys for less screentime and more play.

TinkerBots - http://www.tinkerbots.com/de/

  • a Lego ‘extension’ for smart senors and actor modules. Super simple to control and programm. For Kids starting from 6years!

ManuGoo - http://manugoo.de

OpenHive - http://www.open-hive.org/

  • an open Source Bee Hive Monitoring Project

I got myself a Oscilloscope - MSO5102D

November 16, 2015


It’s a Hantek MSO5102D. See full Details here: http://www.hantek.com/en/ProductDetail_83.html

Review:
http://web.archive.org/web/20150317231035/http://www.legomindstormsrobots.com/miscellaneous/electronics/100mhz-digital-storage-osilloscope-logic-analyzer/

10 Things to consider before buying a Oscilloscope:
http://www.ni.com/white-paper/4333/de/

Call by value vs. Call by name

September 14, 2015

Call by Value

def log(out)
 puts out
 out = 1
 true
end


start = 100
puts start + 10
log(start)
log(start)
log(start.dup)


start = 100
log(start)
puts start + 10

Call by Name

def log_a(out)
 puts out.size
 return out << 1
end


start = [100]
puts start.size
start = log_a(start)

start = log_a(start.dup)
start = log_a(start.dup)
if (start.size > 1)
end

The Expert

September 03, 2015

https://www.youtube.com/watch?v=BKorP55Aqvg

How I revived my iPod classic 4th genation with Flash Memory!!

August 10, 2015

Since a very long time, I had my old 4th Generation iPod laying around. With iPhone, SoundCloud & Co, I hadn’t had any use for it - until now: my Camping Van came ‘only’ with a CD Radio and an Aux-In - which is perfect for my Ipod. If it would only work. Battery was down, and even worse - the Harddisk crashed. But a quick google made me hope: there’s indeed a chance to replace the HD with Flash memory. Faster, cheeper and less power consumption. I had to try it.
So I followed those super easy steps Eddie posted on instructables:

http://www.instructables.com/id/Convert-your-4th-Gen-iPod-to-use-Flash-Memory

Main trick is to get a ‘IDE 50 Pin Male zu CF Compact Flash Female Adapter’ - on ebay or similar for just 5 EURs. Compact Flash memory you get for abt. 1EUR per GB, which int total, made the 40GB replacement quite cheap. It just took me minutes to replace the harddrive, including a new Battery. Now my more than 10 years(!!) old iPod works better like never before! Amazing.

Happy Hacking!

UPDATE: make sure you check DIAG-Mode and HW-Reset too:

First what I would do is wait for it to reboot itself and as soon as the screen turns off begin holding the Center + Play/Pause buttons until Your iPod Enters Disk Mode and THEN plug it into your iMac and select “Restore”. =) Also when you get the chance, Hold Menu + Center and as soon as the screen goes out, Hold Center + Previous/Rewind. You will get a White Screen, select “Manual Test” and then go to “IO” then “Hard Drive” and then select Hard Drive Smart Data and type every character on that screen up here, I will be able to tell you if your iPod’s internal hard disk is going bad

https://discussions.apple.com/thread/2609298?tstart=0

Satzuma Missile Launcher finally works with USB Missile Launcher NZ v1.8.2 on Mac

May 23, 2015

img8077_14925

Good news: I finally got the Satzuma Missile Launcher working on my Mac Yosemite. Solution it the latest (unfortunately unreleased) version of USB Missile Launcher NZ. You can download USB Missile Launcher NZ v1.8.2 from here, a source I found after digging through the comments of version 1.8.1 announcement:

https://dgwilson.wordpress.com/2012/01/11/usb-missile-launcher-nz-v1-8-1-release/#comment-10069

To get started with Satzuma, install v1.8.2, restart you Mac (bummer), open USB Missile Launcher NZ.app, go to Preferences -> Launcher and
enter 1046 for VendorId, 3777 for ProductId and change Controls to Satzuma (see screenshot) - Boom! You’re all set - happy shooting!

Screen Shot 2015-05-20 at 11.41.11

Next, I want to check https://code.google.com/p/pymissile - having a working CLI version would be so much better!

Things 'NOT TO DO' to a Pinball Machine

May 03, 2015

For the record: a great post by Tim Arnold - things ‘NOT TO DO’ to a Pinball Machine:

http://www.zaccaria-pinball.com/misc/arnold.html

Mysql For Beginners

April 01, 2015

MySQL for beginners

Install

  • Homebrew -> http://brew.sh
  • brew install mysql
  • or: http://dev.mysql.com/downloads/mysql/5.5.html#macosx-dmg
  • SequelPro -> http://www.sequelpro.com

Start

  • Connect to root@localhost
  • Graphical vs. CLI vs. progamitacal

Basic Structure

  1. What is Database?
  2. What is Table?
    • columns + datatype (INT, VARCHAR, TEXT)
    • good practice: first column id as primary key & auto increment
  3. Setup:
    • add table users, columns: id (INT), name VARCHAR(255), lastname VARCHAR(255)
    • add table tracks, columns: id (INT), user_id (INT), name VARCHAR(255)
CREATE TABLE `users` (
  `id` INT(11) UNSIGNED NOT NULL AUTO_INCREMENT,
  `name` VARCHAR(255),
  `lastname` VARCHAR(255),
  PRIMARY KEY (`id`)
) ENGINE=INNODB DEFAULT CHARSET=utf8;

CREATE TABLE `tracks` (
  `id` INT(11) UNSIGNED NOT NULL AUTO_INCREMENT,
  `user_id` INT(11),
  `name` VARCHAR(255),
  PRIMARY KEY (`id`)
) ENGINE=INNODB DEFAULT CHARSET=utf8;

Commands

INSERT

 INSERT INTO `users` SET `name` = "Marie";

or multiple line style:

 INSERT INTO `users` (`id`, `name`, `lastname`) VALUES (NULL, 'Marie', 'Parker'), (NULL, 'Peter', 'Parker');

SELECT

All users:

SELECT * FROM `users` 

WHERE

WHERE allows to filter output e.g. All users with name ‘peter’:

SELECT * FROM `users` 
WHERE `name` = 'Peter'

All users whos name contains ‘e’ (‘%’ means don’t care):

SELECT * FROM `users` 
WHERE `name` LIKE '%e%'

Combine conditions:

SELECT `name`, `id` FROM `users` 
WHERE `name` = 'Peter' OR `lastname` = 'Parker')

Select only specific columns:

SELECT `name`, `id` FROM `users` 
WHERE `name` = 'Peter'

UPDATE

Combination of INSERT & WHERE syntax:

UPDATE `users` SET `lastname` = 'muller' 
WHERE `name` = 'Peter';

Best practice, use primary key:

UPDATE `users` SET `lastname` = 'muller' 
WHERE `id` = 1;

SELECT .. JOIN

select & filter data across multiple tables.

Quick and dirty:

SELECT * FROM `users`, `tracks`	
WHERE `user`.`name` = 'Peter' AND `tracks`.`user_id` = `user`.`id`

Better: (see JOIN as dot product of users x tracks)

SELECT * FROM `users`
JOIN `tracks` ON `tracks`.`user_id` = `user`.`id`
WHERE `user`.`name` = 'Peter'

Even include empty rows:

SELECT * FROM `users`
LEFT JOIN `tracks` ON `tracks`.`user_id` = `user`.`id`

==== next session ===

More on SELECT

  • Functions
  • order by
  • limit
  • group
  • indexes, constraints
  • insert select

More

  • csv import
  • mysql CLI
  • variables
  • master/slave
  • access rights
  • transaction
  • rollback

HackedTheHouse - and won 2nd Price!

March 03, 2015

Last weekend I attended HackTheHouse, a 24hrs Hackathon in Berlin, organized by Relayr and BSH - all in the Name of IoT. I teamed up with Chris(ccb23), Clemens and Roby from Italy. We had a blast!

We were given a couple of BSH Home appliances, Relayr WunderBars, Arduino, RaspberryPi, Seeedstudio Grove, Nest etc. - so all the Toys a hardware hacker only could think of.

Our idea was to bring FUN the to dull, boring household tasks.  We solve this problem by gamify cleaning duties - regular household tasks are turned into a big game, you score every-time you fulfill a task, as sooner and thoroughly, as more points you get.

After 24hrs we presented a first prototype - including a dishwasher, a smart trashcan and a smart broom. The jury was amazed and we scored the 2nd price!

https://twitter.com/relayr_cloud/status/572043608691499008

See our Hacklog, Repository, Pictures and more here:

http://www.hackster.io/hackthehouse/wgheld

Maslow's pyramid of code review

February 26, 2015

Must Read:  http://blog.d3in.org/post/111338685456/maslows-pyramid-of-code-review

Great sources for great Software Development

February 02, 2015

There was recently a call on the Softwerkskammer Emailgroup for good sources to improve ones software development skills. Here a summary of all recommendations:

* Buch: Headfirst Design Patterns

* OOSE, Vielleicht hilft Dir das hier weiter: http://www.oose.de/training/objektorientiertes-design-mit-entwurfsmustern/

  - OOSE für die Schulung ‘iSAQB’

* http://www.sigs-datacom.de/seminare/akademien/clean-code-developer.html .

* Videoreihe von Robert C. Martin: http://cleancoders.com

* Robert C. Martin - Agile Software Development, Principles, Patterns, Practices. In this book the notion of SOLID Principles is very well documented. 

* Gang of Four - Design Patterns: Elements of Reusable Object-Oriented Software

* Apprenticeship Patterns http://shop.oreilly.com/product/9780596518387.do 

* Skillsmatters  Von ndc  Oredev

* State: Tennis Kata http://garajeando.blogspot.de/2012/11/refactoring-kata-tennis-to-state-pattern.html

  - Bsp zur Tennis Kata: http://blog.ploeh.dk/2011/05/16/TennisKatawithimmutabletypesandacyclomaticcomplexityof1/

*  “refactoring to patterns” von joshua kerievsky

* Video-seite wurde ich alles von Kevlin Henney

*  “Domain Driven Design” von Eric Evans für Einsteiger ist “implementing Domain driven Design”

* “Being the Worst” podcast 

* m.feathers mit “working effectivly with legacy Code”

* “Refactoring - Improving the Design of Existing Code”. Fowler ist ein Meister des OO-Design und das Buch ist voller Beispiele. http://martinfowler.com/books/refactoring.html

IrDude - A simple android app to control my stereo via IR

December 17, 2014

I recently wrote a simple Android App which allows me to control my HarmonKardon Avr-35 stereo remotely via IR. Presenting: IrDude.

It uses the an undocumented Samsung IR API, so its unfortunately very much tight to Samsung galaxy note 10.1 tablet. Nevertheless, it does the Job for my, and is hopefully a good starting point for other to do similar. Check http://www.remotecentral.com to find hex codes to support other devices.

https://github.com/rngtng/IrDude

[image from IR PUCK]