Gitlab Step w/ Sonar

Once with running docker images inside a continuation integration pipeline, things got to a new level.

One improvement, we can see it for static code analysis, where there is no need of having a separate instance of sonar, we can have it on the fly.

So, to have a step for this scenario, the CI pipeline, will need to:

  • lunch a Sonar instance
  • maybe add some configurations/plugins
  • run static analysis with the sonar instance
  • using Sonar API to retrieve the entire coverage

The additional scripts used by this step are the following:

<code>generate-sonar-token.sh

#!/bin/bash
# Inspired from  https://gist.github.com/cjus/1047794

SONAR_IP=$1
PROP=$2
NAME=$3

function jsonval {
    temp=`echo $json | sed 's/\\\\\//\//g' | sed 's/[{}]//g' | awk -v k="text" '{n=split($0,a,","); for (i=1; i&lt;=n; i++) print a[i]}' | sed 's/\"\:\"/\|/g' | sed 's/[\,]/ /g' | sed 's/\"//g' | grep -w $PROP`
    echo ${temp##*|}
}

json=`curl -u admin:admin -d "name=$NAME" -H "Content-Type: application/x-www-form-urlencoded" -X POST http://$SONAR_IP:9000/api/user_tokens/generate`

data=`jsonval`

echo $data</code>

And `validate-sonar.sh` uses two scripts (maybe with time I will update them)

  • The bash scripts calls a specific python script with the coverage details retrieved from SONAR
<code>#!/bin/bash

SONAR_TOKEN=$1

SONAR_URL=$2

sonar=$(curl -u $SONAR_TOKEN: "$2/api/measures/component?component=d2&amp;metricKeys=coverage")

echo $sonar

exitValue=$(python ci/sonar-coverage.py $sonar 2>&amp;1)

echo $exitValue

exit $exitValue</code>

The python script `sonar-coverage.py` will validate if the coverage retrieve from sonar fulfills or not the expected value of the project

<code>import json
import sys
from decimal import Decimal

resp = json.loads(sys.argv[1])

# extract the coverage value
val = Decimal(resp['component']['measures'][0]['value'])

if val >= 80:
    print 0
else:
    print 1</code>

Scala Exchange 2018 – Follow UP

Interesting things discovered at Scala Exchange 2018.

Type-Driven Development in Practice: Cats and Akka HTTP

  1. creating a web service using Akka HTTP and Cats’ Free, IO, Writer and Reader monads.
  2. Presentation
  3. Code

Concurrency and Asynchrony in Scala

Featuring: Greg Dorrell

  • Scala Futures, Twitter Futures, Akka (both Actors and Streams), Monix, fs2, cats-effect. Monix & cats-effect have some very cool functions. Definitively need to be checked
  • Presentation

Cats Effect, Tagless Final & beyond!

To be continued ….

Git Clean-Up Commands

Sometimes you may find a git repository with a lot of branches and not all of them necessary.

Because of that, you may want to do some cleanup, so I’ve prepared a list of very useful commands:

  • Remove Merged Branches

just to see them / or number them

git branch -r --merged origin/master | grep -v "^.*master" | sed s:origin/:: | xargs -n 1 echo

git branch -r --merged origin/master | grep -v "^.*master" | sed s:origin/:: | wc -l

and to really delete those branches

git branch -r --merged origin/master | grep -v "^.*master" | sed s:origin/:: | xargs -n 1 git push origin --delete 

* these commands take care of the `master branch. If you need to keep another branch, just change the grep -v “^.*master”

  • Remove every remote (beside ‘master)

even much simple + you can use similar commands to see/number these branches

git branch -r | grep -v "^.*master" | sed s:origin/:: | xargs -n 1 git push origin --delete 

How to get the diff between all the commits that occurred on a specified timeframe in Git?

Sometimes you find out from QA department that starting with a specific date an endpoint from your application started to work a little bit slowly

if that date is last week, or I don’t know, it happen last month, the simple solution it will be to do an export of all commit between two specific days

git log -p master@{2016-06-02}..master@{2016-06-02}

git log -p master@{2016-06-02}..master@{now}

git log -p master@{1.hour.ago}..master@{now}

for more info see this page
git-log

Install ruby ‘pg’ library problems

After I’ve installed postgresql, the problem with ‘pg’ library persisted


sudo gem install pg -v '0.11.0'
.....
/System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/bin/ruby extconf.rb
checking for pg_config... yes
Using config values from /usr/local/bin/pg_config
checking for libpq-fe.h... yes
checking for libpq/libpq-fs.h... yes
checking for PQconnectdb() in -lpq... no
checking for PQconnectdb() in -llibpq... no
checking for PQconnectdb() in -lms/libpq... no
Can't find the PostgreSQL client library (libpq)
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary

The solution for this is to run the installer in this way:


sudo env ARCHFLAGS="-arch x86_64" gem install pg